It’s been years since I played through a friends Blues Jr., and I remember liking it quite a bit—but I don’t think it’s neutral. In terms of overall tonality, yes. But the EL84 amps I know (and I’ve owned one for about 20 years now) seem to have a pretty distinct sound, in terms of what the cleans sound like (“chimey” is the usual term), and how it reacts to stronger signals. It’s a sound I love, especially with P90s.
I don’t know much about how different companies’ EL84 vary, so can’t help you much there. You could try changing your V1 preamp tube, to something with a bit less gain, that I think will give you a bit more clean headroom. A 5751 would be just a bit less gain, and they’re typically a good bit quieter than 12AX7s. A 12AT7 is also a common substitute for V1.
The other option is changing the speaker, and I’d recommend plugging in other cabinets and seeing what happens, if you have friends that are willing to experiment with you. Considering how expensive NOS tubes are getting, a speaker trade might be less expensive. (Not that I’m suggesting you go with NOS tubes for these tryouts.) A more efficient speaker would play louder with less strain on the amp, for instance. Or a British style speaker would plump the mids a bit more.
I‘m pretty sure the solid state rectifier is not a chargeable part (without surgery) on the Blues Jr. This would be one more area of potential experimentation. But I think that’s not really doable, and I don’t know if more sag would help matters anyway. (My EL84 amp is also solid state rectified, but there is a tight and loose switch on it. It pretty much has stayed on tight for me.)