I think you've discovered the main reason electrolytics are used. they generally take up less space for their capacitance value. You might want to use an audio probe to track down at what point the noise enters the signal path.
Also according to this there are technical reasons for using them...