Yes, the buffer is always on. And I did think about the pickup loading effect you describe. At least in the way I have my board setup, my first pedal is a tuner, which has a buffer. So in my mind, I'm already buffered, so I've already lost the ability to "unbuffer", unless I change pedal...