I first had the LDR/LED combo pointing at each other in heatshrink (
à la "homemade vactrol") but opposite to you I was getting way too much volume and it would distort at higher VOL settings
View attachment 14756
I ended up placing them the "regular" way (without any covering) I've seen in other builds, and it ended up sounding nicely, with a nice unity volume with just a bit of boost when pushed.
View attachment 14757
So I think you should try different LDR/LED placement to see if that could solve your issue.