You have 1M at the Input to Ground, try 100K resistor from Output to Ground as there is no pulldown resistor there!
Just try 100K from Output Jack Pad to ground Pad on Output jack.
You shouldn't need to add an output pulldown resistor, the volume pot acts as a pulldown. Anywhere from 0 - 100KΩ depending on the setting of the volume pot.
If you can easily disconnect your LED that would eliminate in-rush current as a problem.
Aside from that you're pretty much only left with a noisy footswitch or leaky C1 / C2.
This is where I was heading too, this circuit has been around forever & doesn't have a pulldown resistor on the output.
I believe you have a dodgy footswitch!
Whelp, it’s not lacking pulldown resistors, it’s not the capacitors, it’s not the switch. I think I actually have a grounding issues somewhere/somehow.