Good point, but from the toxicological point of view, the danger excerted by organic lead compounds is way higher.
That's why we have unleaded fuel for decades.
I see the most danger in small batteries that could be swallowed accidentally. Same for those tiny but strong Neodym magnets.
Simply...