You need a switching 2.1 mm jack to take the battery + out of circuit when you plug in a power supply, otherwise you send voltage to the battery, which will then let out the magic smoke!💥
If the battery is not inserted when the power supply is, and vice-versa, it should be ok..