Turn off the fence to add a deer deterrent called Deer Away. It's an egg-based product, and deer don't like the way it smells. Spray cloth strips with the Deer Away and tie them to the fence at 10' intervals. The smell will last for several months before you need to reapply. The combination of the electric fence and the deer repellant sets up a double-negative system. If the deer ignore the smell and approach the fence, they'll be startled when they touch it. Double-negative strategies work well against determined pests.
You can also install a motion-triggered water sprinkler. To install the sprayer, push the post into the ground; then point the sensor away from the garden. This way approaching deer will be squirted (and with any luck it won't spray you when you work in the garden). Attach a garden hose and turn on the water. The sprinkler senses movement near the garden and shoots a strong stream of water around the area to chase pests away.
If pests are a real problem in your area, you can also try motion-activated strobe lights; just make sure you tell the neighbors first. A rubber snake and a fake owl should keep out the birds and maybe even some mice. You have to resort to creative and varied ways to keep all types of pests away from the garden.