Several ghostly residents reportedly haunt The Drover's Inn. These include "George", a former regular whose ashes were for some time kept next to the fireplace, and "Angus the Drover", who was murdered nearby in the 1750s and wanders the inn late at night, screaming in pain.

Overnight guests have also seen a spectral family gathered at the foot of the bed, one of them, a small boy, waving avidly. In the 19th century, when a young girl drowned near the inn, her lifeless body was laid out in room six. Many guests who have slept there have woken in the night to feel a wet, icy-cold body next to them in bed