After writing all of these, I realise that the Southeast has quite a lot of castles! Bodiam Castle was not very far from where we were staying. It is a National Trust property and we spent 2-3 hours here.
We loved going up the spiral stairs and exploring the castle. They also have rare bats here which we could hear but not see. The grounds itself are beautiful to wander around and we stopped and had lunch from the cafe. It also has the country's oldest gate but that is probably only because it is stuck!
Dogs are allowed on leads, they have shops and cafe and toilets so all you need for a perfect day out.