Standing above the car park, just off the main A40 road which runs through the town, are the remains of Llandovery Castle - a small but effective stronghold in its day. Dating from 1116, when it was known as the castle of Cantref Bychan, it saw action on several occasions and was the target of Owain Glandwr in the 1400s.