The town is noted for Portumna Castle and for the lords that lived there. On the grounds of the forest park, Portumna Castle and gardens will provide hours of pleasure. The castle on which considerable restoration work has been carried out by the Office of Public Works, dates back to the 17th century and was the seat of the Earl of Clanrickarde.The castle is actually a semi-fortified house and was built before 1618 by Richard Burke or de Burgo, the 4th Earl of Clanricarde. It was the main seat of the de Burgo family for over 200 years, until it was gutted by fire in 1826.
The conservation works on other parts of the castle are ongoing. To the north of the castle are formal, geometrically laid out gardens. The castle also includes the restored 17th century walled kitchen garden. There is an old Abbey within the Park which dates back to the 15th century.
As with many historic buildings a number of local legends have grown up around the castle. One local legend goes that a child fell out of the upper windows. An Irish wolf hound raced over, broke the child's fall and saved the child. A marker stone now rests on the site. The bones of the dog, called Fury according to a marker stone erected at the time of his death, have been found during excavations, and are now on display in the castle itself.