Its foundation dates back to 1072, when the consul of Fondi Litfredo donated the territory to the Benedictine Monastery of Montecassino.
The building had, from the fourteenth century, a purely defensive and sighting purpose, as can be deduced not only from the strategic position of the place, raised and from which you can enjoy an excellent view of the plain of Fondi and the adjacent mountains, but also from the structure:
The plan, typically medieval, is square and with circular towers placed at the vertices of the sides and has numerous loopholes along the entire perimeter and is spread over 4 levels: ground floor, first floor, second floor and attic.
Over time, the castle underwent numerous changes, not least that of the eighteenth century, which with the addition of a rectangular body with a gabled roof, gave the building its current appearance.
The interior has maintained its initial appearance almost unchanged over time, with the addition between 1400 and 1500 of vaulted roofs with nails (sails) in correspondence with the openings. This intervention was carried out by the Caetani family, who in that period developed an aesthetic sense found in many other buildings in the city.
Legend has it that the countess Giulia Gonzaga and a lover often met inside the fortress: it is said that she accidentally discovered a secret passage that led from the Baronial Castle to Monte Vago, from which she descended to reach the place.