The castle of Torre was built in the Middle Ages where the complex of the large Roman villa once stood. The oldest part of the fortification is represented by a square-based tower, characterized by mighty walls, probably built in the second half of the 13th century by the Patriarch of Aquileia, who initially entrusted it to the ancient House of the Lords of Prata. Long the object of contention between the nobles of Porcia, the Count of Gorizia and the Carraresi, in 1391 the Patriarch of Aquileia, Giovanni di Moravia, granted the castle of Torre to Giovannino di Ragogna.
After the first half of the 15th century, with the Venetian conquest of Friuli, the castle was transformed into a noble residence and was involved in a vast program of architectural and artistic interventions including the fresco decoration with the Annunciation on the ground floor, attributed to Gianfrancesco da Tolmezzo (1450-1511). The staircase leading to the main floor and the loggia were probably built during the second half of the seventeenth century. Between the end of the seventeenth century and the beginning of the eighteenth century, the fresco depicting the main protagonists of the defense of Vienna from the siege of the Turks in 1683 was created in the vault of the ground floor of the tower-keep, probably to commemorate the participation of a member of the Ragogna family in the clash.
Various additions and structural modifications were made to the building during the eighteenth and nineteenth centuries. At the end of the nineteenth century, in the southern sector of the castle, the old loggia building was demolished and a new building was built, which was placed against the northern wall of the main tower. In 1926, the building was replaced with the one that still exists today.
In 1970, the Count of Ragogna died, leaving the Castello di Torre as an inheritance to the Friuli Venezia Giulia Region; in the end, after various testamentary events, the building came to the Municipality of Pordenone. After a careful restoration that took place in the early 2000s, since 2006 it has been home to the Archaeological Museum of Western Friuli.