The moated castle was the center of the Millen of Milina seigneury, which was a fief of the County of Loon. She was first mentioned in 1143. The oldest known lord was Walter van Millen (Walterus de Milne), who was seneschal of Lodewijk II van Loon. The lordship was successively owned by the families de Melin, Proest (1369), van Guygoven (1456), van Berloz (1479), van Houthem (1531), de Fléron (1625) and d'Hemricourt (1741).
The castle was first mentioned in 1365. The original castle was converted into a moated castle in the 14th century. It then consisted of the older keep that was expanded into a square moated castle with a gate tower and two side towers. In the first half of the 17th century, the fortress, which had fallen into ruins, was converted into a castle. It became a comfortable nobleman's residence without military significance. From the 18th century, the castle was no longer permanently inhabited and fell into disrepair. In 1811 it was bought by Baron de Blisia, after which much of it was demolished, including the keep and the tower on the southwest side. Only the east and west wings remained and they have been used as farms ever since.
In 1972, the remains of the castle were purchased by the municipality and restored. The 14th-century left tower in marlstone has a Gothic vault. It is part of the east wing, which may date back to the 12th century.