In the 13th century, the lordship of Wuustwezel and Westdoorn was given in fief by the Duke of Brabant to the Van Wilre family. In 1624, it came into the possession of the Van der Rijt family. According to a drawing (1687) by Jacob de Roy, the castle was completely surrounded by water. In 1745, the lordship came into the hands of the de Vinck family (last lord of Wuustwezel and Westdoorn).
At this moment, the castle with the goods of Wuustwezel is still in the possession of heirs of the de Vinck family (van de Vinck, via du Chastel de Howardries and Gericke van Herwijnen to Van de Werve de Schilde).