The name "Domherenhuis" is derived from the first and also most famous resident of the complex: Baron Lambert de Villenfagne (1753-1822). He was a canon of the chapter of the Saint-Lambert Cathedral in Liège. He was born in 1753 as the second son of the lord of the castle Jean Ignace de Villenfagne. He built the house in 1795 to live there with his unmarried sister, Marie-Françoise de Villenfagne. During the French rule, Lambert de Villenfagne was mayor of Zolder. Over time, the house was usually inhabited by widows or by unmarried sons and daughters of the lord of the castle. It was two ladies, Miss Leen (1887-1966) and her mother Baroness Dowager Leon de Villenfagne de Vogelsanck (1864-1947), who left their mark on this small castle in the twentieth century.
In 1938, they built the eastern wing with a caretaker's house, guest rooms and a house chapel. After the death of "Miss Leen" in 1966, the property first came into the hands of the municipality, later of the Limburg Landscape Foundation and currently a B&B is run there with 4 guest rooms. The entire complex (mansion and park) was protected as a monument on 5 June 2003. The park is open to the public.