The Senne has always been particularly rich in game. Since the Lippe sovereigns had a great interest in hunting, they built their hunting lodge here on the edge of the Senne. In addition, the state claim in this remote part of Lippe "behind the mountains of the Teutoburg Forest" should be asserted and substantiated.
The castle was built from a dairy farm that had existed since the year 850. 1599 was ruled by Simon the VI. commissioned conversion of the complex into a hunting lodge has been completed. The impressive overall complex of hunting lodge and dairy measured a total of 92 meters in width and 153 meters in length.
The complex was surrounded by a moat and could only be reached via a drawbridge, which was intended to ensure that it could be defended in the otherwise unprotected location on the Senne plain. In addition, there were loopholes in the walls. The southern corners of the hunting lodge complex were secured by two round towers and farm buildings stood along the entire west side. The moat is now silted up or filled in, except for the southern section.
The hunting lodge was used for the stately hunting and festival parties and as travel accommodation. In addition, negotiations between the County of Lippe and the Bishopric of Paderborn were held here in a convenient location halfway. During the Thirty Years' War, both Protestant and Catholic military leaders were present as guests. Nevertheless, the castle and the dairy were plundered twice during the war years 1633/34. 36 cows were slaughtered and 100 horses stolen. After the end of the war, however, Count Hermann Adolf zur Lippe had the hunting lodge expanded on a large scale due to the rich stock of game. Instead of the old half-timbered building, a large stone castle was built.
In the 18th century, the Oesterholz hunting lodge lost its importance. The Lopshorn hunting lodge near Augustdorf increasingly took its place. Interest in hunting also declined. Due to the neglect, the complex fell into disrepair, so that today only remnants remain.
While the typical Weser Renaissance half-timbered building to the left of the archway has been preserved, the actual stone castle building to the right of it was removed in 1775. The building materials were used in the construction of the "Zur Rose" spa house in Meinberg. The dairy buildings also disappeared later. The other buildings were then leased out as a farm and used as a head forester's office. Today the old people's home is housed in the former "count's room" to the left of the gate.