The area around Hollenburg came to the diocese of Freising as early as 895. It held Hollenburg with short interruptions until secularization in 1805. The fortifications were mostly managed by burgraves, keepers or bishop captains. The castle was built in 1248.
In 1408, the Freising bishop Berthold von Wähingen expanded the complex, which has since been called Bertholdstein. In 1460 the castle, which was briefly occupied by the Hungarians, was handed over to the Austrians. A year later, Knight Fronauer captured Bertholdstein for Archduke Albrecht VI and built an entrenchment around the complex. A few months later, Archduke Bertholdstein ceded his power to Emperor Friedrich III.
In 1463 a lord of Vöttau conquered Hollenburg. In 1473 the castle was again in the hands of the Hungarian King Matthias Corvinus. This eventful fate had in the meantime made her a ruin. At times it was inhabited by robbers. After their expulsion, Emperor Friedrich III gave the building back to the Bishop of Freising in return for a compensation in 1478. A few years later, Hollenburg fell again to the Hungarians and was not finally returned until 1490. But Bertholdstein was never rebuilt. On Vischer's engraving from 1672 it is clearly visible as a ruin. In 1805 the Freising property in Austria was confiscated by the Imperial Cameral Fund and auctioned off in 1811. The highest bidder was the banker and later baron Johann Jakob von Geymüller. His descendants still own the ruins today.