Already in 1628 the Vierhufengut Roßthal was handed over as a fief by the evangelical provost of Meißen to the electoral valet Ullmann and declared a manor. In 1657, Roßthal Castle was built as a mansion for the owner Alexander von Krahe. In 1736 the manor came into the possession of the von Nimptsch family.
In 1852 Carl Friedrich August Krebs, Baron von Burgk, took over the manor. His family remained the owners of the manor until 1945. From 1858 to 1859, Karl Moritz Haenel rebuilt Roßthal Castle and the eight-sided tower in the German Renaissance style and provided it with two new upper floors. After the Second World War, the administration of the state-owned goods of Saxony was temporarily housed in the castle. The publicly owned Gut Gartenbau Pesterwitz also took over the agricultural land and set up a vocational school. This still exists today as an agricultural science-oriented grammar school.