With the beginning of the rural settlement of the Erzgebirge probably a fortification was built at the end of the 12th century on a plateau above the Thierfelder Bach. This facility has been expanded and expanded over time. The first known owner of the former castle was Meinher I von Werben, who was Burggraf von Meissen in 1173 and who also founded the monastery Zell bei Aue.
From 1406, the county Hartenstein was pledged with the associated villages, forests and castles to the Schoenburger. The territory of the Schönburgischen dominions reached deep into the western and middle Erzgebirge, the territory of the upper county Hartenstein to Elterlein and the Fichtelberg had to be sold but partly in 1559 to the Saxon Electors.
In the 16th century, the castle was rebuilt into a neo-gothic style chateau. The plant had an oval floor plan and existed until the end of April 1945. In the last days of the war, SS units entrenched themselves in the Hartenstein forest, whereupon the castle was almost completely destroyed by American bombs on April 20, 1945. The surviving buildings of the outer courtyard were used for residential purposes after the war. The ruin itself served as an open-air stage.