According to legend, a dwarf named King Goldemar lived with the then owner, Neveling von Hardenberg, who advised him, warned him of enemies, drank with him, and played with him. Goldemar was invisible, revealing only his cold and thin hands. When a kitchen boy attempted to make Goldemar visible by scattering peas and ash, the consequences were terrible: The dwarf smashed the kitchen boy to pieces, roasted and boiled him, and then disappeared, uttering terrible curses.
Finds indicate that a castle existed as early as the 14th century, which was rebuilt several times over the centuries. After the noble residence was no longer inhabited, its decline began in the 18th century. Today, the late medieval castle complex is managed by the Burgfreunde Hardenstein e.V. (Hardenstein Castle Friends Association) and is very popular with tourists in the Middle Ruhr Valley due to its location directly on the Ruhr River. At the Hardenstein ruins there is also a stop for the RuhrtalBahn and the MS Schwalbe.