The castle was founded in 1356 by Charles IV. (after him it was also named - Karlsberg). There were three reasons for establishing this guard castle. The primary necessity was the need to secure the land border with neighboring Bavaria. The second reason was the monarch's effort to protect the gold-bearing area of the Kašperské Hory. Last but not least, Charles IV. he considered it necessary to ensure security on the newly established commercial road called the Golden Path. It connected Bohemia with Bavaria and then with the developed regions of Western Europe. One of its branches led through the Kašperské Hory.
Kašperk quickly became an important support of royal power in southern Bohemia. Although Kašperk was a royal castle, it often served as a pawn, and therefore was ruled by pawnbrokers; the second archbishop of Prague, Jan Očko from Vlašimi, is among the well-known. In the second half of the 15th century, the Kašperk castle passed into the possession of the important Šternberk family from Šternberk. As the end of the 16th century approached, the castle gradually lost its original significance, funds were no longer available for its regular maintenance, and the castle fell into disrepair. The Royal Chamber began to sell off the Kasper estate. Finally, in 1616, the already derelict Kašperk Castle was bought by the nearby City of Kašperské Hory, which is still the owner and operator of Kašperk Castle today. The yield of Ferdinand III. fortunately, the demolition of the castles in 1655 was not carried out by the Kašperskohorský, who used the ruins as a source of building material. In the 20th century, it underwent extensive reconstruction that had already begun before the Second World War.