The castle was built in the second half of the 14th century as the most southwestern stronghold of Mazovia, situated in the marshy forks of the Rawka and Rylka rivers, on an important trade route from Pomerania to Ruthenia.
Thanks to King Sigismund Augustus, from 1569, the Rawka castle was used to deposit and store the quartz tax (established in 1563), collected from the starosty and crown lands, and intended for the Crown Army.
From the 17th century, the castle fell into ruin. The destruction was initiated by the Swedish army during the "Deluge". Partial reconstruction in the second half of the 18th century by the Rawka starosta Franciszek Lanckoroński did not stop the process of decline. Its deliberate destruction, initiated during the Polish-Russian war at the end of the 18th century, was continued by the Prussians, who ruled the city at the turn of the 18th and 19th centuries. After World War II, conservation work began. In the years 1954-1958 the outline of the foundations was uncovered, a section of the curtain walls and the octagonal tower, which houses archaeological and historical exhibitions, were reconstructed.