The city was probably founded on the site of a trading settlement. In 1111, a temple was built by Bolesław Krzywousty, as a votive offering for victories in Pomerania. The building was rebuilt several times, but fragments of Romanesque architecture have been preserved in the structure of today's temple. During the period of the first Piasts, the castellany of Żarnów, together with the castellanies in Sieradz, Łęczyca, Spicymierz, Rozprza, Ruda, Małogoszcz and Skrzynno, formed the province of Łęczyca.
In 1239, the castellanies located on the right bank of the Pilica were incorporated into the Duchy of Sandomierz, which was part of Lesser Poland. Żarnów received a location privilege under Środa law from Casimir the Great around 1360. Then Władysław Jagiełło granted the city Magdeburg law in 1415. From that moment on, Żarnów began to flourish, mainly due to its good communication location. Numerous privileges of subsequent rulers concerning trade spheres resulted in the city gaining a significant position in the grain trade. The end of the city's development was connected with the Swedish-Polish war.