Kittlitz was probably the site of a Slavic fortification as early as the 10th century. In the course of the German settlement of Upper Lusatia, a burgward was set up here, which developed into the political and economic center of the area. In the 12th century, a fortified courtyard was built as the core of a small moated castle east of what is now Weißenberger Straße. West of it, roughly where the castle is today, was the Niederkittlitz Manor.
Until 1368, the owners of both estates were the Lords of Kittlitz, who are considered to be the oldest noble family in Upper Lusatia and had various tax privileges and their own supreme jurisdiction. In the middle of the 14th century, control of these manors passed to the Nostitz family and the von Gussigk family. At times, both manors were owned by Johann von Gussigk before they were divided again into Oberkittlitz and Niederkittlitz in 1527. Until 1704 (Oberkittlitz) and 1750 (Niederkittlitz), these belonged to the von Gersdorff family, but were then sold to Karl Gotthelf von Hund and Altengrotkau. With this sale, the manor finally shifted to the Niederkittlitz manor.
The new owners had the existing manor house remodeled to suit their needs and made Kittlitz Castle into a center of Freemasonry. Karl Gotthelf von Hund and Altengrotkau was one of the leading German Freemasons and from 1743 he was Templar and Army Master of the province of Lower Germany, later head of all German Freemason lodges. Financial reasons forced him to sell his Kittlitz property in 1769. The purchaser was Countess Helena Isabella di Salmour, née Countess Lubieńska, who had had to sell the Zabeltitz manor to the House of Wettin the year before. At the same time, she also acquired the neighboring Undignity Castle. She was the widow of Count Giuseppe Antonio Gabaleone di Salmour from Piedmont, who had received Zabeltitz from his uncle Count Joseph Anton Gabaleon von Wackerbarth-Salmour as a wedding present. Owners changed again in the 19th century. In 1878 the owner E. F. W. Fickler had the palace park laid out. In 1909 Hugo Freiherr von Salza und Lichtenau acquired the manor.