Around 1240 a first castle with a chapel was built; the first documentary mentions Raben von Kirchberg, who came from the von Sulz family (Burg Sulz), as the builder in 1265. The Lords of Kirchberg were probably servants of the Counts of Flügelau; they are attested until 1464. The plants were expanded to 1400. When Kirchberg came into the possession of the imperial cities of Rothenburg, Dinkelsbühl and Hall in 1398 and was itself expanded into a city, the castle complex was included in the city fortifications.
Ludwig Kasimir von Hohenlohe bought the castle back in 1562. Extensive changes took place under Kasimir and his sons: In the years 1590 to 1597 the medieval castle was transformed into a Renaissance castle. The hall construction, which was erected on the site of the first castle, two transepts and connecting passages, i.e. a four-winged structure in total, which was provided with bastions towards the city, date from this time. Only the neck moat, some foundation walls and a tower with key notches on the north-east side, probably built around 1500, remained from the medieval structure, which may have replaced an older tower. In 1650 Count Joachim Albrecht von Hohenlohe came into possession of Kirchberg and made the castle his residence. In 1675 Kirchberg came back into the hands of Langenburg, and from 1699 to 1861 it was the seat of the Hohenlohe-Kirchberg line.
Under Carl August von Hohenlohe, Leopoldo Retti planned the conversion into a residential palace, which took place between 1738 and 1745. The widow's building, which became the town hall in 1800, the royal stables, a well and a sentry box as well as the moat including the bridge, coat of arms, envy's head and sentry box were added to the old building stock.