The castle was originally the centre of the county of Kessel, first mentioned in the tenth century, when Emperor Otto I of the Holy Roman Empire appointed his cousin and vassal Count Ansfried as Count of Kessel, which gave him the right to mint coins for Kessel and levy tolls on the Meuse. This Ansfried later, as Bishop of Utrecht, founded the abbey of Thorn. An illegitimate child of this Ansfried, named Balderik, became his successor as Count of Kessel. Balderik was succeeded by Hendrik I of Kessel. The descendants of this Hendrik remained as residents of the castle until 1279, when it was sold to Reinoud I of Gelre. Since Reinoud did not want to live in the castle himself, he appointed a certain Godfried van Berck as administrator, who promptly appointed himself Lord of Kessel. In 1541 this family died out and a cousin, Caspar van Merwijck, inherited the castle. In 1597 the castle was set on fire by Dutch troops and then rebuilt with a three-storey brick building. The van Merwijck family was important and had contacts with, among others, general Maarten van Rossum, emperor Charles V, prince Maurits and archduke Albert of Austria. In 1779 the van Merwijck family died out and Caspar baron van Keverberg inherited the entire estate, including Kessel castle, as a nephew of the family. It was the first time that the name Keverberg appeared in the history of the castle. This Caspar met his end by drowning in the Maas and since then it is rumoured that he regularly haunts the Maas.
The last noble resident of Kessel castle was Frederik Hendrik Karel de Keverberg de Kessel, the son of Karel Lodewijk van Keverberg van Kessel. This "baron Frits" died a lonely death in 1876, abandoned by his wife and daughter. The castle became a home for barge children and then the heirs of the Keverberg family sold it in 1903 to the congregation of the Sisters of Divine Providence. These sisters set up the castle as a convent and turned it into a girls' boarding school, the St. Aloysiusgesticht.