The castle was built between 1728 and 1731 for the Electorate of the Palatinate Chancellor Baron Jakob Tillmann von Hallberg from the noble Hallberg family. The local lord converted an entire wing into a church, opened it as a place of worship for the Catholics of the area and in 1742 founded a Catholic parish in Fußgönheim for the first time since the Reformation. He had a high altar painting made for the church by Giovanni Antonio Pellegrini, who also painted the Mannheim castle. According to local tradition, recorded by the local pastor Stephan Lederer, Jakob Tillmann von Hallberg is depicted on it as a figure; the only known depiction of him. According to the local chronicle, the castle church was completed in 1741. It is the current northwest wing of the entire complex, with a roof turret. Its baroque interior is completely preserved. From 1788 to 1792, Heinrich Theodor von Hallberg (1725–1792), the Electorate of the Palatinate and Bavaria's ambassador in Vienna, was the lord of the castle.
In 1815, the Hallberg Castle was auctioned off and subsequently changed hands several times. It was used as a shop, barn and warehouse. Among other things, the castle housed a cigar factory, a prisoner of war camp and a Raiffeisen magazine.
In 1972, the castle was purchased by the Catholic parish of Fußgönheim and restored at great expense. A large church room was set up in the formerly secular central section and partially furnished with baroque furnishings. This is today's Catholic church of St. Jakobus. The historic castle church now only forms its side chapel.
The district of Ludwigshafen transferred part of its property to the Catholic parish free of charge. In 1983 and 1984, the former castle garden was restored and made accessible to the public.