In 1542, the inhabitants of Pegli and the neighbouring communities left their home on the Ligurian coast in the wake of the Lomellini, a powerful trading family from Pegli and Genoa, and settled on the island of Tabarca off the coast of Tunisia, where they dived for coral until 1735. When coral mining and thus the tribute to the Lomellini continued to decline and was no longer profitable for all sides, and the conflicts with the Muslims intensified, King Charles Emmanuel III of Savoy, on the initiative of the Piedmontese architect Augusto de la Vallée, made the offer to settle the still uninhabited island of San Pietro. A part of the Tabarchini under the leadership of Agostino Tagliafico accepted the offer and settled on the island off Sardinia in 1738. In honour of the king, they named the new place Carloforte ("Charles the Strong"). Over time, other emigrant families arrived directly from Liguria and helped to strengthen the small, emerging community.
In 1798, Carloforte was attacked by pirates and around 900 inhabitants were captured and held as slaves in Tunis; one of them became the mother of Bey Ahmad I al-Husain. After five years, they were finally able to be ransomed. The parts of the old city wall and the fortress that are still preserved today are evidence of this barbarian invasion.