The Renaissance manor palace built by the Shemets was destroyed by the Swedes during the Northern War in 1704.
The center of the manor was moved to its current location. 19th century a classicist one-story palace building was built, with a two-story classicist rectangular annex decorated with a four-column portico added to its western side. The Vepriai manor was ruled by the Kezgailas, the Kosakovskys, the Oginskis, the Shemetos, and the Tyzenhauses. The last manor was managed by Marijonas Vandalinas Pliateris.