The ruins are on a headland at the southeast end of Scalloway, opposite the harbour. It has an L-shaped floor plan and is still partly preserved up to the full height of four floors. The roof, on the other hand, collapsed, as did the intermediate floors. The masonry is of rubble stone, while the windows are framed by blocks of sandstone, which were also used as quoins at the edges of the building. The corners of the building are reinforced with bay towers with decorated corbels. Below the ruin is a vault that once housed the cellar and kitchen.