Kaunakakai Harbor is both a recreational and commercial harbor that extends about 1,900 feet off the shore. Designed in the 1930's, this harbor ended up costing $105,000 and took 4 years to complete. Don't be fooled, $105,000 in 1934 adjusted for inflation is a cool $2.4mm in todays purchasing power (which is still a great value).
There is a small marked swimming area on the west side of the wharf, but it is not used much (win-win). You'll have to climb down some boulders to get to it.