Southwest Harbor is a town in Hancock County, Maine, United States. Located on Mount Desert Island, the population was 1,756 at the 2020 census. The municipality contains within it the villages of Southwest Harbor, Manset, Seawall, Wonderland, and Pemetic Hills.