The Village of Dering Harbor was incorporated in 1917 after many successful years as a resort community formed in 1873 as Shelter Island Park.

The centerpiece of the community was the great Manhanset House that opened in 1874 serving the well-to-do of the late 19th century during the summer season. As the hotel was nestled overlooking a protected harbor with access to both Long Island Sound and Block Island Sound with LIRR service from Brooklyn, the community attracted many yachting families from the New York Yacht Club, Atlantic Yacht Club, and other clubs from New York Harbor to Long Island Sound to Bristol Rhode Island. One of the many legacies is the large fleet of Herreshoff 12 1/2 footers located at the Shelter Island Yacht Club not to mention the burgees of some of the Club's earliest members displayed as you find your way to the head.

Many of the original cottages built in Dering Harbor were either private homes or houses owned and operated by the hotel. The styles range from period Victorian, Federal Style, Classic Revival to Queen Anne. Two buildings in use today were originally carriage houses. The Village also hosts the third oldest golf course in the Unites States, incorporated as the Manhanset links in 1896 now known as Gardiner's Bay Country Club.

The Village of Dering Harbor has frequently been called the smallest incorporated village in New York State, perhaps in the whole country. The late Ian Brownlie, former mayor, once spoke of his bailiwick as "the smallest political unit in the United States" and has not, so far, been contradicted. Ghost towns of smaller dimensions and zero populations may perhaps be found in the once-golden west but Dering Harbor is decisively not that sort of community. Skeletons in closets, maybe; ghosts not yet!