Once to Southern NJ, water and fuel are available into December...just not every dock or every marina...call ahead. Washing a boat may be an issue, but for a couple days I never worried about it.
Nights are cold so aux heat in some form at night will make it better than camping. If at a marina, even a single space heater in a sleeping cabin will work.
During the day, driving from the inside can be pleasant on sunny, calm days. If windy and cloudy....can be cold unless dressed and moving around to be warm.
I left NJ for Florida each year from 2012 to 2019 in early to mid Dec. Via the Chesapeake as the ocean route would have only worked with a much faster boat to meet weather windows.
Even the Delaware kicked butt a few trips. Almost frozen in within Baltimore harbor one early December.