The first time I traveled south was 2011. Made the trip in our then current boat, a 36' Grand Banks that had a draft of about 3.5'. Made the run from City Island, NY outside to Manasquan, NJ. The next day we went outside again to Atlantic City, then the next day ran outside to Cape May, through the Cape May Canal, up Delaware Bay, into the C&D Canal to Chesapeake City for the night. Averaged about 7.5 knots and had a great/easy trip with no "depth stress."
In our current boat, 46' Grand Banks, we run outside from Sandy Hook, NJ to Atlantic City and then to Chesapeake City. Takes 2 days as the daylight is short going south. Coming north, we run from Cape May to Port Washington, NY in one shot. It's about 145 NM, but the daylight lasts longer. We average 10.5 kts.
Based on the other posts on this topic, NJ is probably doable inside but I would be stressed out the whole time. We draw 4' 2" and the ocean is a far easier trip as I don't need to stare at the depth sounder all day.