Dan, I'm familiar with Rye as it's not too far from my sister's place south of Ashford. Wonderful little town to walk around with a mix of narrow streets with small "shoppes", pubs, etc as well as the various river harbors. The curved commercial harbor on the main river to the east (east of the bowling green, top right in the photo) is worth a visit. The other branch of the river that leads into town is mostly recreational boats. Be sure to visit the waterfronts at both low and high tides. The difference is stark and many of the boats end up high and dry on the mud at low tides. Not something most of us US boaters are used to (maybe Maine?), but quite common in UK coastal harbors.
Anyway, put Rye on your list for at least a couple of days and nights. It can get a bit full of bused in tourists during the height of summer, but that's now the case with almost any quaint photogenic village in England. By staying a night or two, you get to enjoy the relative quiet of early mornings and evenings before and after the buses.
BTW, I think I found McCartney's boat. I have it marked in this screenshot of GoogleEarth. That the one you mentioned?
Further "sleuthing" and I'm now sure that's the boat. Check the 2nd and 3rd photos. Yeah, too much time on my hands.