I like Ocean Marine Yacht Center in down town Portsmouth. Tide Water Marina is also fairly close. Probably your best bets. There is a grocery store within walking distance from Norfolk's Waterside Marina. McArthur Center is within walking distance too if you need a shopping center.
Ocean Marine usually has a fairly competitive fuel price. Maybe not the very lowest.
There used to be a large grocery store, don't recall the name, that would send a van to Waterside Marina to pick you up and take you back after shopping. The marina staff should be able to help you.