While it may not be necessary to remove the green stuff, if you're removing the shaft and replacing the rubber hose and clamps between the stuffing box and shaft tube, I would clean it down to bare metal to determine true condition. While I don't think of the hose as much of a wear item, I would periodically inspect the shaft tube to see if the engine was out of alignment and the shaft was wearing on the tube. Depending on the number of hours per year, this would be something I would consider doing every ten years or when the cutlass bearing(s) were replaced.
Ted