Modbus over STP should be fine. Interestingly, I've been running experiments with Modbus vs noise vs baud. My last experiment was in a very high 60Hz field environment, and showed quite a bit of Modbus frame errors, diminishing when I reduced the rate down to 4800 (from 119,2). BUT, this was on a 345kV air system; your mileage will definitely vary for the better in a boat or factory! This was a place where you received a constant shock just touching any grounded surface. A constant shock, not ESD.
My cabling was indiv. shielded TP with overall shield, but only attached on one end. More work to follow there.
Now, I am chasing the viability of CAN, which is essentially the same physical layer, with built-in error correction, vs the primitive Modbus protocol. Anyway, I am not optimistic on CAN even at 250kb, much less the latest 1Mb flavor. Unless I go with fiber. Ok, i'm now rambling.
Yes, you really should do a block diagram, then schematic, data flow diagram, and algo strategy too. But, I suspect you already know this. Is there any plan for commercialization? or just a one off?