The transistor you replaced, were they blown?
Years ago I had that happen to my unit. One transistor blew shorting and operating the unit to turn hard starboard immediately it got power.
I replaced them although to test I had to cut the caps. and then replace those also.
I realized after reading the instruction/installation manual while testing after the repair that the hydraulic pump was never set up properly. THe pump oil delivery rate was set at maximum per revolution. The rudder movement was FAST, too fast which overloaded the motor never allowing it to get out of the inrush current state.
Once I dealt with the pump delivery rate by backing off the swash plate the rudder actually was quicker to respond since the motor actually was running faster. Even though each revolution delivered less oil the motor could turn faster revs so delivered more oil in the same time with less strain.
My unit had been installed shortly before the P.O. sold the boat to us so I suspect he just didn't quite finish all the tuning.
So if you are blowing transistors , the main power transistors, then check the pump delivery rate. On mine it was simply a timing exercise to determine the oil delivery rate. Too few seconds meant the pump was trying to deliver too much oil overloading/straining the motor.
If your unit is immediately turning one direction or the other when it gets power at turn on , then maybe another transistor has blown from the above problem. Recheck and replace them including the capacitors and then check into the pump delivery rate.