A wiring schematic would be helpful but in lew of that a good photo or two would go a long way, if you have capability to do that where you are. With nothing, everyone is guessing, so here is mine. I'm guessing that your lighting is low voltage, either 12v or 24v. The transformer in question drops the voltage from 120v/240v to 12v or 24v. If your lights are not LED they would run fine on the low voltage AC. If they are LED they would require DC voltage. To get that from shorepower, your low voltage transformer output would then run thru a diode bridge to convert it from AC to pulsing DC. This would run an LED although it would pulse at 120 hz. This would probably not be noticeable, but I have never tried it. If anything else runs off this power, it would likely need a filter )a large capacitor) to convert the pulsing DC to steady DC. You would likely have a power transfer relay to disconnect from the battery source to the shore source if it is available. Do you or did you hear a clicking sound when you connect to shore power?
I suggest you start at the transformer. Do you have 120v (or 240v) AC accross the high voltage leads? If so do you have 12v or 24v AC accross the low voltage leads? If you have high voltage but not low voltage the problem may be the transformer. To confirm this, with all voltage 'off' check the continuity of the high voltage leads with your ohm meter. You shouldn't have more than a few ohms resistance. Do the same with the low voltage leads expecting the same result. If you dont have high voltage, the problem is upstream on the supply side. work your way back checking checking voltage. When you find it, the problem will be between there and next place downstream that doesn't have voltage. The transfer relay, if you have one, would disconnect the battery voltage source and connect the shore power source. I doubt that this is the problem as the battery source would be the normally closed contacts. Shore power would energize a coil which pulls the contacts from the normally closed. battery source to the normally open shore shore source.
This whole scenario seems unlikely though as I would think that low voltage lights would be hooked to the battery only. When hooked to shore power, your automatic charger would carry the lighting load.
There is my two cents worth, now it is time to head to the boat to put on a second coat of bottom paint.