Well Browny, I don't often say this however... with your limited planned usage of a vessel I do believe you would be FAR better off chartering, even considering the not insignificant costs involved.
As my boat sits here, with me living aboard her, SOMETHING is always breaking. Always. I have a modicum of knowledge as to what the problems are, and how to go about fixing them. Most recent was a new alternator and rectifier, a $400 (I bought a spare for the parts inventory) expense.
If you are not aboard things will still go wrong. They will require repairs. Little things can become expensive. Even stupid stuff -- like on Christmas I was at a neighbor's boat. I smelled electrical and was able to sniff out the problem. One of his cell phone chargers was overheating. Had he been ashore, that could have been Very Serious.
At this stage in your life with kidlets, if you want to have that week long vacation aboard a vessel, rent one. By the time yours are a bit older you will have experienced many different boat layouts. You'll know what you like, and more importantly, those things that are AWFUL for you.
I love galley up. My neighbor with the Hatt50 loves galley down. We each have the boat of our dreams.
In your rental/chartering times, do look into the engine room. If you cannot easily reach the impeller to swap it out, that routine maintenance most likely will not have been done regularly. Can you get to the batteries to check them? Thru hulls accessible? If you're going to have to fix it, make sure it's easy to do so.
Anyway, as much as I LOVED growing up aboard a boat (born and raised aboard a 40'er) this was our home. For you, if I understood it correctly, you want a boat to use but not live aboard. Unless you can pay for someone to take care of the routine issues, you're simply buying another project. I suspect you have plenty of those on your plate at present.
P.S. - Good luck with the newest. I'm going to be Grand again in a couple months. Yeah for babies!!!