Read other letters about this article
You know exactly what dissatisfies you about your current living situation. More importantly, you know some things that would improve how you feel during your daily life. If I were you, I work toward a more palatable living situation as long as your wife can be happy too.
The danger of trying to live with such a situation is that you will adjust to it. You will no longer try to do the things that were once convenient to enjoy. You may lose the qualities you like about yourself, feel bitter and perhaps eventually forget what you wanted to change in the first place.
Some people are cut out for a focus on their own homes, gardening, driving everywhere, etc. Some people live for the museum's late nights and the local,non-diner restaurant that serves until 4:00 am, just in case. If you're the latter, don't try to turn yourself into the former just because it seems like the sensible thing to do.