I agree with you.
As our society's vision regarding privacy rights expands, all gender bathrooms and locker room areas that provide individuals privacy from others including those of the same gender are essential.
Competing with that ideological belief are pragmatic issues, sanitary issues and ideological challenges.
The pragmatic problem is simply having the capacity/space to offer "private" all gender bathrooms. It does add a significant costs and design challenges to building owners. But I think many establishments have gone that way in recent years and over time I expect that problem will solve itself with a little push in that direction with revised building codes.
Sanitation is also a problem. As many women I know have said, "men are pigs" and they don't want men using the same bathroom. Men they claim (and rightfully to, if my observations are correct) are too lazy to lift seats and end up peeing all over them. But that can be solved with technology, I installed a toilet that does so automatically, when you stand in front of it, but they are expensive. Now we just need a toilet that polices the area around the toilet to clean-up "misses". One of my favorite bathroom signs are "We aim to please, so please aim."
Your points regarding caregivers of a different sex /gender than their patients are another good reason for all gender bathrooms.
As the saying goes, all of the above issues can be solved with money, though the "sanitation" issue may require a new toilet design that prevents people standing in front of them and peeing, but some bright engineer can likely overcome that issue... perhaps a toilet that "pees" back at you if you stand in front of it (just kidding).
The third issue is ideological and not easily solved. Many transgender people are making their right to "share" a bathroom with biological females an ideological issue to assert their gender identity. While they are quick to point out, they should not have to share a bathroom with someone of the same biological sex as they are.
They don't seem to see the hypocrisy, that just as they are demanding privacy from people they see as a different gender than they are, that people who see themselves as a different biological sex may not want to share their privacy with them.
In the minds of many trans people, biological women not willing to share their bathrooms and locker rooms is bigotry. But the same logic applies to transwomen (biological men) and men not sharing the same bathrooms and locker rooms. That logic is simple, does society get to decide who can violate your privacy or should we attempt to allow individuals to decide who can invade their privacy.
The concept is not particularly new and has been applied to sex. Many colleges require men and women to openly express consent before having sex. Sex is normally an intensely personal and private act shared with other people through deliberate consent. Anything less border or crosses the line into rape.
So, the concept of individual consent is not new, and we need to extend it to locker rooms and bathrooms recognizing the pragmatic challenges and technical ones may require some time before we achieve adequate access to all gender bathrooms and how to handle the transitional period.
At the societal level, our society was built around the individual pursuit of happiness which our courts have defined as conferring privacy rights to individuals. So, we need to end the ideological debate and move on to the more practical challenges of providing each person the choice of who they want to share their bathroom and locker room with, if anyone at all in some instances.
