You could say the exact same thing about people being fired for racism or homophobia and being "harassed" (I wouldn't use that word for fighting bigotry, personally) for being racist or homophobic. Is that evidence that racism should be allowed everywhere?

> The reason was that heterosexual males (who called themselves women) had attended previous such events, despite lesbian women of course not being interested in them, and were being creepy.

There is literally no evidence of this being a common thing, while there is quite a bit of evidence of nutty anti-trans people accusing cis women of being trans because they "looked manly", even though they were just women born looking like that. Meanwhile, the idea that male sex offenders would rather pretend to be female to gain access to private spaces (where if they start doing anything creepy, they can be kicked out by the majority regardless of whether the majority can tell what their birth gender is or not) as opposed to doing what actual sexual abusers do, which is using strength and or manipulation in one on one situations, not joining women-only groups for the opportunity.

It's all just fear mongering lies by a minority of bigots who hate gays just as much as trans people, which are then believed by another minority of people who believe the bigots really do have women's interests at heart. Statistics don't show that people who claim to be trans are more likely than people who don't to sexually offend, and in fact the vast majority of sexual offenses are committed by straight, cis men who aren't trans nor are pretending to be trans. And that fear of "men" can translate into fear of women who were born as men, even though statistically they're less likely to be sexual abusers.