Your problem is that you're not understanding boolean logic. If you set a trait to AND, it means that all visitors MUST have this trait. Setting AND Gay, AND Female means that in addition to anything else, they must be gay/female. Setting something to NOT means that a visitor must NOT have that trait: NOT vampire, for instance, means No Vampires. So if you set AND Gay, AND Female, NOT vampires, you get gay, female, non-vampires. If you set "everything else" to "not", you have created a condition where the visitable set is the empty set. For instance, NOT male, NOT female means that your visitors must not be male, and must not be female. Since all sims are one or the other, there are no neutermales, you now have no visitors.
I was unclear: "everything else" referred only to the other orientation options. I definitely didn't go in and click "not" on every single option on every single submenu on all the lots I tested on. What I meant was:
1) If the orientation options are set at "and-gay, not-het, not-bi, not-asexual," then only male gay sims show up, while changing it to "and-gay, or-het, or-bi, or-asexual" without touching any other menu allows female gay sims to show up too. Just remembering to use "or" isn't a problem, but I don't think that's supposed to happen, it's not happening with other options, and I thought it might be related to the second bit.
2) If the orientation options are set at something like "or-gay, or-bi, not-asexual, not-heterosexual," then heterosexual females are showing up (although the male ones aren't), and the controller gives "Allowed: Code 0" when they are made selectable and visitability is checked. They don't show up if one of the other orientations is on "and," so it can be dealt with by setting up more than one selector block, but that gets a little annoying if there are several other conditions, given that it shouldn't be happening.
I otherwise haven't had trouble getting the selector to work, although I haven't tried much beyond restrictions by age and playability. It's just the orientation bit, afaict, that is acting strangely.