Why do married men call the woman they’re seeing their “girlfriend” instead of their mistress?

I work as a hostess, and I often hear men say things like, “My girlfriend…” But when you get down to it, the man is married. Why don’t they just say, “My mistress…”? Why do they avoid calling her that?