We survey the treatment of sex and gender in the Computer Graphics research literature from an algorithmic fairness perspective. We conclude current trends on the use of gender in our research community are scientifically incorrect and constitute a form of algorithmic bias with potential harmful effects. We propose ways of addressing these trends as technical limitations.