In this critical survey, we analyze typical claims on the relationship between XAI and fairness to disentangle the multidimensional relationship between these two concepts.Based on a systematic literature review and a subsequent qualitative content analysis, we identify seven archetypal claims from 175 papers on the alleged fairness benefits of XAI.We present crucial caveats with respect to these claims that emerged from our literature review and provide an entry point for future discussions around the potentials and limitations of XAI for specific fairness desiderata.While XAI appears to be applicable across several fairness desiderata, we notice a misalignment between these fairness desiderata and the capabilities of XAI.We encourage to conceive XAI as one of many tools to approach the multidimensional, sociotechnical challenge of algorithmic fairness and to be more specific about \textit{how} exactly \textit{what} kind of XAI method enables \textit{whom} to address \textit{which} fairness dimension.