For the fully autonomous driving systems that need to work under any circumstances, it is essential to be able to detect if there is any degradation of the perception models and be aware of the robustness of these algorithms to the different weather conditions. A sun glare has always been an issue for manual driving and is becoming a real problem for autonomous driving as well. Since it can obstruct critical information on the overexposed region. Ignoring and letting the algorithms work on corrupted camera images can lead to fatal consequences. In order to achieve the self-awareness, in this paper, we propose a sun glare detection approach and robustness benchmark to sun glare corruption based on glare rendering. In the benchmark, different severity levels of glare are added to assess the vulnerability of CNN detectors. With the help of self-supervised learning, our detection approach tackles the problem of glare data collection and annotation. Online glare synthesizing allows the CNN to take various and diverse training data, which makes the model robust and easy to generalize. We experimentally show that our method outperforms the state-of-the-art methods.