Driven by a higher need for reliability, computational histopathology faces challenges in generalizing across domains due to variations in staining procedures and imaging equipment used by different hospitals. We focus on methodically simple options to improve results, which can be applied to a wide range of architectures. We draw inspiration from the tradition of shape-based recognition by hypothesizing that focusing on nuclei will improve the out-of-domain generalization in cancer detection. We propose training with a mixture of original images and segmentation masks while leaving the test time protocol unchanged. Going beyond mere data augmentation, we show that regularization, which enforces a higher similarity between mask and image representation, is a required component to achieve better out-of-domain generalization. Furthermore, we evaluate sensitivity to image corruptions and show that the proposed method improves robustness to such image distortions.