Differential measurement error, which occurs when the level of error in the measured outcome is correlated with the treatment renders the causal effect unidentifiable from observational data. We study conditional differential measurement error, where a subgroup of the population is known to be prone to differential measurement error. Under an assumption about the direction (but not magnitude) of the measurement error, we derive sharp bounds on the conditional average treatment effect and present an approach to estimate them. We empirically validate our approach on semi-synthetic and real data, showing that it gives a more credible and informative bound than other approaches.