Project Similarity Does Not Reliably Predict Cross-Project Defect Transfer

Cross-project defect prediction reuses labelled data from one software project to predict defects in another, but it often works poorly, and there is no reliable way to know which source project will transfer well to a given target. Naive project-level similarity is the natural first predictor to check, and prior work has moved past it toward domain adaptation and learned representations on the assumption that it is weak. We quantify how weak it actually is across 20 public defect datasets from two ecosystems (12 NASA MDP projects in C/C++, 8 PROMISE Java projects) and 380 source-to-target transfer pairs, comparing two project-level predictors of transfer AUC: the difference in defect rates, and a structural similarity score from each project's code-metric distribution. Defect-rate alignment is the more consistent predictor, though not uniform across classifiers. Structural similarity shows no measurable effect for logistic regression or random forest, a weak non-robust effect for SVM, and a small out-of-sample-validated effect only for naive Bayes, which we report as an unexplained anomaly rather than a general result. A learned graph-neural-network similarity over abstract syntax trees fared no better. We find no general-purpose project-level predictor of transfer success, and document a label-binarisation defect whose correction materially changed our conclusions.