Case/control Genome-wide association studies (GWAS) for late onset Alzheimer's disease (AD) may miss genetic variants relevant for delineating disease stages when using clinically defined case/control as a phenotype since the cases highlight advanced AD and widely heterogeneous mild cognitive impairment patients are usually excluded. More precise phenotypes for AD are in demand. Here we use a transfer learning technique to train three-dimensional convolutional neural network (CNN) models based on structural Magnetic Resonance Images (MRI) from the screening stage in the ADNI consortium to derive image features that reflect AD progression. CNN-derived image phenotypes are significantly associated with genetic variants mapped to candidate genes enriched for amyloid beta degradation, tau phosphorylation, calcium ion binding-dependent synaptic loss, APP regulated inflammation response, and insulin resistance. This is the first attempt to show that non-invasive MRI biomarkers are linked to AD progression characteristics, reinforcing their utilizations in early AD diagnosis and progression monitoring.