Geometric Attention Fine-Tuning Is Representation-Relative: Autonomous ML Research with AI-Native Workflow
Yuhe Sui ⋅ Jianing Zhang ⋅ SHUFANG CHEN ⋅ 李晓彤 ⋅ Wang Zimeng
Abstract
Can explicit attention-compatibility adaptation add useful capacity beyond adapting query/key representations? We study this question through GeoAFT, an autonomous ML research programme managed by \emph{AI-Native Workflow}. At fixed hidden states, low-rank Q/K updates obey row-centered feature-subspace and rank constraints, while one-dimensional $\ell_1$ compatibility yields centered rank $N-1$; controlled interventions recover the reported static generators. On pretrained Web-of-Science classification, Q/K-LoRA rank 4 reaches $0.7291$ micro-F1 versus $0.5142$ for the geometry-bank residual, and matched specificity tests find $0/90$ head/layer cells where a named non-dot residual wins. Together these results establish a \emph{representation-relative} boundary: fixed-state compatibility complexity need not translate into transferable pretrained utility once representations can adapt. The autonomous system made a decisive contribution by executing the bounded positive search, escalating discriminating controls, preserving the evidence that changed the paper-level conclusion, and returning it for human scientific reconciliation. \sys\ supports this process with durable scientific intent, bounded execution, evidence-gated verification, and compact decision packets, targeting lower human-attention amount and intensity per unit of verified research progress.
Chat is not available.
Successful Page Load