Tracking Scientific Information Loss in LLM-Assisted Biomanufacturing Process Development
Abstract
Scientific assistants must preserve evidence as it moves from experimental design to comparison, decision, and operating instructions. We introduce BioProcessLineage, an evaluation with 12 linked workflows, 78 source-backed facts, and 109 fact-stage requirements from 11 openly licensed publications. Across ten hosted open-weight model routes, conventional screens are strong, yet the nine routes with linked outputs retain only 40.1\% of complete facts and 7.2\% of multi-stage facts consistently, despite 95.6\% exact-evidence precision. This separates grounding (support for included claims) from completeness. We performed a matched comparison of 8 routing methods where, model-created lineage and graph structures reduce retention, source-unit coverage certificates add 10.3 percentage points, and deterministic fact-slot compilation reaches 93.5\%. The latter preserves content by design but exposes stage routing as the next bottleneck. We explore routing source interventions that support this diagnosis: removing a target evidence unit lowers target-term retention by 46.7 points, while restoring it recovers 44.5 points. The results motivate evaluating scientific systems as information pipelines and protecting source facts before generation.