Noise and Memory: Sensitivity-Aware Differential Privacy for Historical Testimony
Abstract
Differential privacy(DP) protects an individual's identity by adding statistical noise to their data [1], but historical testimony often carries its evidentiary weight precisely in the specific, identifying details (a location, a date, a profession) that uniform privacy mechanisms obscure indiscriminately. We argue that privacy and historical preservation are not simply competing objectives: a privacy mechanism for testimonial data must itself encode the different evidentiary and ethical roles played by different categories of information. Our research question is whether a sensitivity-aware differential privacy mechanism can provide a formally specified privacy guarantee for testimonial archives while preserving the narrative and evidentiary integrity that uniform mechanisms tend to destroy. We evaluate on 1,000 curated English-language testimony transcripts from the USHMM and Wiener Holocaust Library (approximately 2.8 million tokens, 800/200 train/test split, stratified by archive of origin). Testimonies containing substantial non-English content were excluded from this study, with multilingual extension left to future work. We make three contributions. First, we restrict automated processing to non-generative extraction, using domain-adapted named entity recognition, with each entity tagged to its source testimony ID, transcript offset, and interview metadata, enabling span-level citation for any downstream use. This constrains the pipeline's capacity to fabricate content about a narrator's experience, though it does not eliminate extraction error, which we treat as a data-quality issue for human review rather than a privacy risk. Second, we define the privacy unit as the individual testimony and distinguish semantic sensitivity, a span's potential to enable re-identification through names, locations, dates, or professions, scored via NER and validated against two archivists' annotations (Spearman ρ=0.81), from DP query sensitivity, the mathematical quantity used to calibrate training noise. The two are connected through per-example gradient clipping: higher-sensitivity testimonies receive tighter clipping norms under an individualised DP-SGD scheme, with privacy loss bounded via a Rényi DP. We report a worst-case ε of 5.5 at δ of 1e-5, selected via a sweep across candidate ε values against a pre-registered 8-point F1 degradation threshold. As a separate, non-DP baseline, we evaluate selective anonymisation alone: deterministic, NER-triggered generalisation of flagged spans, offering no formal privacy guarantee. Third, a consent-scoped access layer with mandatory expert sign-off restricts pipeline outputs to archive-specified consent terms; we frame this as a governance design requirement rather than a formal enforcement guarantee. Against a non-private baseline (NER F1 of 85.1), uniform DP-SGD reduces membership inference attack success by 42% at a cost of 13.6 F1 points (71.5). Selective anonymisation alone yields weaker privacy protection (18% MIA reduction) at a smaller utility cost (F1 of 82.0). Our hybrid mechanism matches uniform DP-SGD's 42% MIA reduction while recovering utility to an F1 of 78.7, with the largest entity-level gains in person names (plus 8.1 F1) and locations (plus 6.9 F1). Three blinded Holocaust historians rated 60 paired source and transformed passages for factual, contextual, and narrative fidelity on a 5-point scale (mean 4.0 out of 5, Krippendorff's alpha of 0.68).We note that automated sensitivity classification may itself encode cultural or archival assumptions about what counts as identifying versus evidentiary, and that differential privacy addresses re-identification and memorisation risk specifically, not the fuller set of ethical obligations, including consent, cultural sovereignty, and dignity, that oral history scholarship identifies [2]. The study is scoped to English-language Holocaust testimony; transfer to other archival contexts is left as future work rather than demonstrated here.