NyayaReasoner: A Verified, Retrieval-Grounded Reasoner for Indian Supreme Court Judgment Prediction with Explicit Abstention

Court Judgment Prediction for the Indian Supreme Court is challenging because existing approaches fail in ways an advocate cannot trust. Binary-outcome classifiers built on domain-adapted encoders such as InLegalBERT (Paul et al., 2022) reach around 70% accuracy on the Indian Legal Documents Corpus (Malik et al., 2021) but explain themselves only through attention heat-maps. Instruction-tuned large language models can reason plausibly over long documents but occasionally invent precedent, while neither approach provides an explicit evidence-based mechanism for abstention. We therefore ask: can retrieval-grounded, step-wise verification of LLM reasoning produce a legal judgment prediction system that both improves accuracy and abstains reliably when the evidence is thin? We present NyayaReasoner. The system combines three ideas. First, a hybrid retriever over ~13,000 Supreme Court judgments fuses a whole-document BM25 index with a dense InLegalBERT index via Reciprocal Rank Fusion, so the reasoner sees evidence that is at once keyword-faithful and semantically related. Second, a step-wise Reasoning-and-Acting policy (Yao et al., 2023) alternates thought, retrieval, and answer actions and emits a structured trace whose every claim can later be audited. Third, a two-tier verifier grades each atomic claim in the trace against the retrieved evidence, with one tier reusing the LLM as a self-judge and the other using an external DeBERTa-v3 NLI cross-encoder. The trace's verified confidence, formed by combining policy confidence with mean per-step support, gates a selective-prediction rule that maps outputs to allowed, dismissed, or abstain. We additionally fine-tune Llama-3.1-8B-Instruct with LoRA on verifier-graded reasoning traces and aggregate three-sample self-consistency draws. The case under analysis is removed from the retriever's index at evaluation, and the case-number prefix from which the case-category label is derived is stripped from the model input to prevent trivial metadata leakage. Evaluated on a held-out 200-case sample drawn i.i.d. from 2022–2024, with 46 cases excluded from the binary-outcome evaluation due to missing or out-of-scope outcome labels, the system produces a monotone risk-coverage curve: accuracy is 82.1% at 100% coverage (n=154, 95% Wilson binomial CI [0.75, 0.87]) and rises to 87.9% when the system commits on only 31.8% of cases (n=49, 95% Wilson CI [0.76, 0.94]), providing evidence that verified confidence is a useful abstention signal rather than a nominal label. LoRA fine-tuning on verifier-graded traces lifts case-category macro-F1 from 0.607 to 0.645, and adding three-sample self-consistency lifts it further to 0.883, primarily by recovering rare classes such as Writ Petition and Special Leave Petition. Replacing the self-judge with the external NLI tier lowers mean trace-support from 0.957 to 0.875, suggesting that the self-judge systematically overestimates its own claims, and gives a wider useful abstention range. To our knowledge, this is among the first Indian Supreme Court judgment-prediction systems to combine retrieval-grounded reasoning, claim-level verification, and explicit evidence-based abstention.