Fact-checking systems tend to fall into one of two traps: they either retrieve too much noise and drown the model in irrelevant context, or they stay too conservative and miss the evidence needed to verify a claim. Building AutoAdapt-Fact taught me that the retrieval strategy matters more than the generation model, and that making the pipeline agentic (capable of self-correction and iterative refinement) is what bridges the gap between a demo and something that actually works.
The Claim Extraction Problem
Before you can fact-check anything, you need to isolate the verifiable claims from a piece of text. A single paragraph might contain five or six distinct assertions mixed with opinion, context, and hedging language. I used a two-stage extraction pipeline: first, spaCy's dependency parser to break sentences into subject-predicate-object triples, then a BART-based model to rewrite those triples into self-contained claims that can be independently verified.
This separation matters because a claim like "revenue grew 12 percent" might be true in one context and false in another. Each extracted claim needs to carry enough context to be evaluable on its own. The BART model was fine-tuned on a dataset of journalistic claims to handle this rewriting step, with human evaluation confirming that roughly 85 percent of extracted claims were self-contained and verifiable.
Hybrid Retrieval: Why Both Matter
Dense retrieval (FAISS with sentence-transformer embeddings) captures semantic similarity well. If the claim is about "climate policy changes in 2024" and the source document discusses "environmental regulation updates this year," dense retrieval will connect them. But it also surfaces documents that are topically related but factually contradictory.
Sparse retrieval (TF-IDF with BM25 scoring) is the opposite. It matches on exact terms and is robust against semantic drift, but it will miss paraphrased or synonym-heavy evidence. Running both and merging results with a weighted reciprocal rank fusion gave the best recall. On a test set of 500 claims checked against a 50,000-document corpus, hybrid retrieval achieved 78 percent evidence recall compared to 62 percent for dense alone and 58 percent for sparse alone.
The weight between dense and sparse scores was not static. The pipeline learned to adjust it based on claim type: fact-heavy claims (statistics, dates, names) received more weight from sparse retrieval, while interpretive claims (causation, prediction) leaned on dense retrieval. This adaptive weighting was a simple logistic regressor trained on labeled claim-type annotations.
The Verification Loop
A single pass verification is not reliable enough for production use. The agent architecture introduces a verification loop: the model generates an initial verdict (supported, refuted, or insufficient evidence) with a confidence score. If confidence falls below a threshold (0.7 in our setup), the pipeline triggers a second retrieval round using reformulated queries. The model rewrites the original claim from different angles (negation, temporal shift, scope change) and retrieves evidence for each variant.
This second pass caught contradictions that the first pass missed roughly 18 percent of the time. A claim might appear supported by one source but refuted by a more authoritative source that only surfaced when the query was reformulated. The loop was capped at three iterations to keep latency bounded, though in practice most claims resolved within one or two rounds.
Self-Optimization
The "self-optimizing" part of AutoAdapt-Fact is a meta-layer that tracks which retrieval and verification strategies work for different claim categories. Over time, it adjusts the dense/sparse weight, the confidence threshold for triggering additional retrieval rounds, and the claim rewriting prompts. This was implemented as a simple bandit algorithm that selects strategy configurations based on their historical accuracy on similar claim types.
In offline evaluation on a held-out set of 200 claims, the self-optimizing pipeline improved verification accuracy from 71 percent (static configuration) to 79 percent after processing roughly 1,000 claims. The gains were largest on claims that fell into ambiguous categories where the optimal strategy was not obvious upfront.
The Interface Layer
All of this was exposed through a Streamlit dashboard that let users paste articles, review extracted claims, see the evidence retrieved for each claim, and trace the verification path including any additional retrieval rounds. The interface made it possible for human reviewers to catch systematic errors and recalibrate the pipeline without changing any code.
Evidence snippets were presented alongside confidence scores and retrieval provenance (dense vs. sparse, which iteration found it). This transparency was arguably as important as the accuracy gains. In a fact-checking context, knowing why a system made a particular call matters as much as whether the call was correct.
Takeaways
Hybrid retrieval is not optional for this kind of task. Neither dense nor sparse alone gives you the coverage you need, and the fusion strategy does not need to be complex. A simple logit-based weight adjustment based on claim type was enough to capture most of the benefit. The verification loop was the single highest-impact change, and it is something you can add to any existing fact-checking pipeline without rearchitecting the whole thing. Self-optimization helped at scale but was not the difference between a bad system and a good one. Start with retrieval, add the loop, and only then think about adaptive strategies.