ActiveRAG — Experiment 2

Building your AI Agentic Anchor

From paper to pipeline: implementing ActiveRAG for domain-specific AI using structured expert elicitation and a four-layer epistemic anchoring document.

Based on arXiv 2402.13547 — Read the paper on arXiv

ACTIVERAG: Building the epistemic anchor on the Expert Six

By The Engineering Team

If you have built a standard RAG pipeline, you know the dirty secret: it is often just a glorified search engine attached to a smooth talker. The model passively receives retrieved data and produces a response that sounds confident — whether or not it is correct.

The problem is not the retrieval. It is the passivity. Vector search can surface the right passage, and the LLM still fails to reason from it correctly, because it has no structured way to integrate external knowledge with what it already knows.

Data engineering is not just about moving bytes. It is about structuring truth. What we call "vector soup" — raw embeddings with no epistemic scaffolding — is inadequate for domain-specific work where precision matters. The ACTIVERAG framework, grounded in Constructivism theory and validated at Carnegie Mellon and partner universities, gives us the methodology to do better.

Section 1

The methodology: reviving expert elicitation

Before AI, before even modern databases, the field of Knowledge Engineering in the 1990s solved a harder version of this problem: how do you capture what an expert knows and make it machine-usable? The answer was expert elicitation — a structured interview and formalization process that extracted not just facts, but the rules, heuristics, and judgment calls that domain experts apply in real work.

ACTIVERAG revives this discipline for LLMs. The Epistemic Anchoring Document is the formalized output of expert elicitation — a structured knowledge artifact that provides the LLM with context it cannot derive from pretraining alone.

To ground the implementation, we use high-end perfumery as a demonstration domain, with an expert panel called the "Perfume Expert Six" — domain authorities including Victoria Belim-Frolova and Mark Behnke, spanning perfume history, chemistry, and critical evaluation.

The domain choice is deliberate: perfumery is precise, jargon-heavy, contains hard quantitative facts, and is exactly the kind of domain where general-purpose LLMs hallucinate confidently. If the framework works here, it works anywhere.

Section 2

The implementation: the Epistemic Anchoring Document

The Epistemic Anchoring Document is not a monolithic knowledge base. It has four distinct layers, each serving a different function in the ACTIVERAG pipeline. Together they form the Static Context Augmentation that the Cognitive Nexus fuses with the model's reasoning.

Layer A

The Expert Network — Provenance Layer

The provenance layer establishes an authority roster: who are the recognized experts in this domain, what are their credentials, and how should conflicting claims be resolved? This is the epistemic chain-of-custody that gives the LLM a basis for weighting information sources rather than treating all retrieved text as equally valid.

In the perfumery domain, this means listing recognized critics, master perfumers, historians, and chemists — with their specific areas of authority. When a retrieved passage contradicts another, the authority ranking determines which claim the model should prefer.

Layer B

The Rule-Base — Logic Layer

The logic layer converts expert intuition into explicit heuristics with conditional logic. Experts do not reason from flat facts — they reason from rules, and those rules often carry conditions, exceptions, and causal chains that are invisible in raw text.

Example from the perfumery domain — "Mark Behnke's Concentration Realism":

Increasing oil concentration can distort the scent rather than improving performance.

This rule is not derivable from a collection of product descriptions. It requires expert judgment, and without it, a model asked to recommend fragrance concentrations will produce plausible-sounding but incorrect guidance. The rule-base makes this expertise explicit and machine-usable.

Other examples in this layer include bergamot photosensitivity warnings, appropriate concentration ranges by accord type, and stability rules for specific ingredient combinations.

Layer C

The Specialized Glossary — Data Dictionary

Domain terminology is where LLMs fail most visibly. They predict tokens that are statistically common — not tokens that are technically precise. The data dictionary pins the exact meaning of domain terms so the model cannot drift into vague or colloquial usage.

Example: the term "sillage" in perfumery refers specifically to the scent trail left in the air after the wearer has passed — it is distinct from longevity (how long the scent lasts on skin) and projection (how far from the body the scent radiates). A model without this disambiguation will conflate all three.

The glossary also disambiguates homonyms. "Oud" may refer to the wood resin, a synthetic substitute, a fragrance family, or a cultural classification — the data dictionary specifies which meaning applies in which context, preventing the model from pattern-matching on surface similarity.

The "Attars" category — traditional oil-based perfumes with distinct cultural and regulatory implications — is another example where precise definition matters for both accuracy and responsible sourcing guidance.

Layer D

Hard Verification Markers — Unit Tests for Hallucination

The final layer functions as unit tests: quantitative facts so specific that any model response can be evaluated against them for correctness. These are hallucination shields — not hints, but hard constraints.

Example:

Chanel No. 5 contains exactly 1% aliphatic aldehydes.

If a model produces a response that contradicts this fact, the Cognitive Nexus mechanism — the third step in the ACTIVERAG pipeline — detects the contradiction and recalibrates the chain-of-thought before the response is surfaced.

The "Oud Disambiguation Rule" is another example: a verification marker that distinguishes authentic oud resin from synthetic substitutes, with specific chemical markers that allow the model to evaluate claimed provenance.

Hard verification markers are the direct operationalization of the ACTIVERAG Cognition agent — the layer responsible for mitigating knowledge that contradicts the model's pre-existing understanding.

Section 3

The future: from storage to reasoning

The four-layer Epistemic Anchoring Document is not a static reference artifact. It is the foundation for active reasoning. When the Logician Agent is engaged — the second of the three ACTIVERAG agents — it uses the rule-base and verification markers not just to retrieve information but to perform deductive inference: drawing conclusions that are not explicitly stated in any retrieved passage but follow logically from the structured knowledge.

This is the transition from storage to reasoning. A knowledge base that contains only facts enables retrieval. A knowledge base that contains rules, authority rankings, precise terminology, and hard constraints enables judgment — and that distinction is what separates a domain-specific AI deployment that is trustworthy from one that is merely fluent.

The structured data engineering that produces an Epistemic Anchoring Document is itself a form of cognitive engineering. It requires domain experts who know the rules, data engineers who can formalize them, and quality processes that can validate the result against ground truth. This is not a one-time task — it is an ongoing discipline that improves as the domain evolves.

The takeaway

Data engineering is not moving bytes from A to B. It is structuring truth in a form that machines can reason from. The ACTIVERAG framework provides the pipeline; the Epistemic Anchoring Document provides the knowledge scaffolding; expert elicitation provides the raw material.

The teams that get this right will not win because they have the biggest model. They will win because they have the best-structured domain knowledge — and the engineering discipline to keep it current, verifiable, and causally sound.

That is what AIDC builds.

Talk to AIDC ExpertSix methodology
Experiment 1

ActiveRAG Overview: The 3-Step Playbook

Knowledge Construction, Dynamic Action, and Cognitive Nexus — the foundational framework and deployment playbook from the arXiv paper.

Read Experiment 1
Back to ActiveRAG