From Retrieval to Reasoning: How LangChain Gave Our RFP Assistant Market Intelligence

The Problem No One Talks About

Government contractors spend enormous time trying to price RFP responses competitively. But most AI-powered RFP tools share a critical blind spot: they know everything inside the procurement document, and absolutely nothing about what comparable work actually costs in the federal market.

Our RFP Assistant was no exception. Ask it for a price estimate, and it would respond: "I cannot provide an estimate based on the provided RFP information." That answer was technically correct — and completely useless. The root cause wasn't a model failure. It was an architectural gap between what RAG retrieves and what contractors actually need to make a bid decision.

The RAG-Only Boundary Problem

Retrieval-Augmented Generation is a powerful architecture for document Q&A. Embed documents, retrieve relevant chunks, ground the LLM's response in retrieved text. For factual questions about a specific RFP, it works exceptionally well. The problem emerges at the boundary of the document.

Government contracting involves decisions that require knowledge no single RFP contains — competitive pricing, historical award patterns, set-aside rates, and incumbent data. These live entirely outside any single RFP, in federal award databases and market intelligence sources. The space between document-bound knowledge and market-level knowledge is exactly where contractors make or lose money. Knowing requirements is table stakes. Knowing the competitive landscape is the edge.

The Gap We Didn't Know We Had

Here's what made this frustrating: we already had the data. ScopeBase stores NAICS codes for every work scope extracted from an RFP. We had a working usaspending_service.py that queries USASpending.gov for comparable federal awards. We had a CostEstimatorCrew built months earlier. The three pieces were just never connected to the chat interface.

The Architecture — A Layered LangChain Approach

We designed the fix around three hard constraints: don't touch existing agent code, fail gracefully, and use existing data. These constraints led to a subclass pattern: LangChainEnhancedOrchestratorService extends the existing AgentOrchestratorService. It intercepts estimation-intent questions before they reach the original orchestrator. Everything else passes through unchanged.

Implementation footprint: 2 new files, 1 import line change. No existing agent code modified.

The 6-Step LangChain Pipe Chain

The key insight: LangChain's Runnable base class overloads Python's __or__ method, so any two Runnables joined with | automatically compose into a RunnableSequence. This is the same pipe operator pattern taught in NVIDIA's "Building RAG Agents with LLMs" course — now running in production government contracting.

The Estimation System Prompt — Specialization, Not Loosening

The most important design decision wasn't the architecture. It was the system prompt.

This isn't "loosening" the AI. It's prompt specialization. The two prompts serve fundamentally different epistemic tasks. Intent detection routes each question to the right prompt automatically — no user configuration required.

Measuring the Improvement — LLM-as-Judge & The Data Flywheel

Measuring quality isn't a dashboard problem — it's a chain problem. We built lc_evaluator.py, an LLM-as-judge evaluator that fires automatically after every estimation response as a non-blocking background task (asyncio.create_task). Zero added latency to the user. Every scored response feeds back into the evaluation log — creating a continuous improvement loop, or Data Flywheel.

Four dimensions are scored per response: Provides Estimate (gives a specific dollar figure), Cites Contracts (names real USASpending.gov awards), Actionable (a contractor could use it to make a real bid decision), and Shows Reasoning (transparent logic and data sourcing). These average into a composite_score from 0.000–1.000.

Query by is_lc_chain in the estimation_eval_log and you get a direct A/B distribution comparison between the new and old paths. The flywheel closes: every estimation query scores itself, surfaces signal, and sharpens the next prompt iteration.

What's Next — From Aggregate Stats to Semantic Similarity

The current USASpending integration returns aggregate statistics — median, average, and range. The next upgrade replaces this with semantic similarity search using FAISS (Facebook AI Similarity Search), enabling responses like: "The most similar past award was [Contract X], awarded for $1.8M in Texas in 2023 for a custom data migration scope — which closely matches your Scope 003."

The Broader Lesson

This project illustrates a pattern that applies across all government contracting AI: the transition from document-bound Q&A to externally-grounded reasoning. The same pipe operator pattern demonstrated in NVIDIA's "Building RAG Agents with LLMs" course is now running in a production government contracting platform used by DBE, HUB, WBE, MBE, and ACDBE certified firms.

The gap between document-bound and market-grounded AI is not a research problem — it's an engineering problem. And it's solvable with two new files and one import line change. If you have NAICS codes, a working LLM integration, and access to USASpending.gov, you have everything you need to close the gap today.