A Context-Enriched NL-to-SQL Agent for Premier League Football Database Querying
Scaling Trustworthy NL-to-SQL for Complex Enterprise Domains
The Challenge: The "Living Document" Problem
Enterprise data in professional sports is not only large and relationally complex; it is governed by dense, annually updated rulebooks, handbooks, and laws. In this setting, correct SQL generation depends on more than schema knowledge. A model must also understand external regulatory logic that may not appear anywhere in the database itself.
This project investigates that problem through the English professional football domain, where questions about registrations, eligibility, competitions, and squad rules often require synthesizing structured database values with external Premier League, FA, and IFAB rules.
Architecture & Evaluation
I developed a 400-question benchmark across multiple difficulty tiers to evaluate NL-to-SQL performance on a representative football database. The benchmark tests both syntactic SQL complexity and semantically difficult questions requiring external domain-specific knowledge.
I compared two retrieval-augmented architectures against strong in-context learning baselines:
-
Intermediate Refinement Pipeline (RAG-R): A decomposed, multi-stage pipeline that first uses retrieved regulatory context to refine the user’s natural-language query, then passes the clarified prompt to a SQL generation step.
-
Direct Context Augmentation Agent (RAG-C): A monolithic agentic workflow that retrieves external context and attempts to reason, plan, and generate SQL within a single execution loop.
Average Exact Set Match (EM) Accuracy: RAG-Refinement (62.7%) vs. Baseline Chain-of-Thought (51.1%)
Key Findings & Metrics
The strongest result was not simply that RAG improved performance, but that architecture mattered. The decomposed RAG-R pipeline outperformed both the direct agentic RAG-C approach and the strongest Chain-of-Thought baseline.
-
Average Accuracy: RAG-R improved average Exact Set Match accuracy from 51.1% with ten-shot Chain-of-Thought prompting to 62.7%.
-
Highest-Difficulty Queries: On regulation-dependent semantic stress-test queries, Exact Set Match accuracy improved from 24.4% to 52.2%, a 27.8 percentage-point gain.
-
Architecture Tradeoff: The direct agentic RAG-C architecture failed to outperform the best non-RAG baseline, suggesting that stuffing retrieved context, reasoning, and SQL generation into one agentic loop can degrade performance in dense, legalistic domains.
.png)
The RAG-R architecture separates query refinement from SQL generation, making the system more auditable and reducing errors caused by context stuffing.
Why This Matters for Enterprise AI
This project frames NL-to-SQL not just as a generation problem, but as a production AI systems problem. In high-compliance domains, the best architecture is not always the most autonomous one. Systems need to be accurate, explainable, maintainable, and debuggable.
-
Explainability: RAG makes it possible to retrieve and inspect the specific rule passages used to clarify a query, supporting a more auditable “glass box” workflow than fine-tuning alone.
-
Maintainability: In living-document domains, updated rulebooks can be re-embedded without retraining the underlying model, making RAG more practical than fine-tuning for frequently changing regulatory corpora.
-
Failure Isolation: Separating query refinement from SQL generation makes it easier to identify whether a failure came from retrieval, domain interpretation, or query construction.
-
Production Cost: Decomposed pipelines also make architecture decisions more explicit, allowing tradeoffs around token cost, latency, context size, and model capability to be evaluated rather than hidden inside a single end-to-end agent.
The performance gap is most important at higher difficulty levels. While standard prompting degrades as questions require more external regulatory knowledge, the decomposed RAG-R pipeline maintains stronger Exact Set Match accuracy by isolating context enrichment before SQL generation.