StocksTalk: A Voice-Enabled Conversational Agent for Structured Query Generation over Web Data

arXiv:2608.181052026-08-20

A voice assistant that turns spoken stock-screening requests into checked SQL queries

StocksTalk is a conversational system that converts spoken investment screening requests, like 'large-cap IT stocks with P/E below 25,' into executable SQL queries over real financial data. It chains streaming speech recognition, retrieval-augmented constraint extraction, schema-grounded LLM SQL generation, rule-based validation, and a human confirmation step inside one dashboard. On a 150-prompt benchmark, the full pipeline beat a plain GPT-4o baseline by wide margins on logical consistency and multi-turn stability.

METAL MEDIA explanatory visual

A voice assistant that turns spoken stock-screening requests into checked SQL queries

01Built a pipeline that turns spoken investment constraints (P/E ratio, dividend yield, debt-to-equity, etc.) into SQL queries executed against live financial data from Screener.in
02Designed a four-stage architecture—streaming speech-to-text (ElevenLabs), retrieval-augmented constraint extraction, schema-grounded GPT-4o SQL generation, rule-based validation, and human-in-the-loop verification—exposing every intermediate result on a dashboard so users can confirm or correct each step before execution
03Curated and plans to publicly release FinScreenBench, a benchmark of 150 spoken financial screening prompts (300 audio recordings total) across three strategy categories—growth, dividend, value—and two recording conditions, clean and noisy
04On metrics covering constraint extraction accuracy, SQL executability, logical consistency, query edit distance, and multi-turn stability, the full system with human verification outperformed a plain GPT-4o baseline by 27.8 percentage points on logical consistency and 37.4 points on multi-turn stability
05An ablation study showed each component fixes a different failure mode: retrieval-augmented generation mainly helps constraint extraction, constrained decoding mainly helps executability, and rule-based validation mainly helps logical consistency; removing human verification let errors persist across turns in 34% of multi-turn cases, versus just 9% with it

An explanatory diagram made by METAL MEDIA, not a figure supplied by the paper's authors.

What they did

Built a pipeline that turns spoken investment constraints (P/E ratio, dividend yield, debt-to-equity, etc.) into SQL queries executed against live financial data from Screener.in
Designed a four-stage architecture—streaming speech-to-text (ElevenLabs), retrieval-augmented constraint extraction, schema-grounded GPT-4o SQL generation, rule-based validation, and human-in-the-loop verification—exposing every intermediate result on a dashboard so users can confirm or correct each step before execution
Curated and plans to publicly release FinScreenBench, a benchmark of 150 spoken financial screening prompts (300 audio recordings total) across three strategy categories—growth, dividend, value—and two recording conditions, clean and noisy
On metrics covering constraint extraction accuracy, SQL executability, logical consistency, query edit distance, and multi-turn stability, the full system with human verification outperformed a plain GPT-4o baseline by 27.8 percentage points on logical consistency and 37.4 points on multi-turn stability
An ablation study showed each component fixes a different failure mode: retrieval-augmented generation mainly helps constraint extraction, constrained decoding mainly helps executability, and rule-based validation mainly helps logical consistency; removing human verification let errors persist across turns in 34% of multi-turn cases, versus just 9% with it

Figure 1: Overview of the StocksTalk architecture. The system transforms spoken financial queries into validated SQL queries, retrieves real-time market data, and enables human-in-the-loop verification through an interactive dashboard.

Table 1: Comparison of StocksTalk against baselines on 150 clean-input prompts. CEA = Constraint Extraction Accuracy; EX = SQL Executability; LCR = Logical Consistency Rate; QED = Query Edit Distance (lower is better); MTS = Multi-turn Stability.
System	CEA (%)	EX (%)	LCR (%)	QED	MTS (%)	LAT (s)
GPT-4o (plain)	63.4	81.2	54.7	8.3	51.2	1.5±0.1
GPT-4o + RAG	79.8	88.6	67.3	5.1	68.4	2.1±0.2
GPT-4o + RAG + Validation	88.3	96.9	91.2	2.8	82.7	2.3±0.2
StocksTalk (full)	91.2	97.5	93.8	2.1	88.6	3.1±0.4

Table 2: StocksTalk performance on 150 prompts under clean and noisy ASR conditions.
Metric	Clean Input	Noisy Input
Constraint Extraction Acc. (%)	91.2	78.4
SQL Executability (%)	97.5	89.3
Logical Consistency Rate (%)	93.8	82.1
Query Edit Distance (tokens)	2.1	5.7
Multi-turn Stability (%)	88.6	74.3
Avg. End-to-End Latency (s)	3.1±0.4	3.6±0.7

Table 3: Ablation study on 150 clean-input prompts. Δ denotes absolute drop from the full system.
Configuration	CEA (%)	EX (%)	LCR (%)
Full system	91.2	97.5	93.8
w/o RAG retrieval	74.3 (−16.9)	95.1 (−2.4)	87.2 (−6.6)
w/o validation layer	90.8 (−0.4)	96.9 (−0.6)	71.4 (−22.4)
w/o constrained decoding	88.5 (−2.7)	79.2 (−18.3)	84.3 (−9.5)
w/o human-in-the-loop	89.1 (−2.1)	94.3 (−3.2)	80.6 (−13.2)

Table 4: Effect of HITL verification on SQL executability (%) by prompt category, clean input.
Prompt Type	w/o HITL	w/ HITL	Gain
Growth-focused	94.2	98.6	+4.4
Dividend-oriented	91.7	97.3	+5.6
Value-based	88.3	96.8	+8.5
Overall	91.4	97.5	+6.1

Why it matters

Turning complex spoken conditions directly into runnable database queries is directly applicable to banks, brokerages, and any service building natural-language search interfaces. The design choice of exposing intermediate steps for user correction offers a concrete template for where humans should step in when an AI system might be wrong.

Terms in this paper

RAG (retrieval-augmented generation) · an AI technique that looks up relevant documents or knowledge before generating an answer
SQL · the query language used to search and filter data stored in a database
human-in-the-loop · a process where a person reviews or corrects an AI's output before it is acted on
ASR (automatic speech recognition) · technology that converts spoken audio into text
schema-grounded prompting · giving an AI the structure of a database table so it only generates queries valid for that structure

Original abstract (English)

StocksTalk is a voice-enabled conversational system for transforming spoken financial screening requests into executable and validated structured queries over real-world market data. The system combines streaming speech recognition, retrieval-augmented constraint extraction, schema-grounded LLM-based SQL generation, rule-based validation, and human-in-the-loop verification within an interactive dashboard. Unlike traditional template-driven financial assistants, StocksTalk exposes intermediate reasoning artifacts, including extracted constraints, normalized financial metrics, operator grounding, and generated queries, allowing users to inspect and refine each stage before execution. To evaluate the system, we curate a benchmark of 150 spoken financial prompts spanning multiple investment strategies and input noise conditions. Experimental results show that retrieval grounding, constrained query generation, and interactive verification substantially improve constraint extraction accuracy, SQL executability, logical consistency, and multi-turn stability compared to baseline LLM-based approaches. StocksTalk demonstrates how transparent, voice-driven interfaces can bridge natural language interaction and structured financial analysis, providing an effective framework for conversational stock screening and decision support.

Authors · Akshat Parmar, Vikranth Udandarao, Abhay Shakya, Tanmay Hire, Avinash Anand, Rajiv Ratn Shah, Daniel Wang Zhengkui

Read on arXiv

Latest papers

All papers →

Latest from METAL MEDIA

NCT 127 drop BLINGY, their first full album in two years

Figures: Akshat Parmar et al., arXiv:2608.18105, CC BY 4.0