vqr-testing-audit

Tests verified queries against Snowflake semantic views without VQR hints to measure completeness.

Updated Jun 9, 2025
One-click install
npx skills add https://github.com/RajaPoseidon/snippets_repo --skill vqr-testing-audit-rajaposeidon
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: vqr-testing-audit
Source: https://github.com/RajaPoseidon/snippets_repo/tree/main/snowflake/skills/audit/vqr_testing
Command: npx skills add https://github.com/RajaPoseidon/snippets_repo --skill vqr-testing-audit-rajaposeidon

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Semantic views in Snowflake rely on verified queries (VQRs) to guide Cortex Analyst SQL generation, but teams lack a systematic way to know whether the semantic view alone is sufficient. This Skill audits semantic view completeness by testing every verified query without VQR hints and identifying which questions fail. ## Core Features & Use Cases - VQR Extraction and Selection: Extracts verified query questions and ground truth SQL from semantic model YAML files and lets the user pick which ones to audit. - Parallel Evaluation: Generates SQL via Cortex Analyst with VQR retrieval disabled, executes SQL pairs with eval_sql_pair.py, and compares result sets for exact matches. - Structured Results Reporting: Presents pass/fail summaries with generated SQL and specific differences for failing queries, then routes to debug mode or further auditing. - Use Case: A data engineer publishes a semantic view and wants to verify that Cortex Analyst can answer all verified questions correctly without relying on stored VQR hints before rolling it out to analysts. ## Quick Start Run a VQR testing audit on my semantic view to find which verified queries fail without VQR hints.

Frequently Asked Questions about vqr-testing-audit

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I test verified queries in a Snowflake semantic view?▼

Extract the verified queries from the semantic model YAML, then query Cortex Analyst with each question using a semantic model copy that has VQRs removed. Compare the generated SQL execution results against the ground truth SQL results to determine pass or fail.

How to check if Cortex Analyst works without VQR hints?▼

Call snowflake_multi_cortex_analyst with skip_vqr_retrieval enabled and a semantic model file with verified queries stripped out. Then execute both the generated SQL and ground truth SQL with eval_sql_pair.py and compare the result sets.

What counts as a pass or fail in VQR testing?▼

A VQR passes only when the generated SQL execution results match the ground truth results exactly. It fails when results differ, the SQL errors, or no SQL is generated at all. SQL structure alone never determines pass/fail.

Why does VQR evaluation require comparison output files for every query?▼

The workflow mandates that the count of generated SQLs equals the count of comparison result files before categorization. This checkpoint prevents skipping evaluations and ensures every VQR verdict is based on actual execution evidence.

Can I audit only some verified queries instead of all of them?▼

Yes. After extraction, the skill presents all VQR questions numbered and waits for your selection. Enter specific indices like "1,3,5" to audit a subset, or "ALL" to test every verified query.