databricks-ai-functions

Apply Databricks built-in AI Functions to SQL and PySpark data pipelines.

4|1|Updated May 22, 2026
One-click install
npx skills add https://github.com/ThomazRossito/ai-data-agents --skill databricks-ai-functions-thomazrossito
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: databricks-ai-functions
Source: https://github.com/ThomazRossito/ai-data-agents/tree/main/plugins/ai-data-agents/skills/databricks-ai-functions
Command: npx skills add https://github.com/ThomazRossito/ai-data-agents --skill databricks-ai-functions-thomazrossito

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve? Adding LLM capabilities like classification, extraction, summarization, and translation to data pipelines normally requires managing model endpoints, API keys, and boilerplate code. This Skill shows how to call Databricks built-in AI Functions directly from SQL and PySpark so AI inference runs on table columns like any other function. ## Core Features & Use Cases - Task-Specific AI Functions: Use ai_classify, ai_extract, ai_summarize, ai_mask, ai_translate, ai_fix_grammar, ai_gen, ai_analyze_sentiment, and ai_similarity for batch inference without endpoint setup. - Document Processing & RAG: Chain ai_parse_document and ai_prep_search into staged Delta pipelines for parsing PDFs and building custom RAG indexes with Vector Search. - Advanced Querying & Forecasting: Use ai_query for custom endpoints, multimodal input, and structured JSON output, plus ai_forecast for table-valued time series forecasting. - Use Case: Parse a volume of vendor invoices with ai_parse_document, classify them with ai_classify, extract invoice number and total with ai_extract, and persist results to a Delta table for downstream analytics. ## Quick Start Use the databricks-ai-functions skill to classify the sentiment and extract key fields from the customer_feedback table using SQL.

Frequently Asked Questions about databricks-ai-functions

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I use Databricks AI Functions in SQL queries?▼

Call functions like ai_classify or ai_extract directly in a SELECT statement on table columns, for example ai_analyze_sentiment(review_text). Each call is an LLM inference billed per token, so run it once per row and persist results to a Delta table.

When should I use ai_query instead of ai_extract or ai_classify?▼

Use ai_query only as a last resort when no task-specific function fits, such as custom or external endpoints, multimodal image input via files, or JSON structures beyond ai_extract's limits. Task functions are pre-configured and need no endpoint selection.

What Databricks runtime is required for ai_parse_document and ai_prep_search?▼

ai_parse_document requires DBR 17.3 or higher, while ai_prep_search requires DBR 18.2 or higher with serverless environment v3. Most other task functions need DBR 15.1+ in notebooks or 15.4 ML LTS for batch jobs.

How do I build a RAG pipeline with Databricks AI Functions?▼

Chain ai_parse_document to parse binary files, then ai_prep_search for semantic chunking and context enrichment, and persist chunks to a Delta table with Change Data Feed enabled. Point a Vector Search Delta Sync index at chunk_to_embed and return chunk_to_retrieve to the LLM.

Why does ai_forecast fail on my SQL warehouse?▼

ai_forecast requires a Pro or Serverless SQL warehouse and is not available on Classic or Starter warehouses. It is a table-valued function, so call it with SELECT * FROM ai_forecast(...) or via spark.sql() in PySpark.

How do I handle errors from ai_query in batch jobs?▼

Set failOnError => false so ai_query returns a STRUCT with response and errorMessage fields instead of raising. Route rows where errorMessage is not null to a sidecar table so the batch run does not crash.