databricks-jobs

Develop and deploy Lakeflow Jobs on Databricks using DABs, Python SDK, or CLI.

4|1|Updated May 22, 2026
One-click install
npx skills add https://github.com/ThomazRossito/ai-data-agents --skill databricks-jobs-thomazrossito
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: databricks-jobs
Source: https://github.com/ThomazRossito/ai-data-agents/tree/main/plugins/ai-data-agents/skills/databricks-jobs
Command: npx skills add https://github.com/ThomazRossito/ai-data-agents --skill databricks-jobs-thomazrossito

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve? Building and operating multi-task data workflows on Databricks requires knowing the correct job configuration syntax across three different interfaces (Asset Bundles, Python SDK, CLI), plus triggers, compute, retries, and monitoring. This Skill provides the complete reference and working patterns so jobs are configured correctly before implementation begins. ## Core Features & Use Cases - Job Scaffolding and Deployment: Scaffold new job projects with databricks bundle init, then validate, deploy, and run via Declarative Automation Bundles across dev, staging, and prod targets. - Task and Trigger Configuration: Configure all task types (notebook, Python wheel, SQL, dbt, pipeline, JAR, run_job, for_each) and triggers (cron, periodic, file arrival, table update, continuous) with DAG dependencies and run_if conditions. - Monitoring and Reliability: Set up email and webhook notifications, health rules, timeouts, retries, and run queue settings for production workloads. - Use Case: Build a daily ETL pipeline where three extract notebooks run in parallel on a shared job cluster, a transform task waits for all extracts, and failures trigger PagerDuty webhooks with automatic retries. ## Quick Start Ask the agent to create a Databricks job that runs an extract notebook followed by a transform notebook on a daily cron schedule using Asset Bundles.

Frequently Asked Questions about databricks-jobs

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I create a Databricks job with multiple tasks?▼

Define tasks in an Asset Bundles YAML file with unique task_key values and wire dependencies using depends_on lists. Control execution with run_if conditions like ALL_SUCCESS or ALL_DONE, then deploy with databricks bundle deploy.

How to schedule a Databricks job with cron or triggers?▼

Use a schedule block with a quartz_cron_expression and timezone_id for time-based runs, or a trigger block with periodic, file_arrival, or table_update for interval and event-based execution. Set pause_status to UNPAUSED to activate.

Should I use Asset Bundles, Python SDK, or CLI for Databricks jobs?▼

Asset Bundles (DABs) are recommended for versioned, multi-environment deployments with dev/staging/prod targets. The Python SDK suits programmatic job management, while the CLI works for quick one-off operations like run-now or cancel-run.

Does Databricks jobs support serverless compute?▼

Yes, notebook and Python tasks use serverless compute automatically when no cluster configuration is provided. For custom dependencies on serverless, define an environments block with spec.client set to "4" and a dependencies list.

Why is my Databricks job schedule not triggering?▼

Schedules fail to trigger when pause_status is not UNPAUSED or the timezone_id is invalid. For file arrival triggers, verify the storage URL permissions; for table update triggers, confirm the tables are in Unity Catalog with proper grants.

How do I add retries and failure notifications to a Databricks job?▼

Set max_retries, min_retry_interval_millis, and retry_on_timeout on individual tasks. Add email_notifications or webhook_notifications blocks at job or task level with on_failure recipients, referencing a notification destination ID for webhooks.