gcp-managed-airflow-recommendations

Diagnose Cloud Composer environments and generate evidence-based Airflow tuning recommendations.

Updated Aug 26, 2026
One-click install
npx skills add https://github.com/Lathika-laa/Recipe_Box --skill gcp-managed-airflow-recommendations-lathika-laa
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: gcp-managed-airflow-recommendations
Source: https://github.com/Lathika-laa/Recipe_Box/tree/main/.github/.gemini/skills/gcp-managed-airflow-recommendations
Command: npx skills add https://github.com/Lathika-laa/Recipe_Box --skill gcp-managed-airflow-recommendations-lathika-laa

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve? Tuning Managed Service for Apache Airflow (Cloud Composer) environments is hard without data: teams often blindly upsize machines or guess at configuration changes when facing workload restarts, slow DAG parsing, or resource saturation. This Skill gathers real telemetry from Cloud Monitoring and produces evidence-based recommendations for scaling, configuration, and reliability. ## Core Features & Use Cases - Telemetry-Driven Diagnostics: Runs Python scripts that query Cloud Monitoring via PromQL to collect CPU, memory, disk usage, restart counts, DAG parsing stats, and environment health percentages for each Airflow component (scheduler, workers, webserver, triggerer). - Configuration Inspection: Uses gcloud commands from the bundled reference to inspect environment size, scaling parameters, bucket contents, and component logs. - Structured Recommendations: Produces categorized advice covering infrastructure and scaling, airflow.cfg overrides, bucket hygiene, and production best practices such as high-resilience mode and deferrable operators. - Use Case: Your Cloud Composer scheduler keeps restarting and DAGs are delayed. Ask the Skill to analyze the environment; it pulls restart counts, memory usage, and DAG parsing times, then identifies an OOMing scheduler and recommends specific memory limits and parsing optimizations. ## Quick Start Analyze my Cloud Composer environment 'prod-airflow' in project my-gcp-project, region us-central1, over the past 24 hours and give me tuning recommendations.

Frequently Asked Questions about gcp-managed-airflow-recommendations

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I diagnose Cloud Composer performance issues?▼

Run the provided Python scripts with your project ID, location, environment name, and query duration in hours. They query Cloud Monitoring via PromQL and return CPU, memory, disk, restart, and DAG parsing metrics per Airflow component for analysis.

How to check why Airflow scheduler keeps restarting in Cloud Composer?▼

Use the workload_restarts.py script to get restart counts per component, then correlate with workload_memory_usage.py output. High scheduler restarts typically indicate out-of-memory conditions or unoptimized DAGs blocking the event loop.

What metrics does this Skill collect for Managed Airflow environments?▼

It collects environment health percentages, database and webserver health, CPU usage in cores, memory and disk usage in GiB, component restart counts, and DAG parsing stats including parse errors, processor timeouts, and total parse times.

Does this Skill work with both Cloud Composer Gen 2 and Gen 3?▼

Yes, but recommendations must match the architecture since Gen 2 and Gen 3 differ in infrastructure. The Skill instructs the assistant to distinguish generations before suggesting scaling or configuration changes.

What permissions or setup are required to run the diagnostic scripts?▼

The scripts use the gcloud CLI to obtain an application-default access token and call the Cloud Monitoring PromQL API. You need gcloud installed, authenticated, and sufficient IAM permissions to read monitoring data for the target project.

Why does DAG parsing take so long in Cloud Composer?▼

Common causes include heavy top-level code in DAG files, database calls for Variables or Connections at parse time, and dynamically generated DAG files. The dag_parsing_stats.py script measures parse times and errors to confirm the bottleneck.