skills.rest
1919114514yasenpai-maker
1919114514yasenpai-makerCOMMUNITY
Explore all skills by this author →
GitHub

gcp-managed-airflow-recommendations

Diagnose Cloud Composer environments and generate evidence-based Airflow tuning recommendations.

Updated Aug 16, 2026
Search Tags:#monitoring#gcp#diagnostics#performance tuning#promql#apache airflow#cloud composer
One-click install
npx skills add https://github.com/1919114514yasenpai-maker/focus-quest --skill gcp-managed-airflow-recommendations-1919114514yasenpai-maker
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: gcp-managed-airflow-recommendations
Source: https://github.com/1919114514yasenpai-maker/focus-quest/tree/main/.gemini/skills/gcp-managed-airflow-recommendations
Command: npx skills add https://github.com/1919114514yasenpai-maker/focus-quest --skill gcp-managed-airflow-recommendations-1919114514yasenpai-maker

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve? Tuning Managed Service for Apache Airflow (Cloud Composer) environments is hard without data: teams often guess at scaling parameters, suffer unexplained scheduler or worker restarts, and overspend by blindly upsizing machines. This Skill gathers real telemetry from Cloud Monitoring and turns it into concrete, evidence-based recommendations. ## Core Features & Use Cases - Telemetry-Driven Diagnostics: Runs Python scripts that query Cloud Monitoring via PromQL for CPU, memory, disk usage, workload restarts, DAG parsing times, and overall environment health. - Structured Recommendation Workflow: Follows a five-step process covering context gathering, configuration inspection via gcloud, metrics collection, diagnosis, and categorized recommendations (infrastructure, Airflow config, bucket hygiene, production best practices). - Use Case: Your Airflow scheduler keeps restarting and DAGs are delayed. Ask the Skill to analyze your environment; it pulls restart counts, memory usage, and DAG parsing stats, then identifies an OOM issue and recommends specific memory limits and airflow.cfg overrides instead of a blind upgrade. ## Quick Start Analyze my Cloud Composer environment 'prod-airflow' in project my-gcp-project, region us-central1, over the past 24 hours and give me tuning recommendations.

Frequently Asked Questions about gcp-managed-airflow-recommendations

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I diagnose Cloud Composer worker or scheduler restarts?▼

Run the workload_restarts.py script with your project ID, location, environment name, and query duration. It queries the composer.googleapis.com/workload/restart_count metric per component, and high restart counts typically indicate memory issues or unoptimized DAGs blocking the event loop.

How to check CPU and memory usage of Airflow workloads on GCP?▼

Use the workload_cpu_usage.py and workload_memory_usage.py scripts, which query Cloud Monitoring via PromQL and report total limits plus average and max usage per component. You can filter specific components like SCHEDULER or CELERY_WORKER with the --components flag.

What metrics indicate DAG parsing problems in Cloud Composer?▼

Key metrics are total_parse_time, parse_error_count, and processor_timeout_count under composer.googleapis.com/environment/dag_processing. High parse times often come from top-level code in DAG files, and you can inspect the dag-processor-manager logs for details.

Does this tool require gcloud authentication to fetch metrics?▼

Yes, the scripts obtain an access token via 'gcloud auth application-default print-access-token' to call the Cloud Monitoring PromQL API. You must be authenticated with a principal that has monitoring read permissions on the target project.

Why should I avoid top-level code in Airflow DAG files?▼

Top-level code runs on every DAG parse cycle, blocking the DagProcessor and increasing scheduler CPU usage. Heavy processing, database connections, or Variable lookups should be moved inside task execution or operator methods.

Related Skills

Explore More Software Engineering Skills→
kubonykubony
10

gcp-firewall

Manage Google Cloud Platform firewall rules across VPC networks using the gcloud CLI.

Community
Intermediate
rajyeole6rajyeole6

agency-studio-operations

Design standardized operating procedures and track operational metrics for studio workflows.

Community
Advanced
ilorozco11ilorozco11

airflow-dag

Automate Airflow DAG development with TaskFlow API and GCP integrations.

Community
Advanced
ssmleossmleo

gke-observability

Set up GKE logging, monitoring, and Prometheus integration.

Community
Advanced