skills.rest
SmileAfterBurn
SmileAfterBurnCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

federate-lakehouse-catalog

Configures Google Cloud Lakehouse federated catalogs connecting BigQuery to Databricks Unity and AWS Glue.

Updated Aug 26, 2026
Search Tags:#bigquery#iceberg#lakehouse#cross-cloud#aws glue#federated catalog#databricks unity
One-click install
npx skills add https://github.com/SmileAfterBurn/pani-dumka-ai --skill federate-lakehouse-catalog-smileafterburn
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: federate-lakehouse-catalog
Source: https://github.com/SmileAfterBurn/pani-dumka-ai/tree/main/.gemini/skills/federate-lakehouse-catalog
Command: npx skills add https://github.com/SmileAfterBurn/pani-dumka-ai --skill federate-lakehouse-catalog-smileafterburn

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Organizations with data stored in remote catalogs like Databricks Unity or AWS Glue cannot query that data from Google Cloud engines without complex data migration. This Skill guides the setup of cross-cloud federated catalogs so remote Iceberg data becomes queryable directly from BigQuery and Spark on GCP. ## Core Features & Use Cases - Databricks Unity Federation: Creates regional Secret Manager credentials and a federated BigLake Iceberg catalog pointing to a Databricks Unity instance with automatic metadata refresh. - AWS Glue Federation: Walks through IAM role creation with placeholder trust policies, scoped Glue and S3 permissions, catalog creation, and trust policy updates using the provisioned Google service account. - Region Pairing Guidance: Recommends optimal GCP regions paired with AWS regions to minimize cross-cloud query latency. - Validation and Troubleshooting: Verifies the federation by listing namespaces via the bq CLI and proposes meaningful analytical queries on the discovered data. - Use Case: A data team keeps tables in AWS Glue on S3 but wants analysts to run BigQuery SQL against them without copying data; this Skill sets up the federated catalog end to end. ## Quick Start Ask the agent to federate your lakehouse catalog to Databricks Unity or AWS Glue so you can query the remote data from BigQuery.

Frequently Asked Questions about federate-lakehouse-catalog

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I query Databricks Unity data from BigQuery?▼

Create a federated BigLake Iceberg catalog of type unity pointing to your Databricks instance, storing the OAuth client ID and secret in a regional Secret Manager secret. Once created and granted secret access, tables appear queryable through standard BigQuery SQL.

How to connect AWS Glue Data Catalog to Google Cloud BigQuery?▼

Create an AWS IAM role with a placeholder trust policy, attach Glue and S3 read permissions, then create a federated BigLake catalog using your AWS account ID as the glue-warehouse. Finally update the trust policy with the provisioned Google service account ID and enable background refresh.

Which GCP region should I pair with my AWS region for federation?▼

Choose GCP regions with low-latency dedicated connections to your AWS region. For example, AWS us-east-1 pairs best with GCP us-east4, and AWS eu-central-1 pairs with europe-west3, minimizing cross-cloud query latency.

Why does my federated catalog return permission errors or empty results?▼

Common causes include the Secret Manager secret and catalog being in different regions, the catalog service account lacking secretAccessor role, or an AWS trust policy not referencing the correct biglake-service-account-id. Verify each and consult the cross-cloud troubleshooting documentation.

Can this skill run Databricks SQL queries or manage AWS Glue jobs?▼

No. This Skill only sets up federated catalog connections so remote metadata and data become queryable from GCP engines. It does not execute remote SQL on Databricks or manage Databricks clusters and AWS Glue jobs.

Related Skills

Explore More Data & Analytics Skills→
gemini-cli-extensionsgemini-cli-extensions
161

federate-lakehouse-catalog

Federate Google Cloud Lakehouse catalogs to remote Iceberg catalogs.

Official
Advanced
subhamay-bhattacharyyasubhamay-bhattacharyya

aws-config-iam-policies

Create and validate AWS IAM policies for Snowflake data lake access.

Official
Intermediate
BAIZ1DBAIZ1D

federate-lakehouse-catalog

Configure cross-cloud lakehouse catalog federation for GCP analytics engines.

Community
Advanced
Sea-GoSea-Go
98

review-phase-level

Validate phase-level climate, geographic, and budget constraints with structured scores.

Official
Intermediate