skills.rest
tottenjordan
tottenjordanCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

federate-lakehouse-catalog

Configures Google Cloud Lakehouse federated catalogs connecting BigQuery to Databricks Unity and AWS Glue.

1|Updated Mar 19, 2026
Search Tags:#bigquery#iceberg#lakehouse#cross-cloud#aws glue#federated catalog#databricks unity
One-click install
npx skills add https://github.com/tottenjordan/me-skittles --skill federate-lakehouse-catalog-tottenjordan
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: federate-lakehouse-catalog
Source: https://github.com/tottenjordan/me-skittles/tree/main/gemini/federate-lakehouse-catalog
Command: npx skills add https://github.com/tottenjordan/me-skittles --skill federate-lakehouse-catalog-tottenjordan

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Querying data stored in remote catalogs like Databricks Unity or AWS Glue from Google Cloud normally requires complex cross-cloud setup involving secrets, IAM roles, and trust policies. This Skill guides the agent through the entire federation workflow so remote Iceberg data becomes queryable from BigQuery and Spark without manual trial and error. ## Core Features & Use Cases - Databricks Unity Federation: Creates regional Secret Manager secrets for OAuth credentials and provisions a federated BigLake Iceberg catalog pointing at Unity Catalog. - AWS Glue Federation: Builds AWS IAM roles with placeholder trust policies, attaches scoped Glue and S3 read permissions, and finalizes the trust relationship using the catalog's Google service account ID. - Region Pairing Guidance: Recommends optimal GCP regions paired with AWS regions to minimize cross-cloud query latency. - Use Case: A data engineer wants to run BigQuery analytics over tables managed in AWS Glue. The Skill walks through IAM role creation, catalog provisioning, trust policy updates, and validation until queries return results. ## Quick Start Ask the agent to federate your lakehouse catalog to Databricks or AWS Glue so you can query the remote data from BigQuery.

Frequently Asked Questions about federate-lakehouse-catalog

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I query Databricks data from BigQuery?▼

Create a federated BigLake Iceberg catalog of type unity pointing at your Databricks workspace, storing the OAuth client ID and secret in a regional Secret Manager secret. Once created, query tables directly with standard BigQuery SQL using the catalog name as a prefix.

How to connect AWS Glue Data Catalog to Google Cloud BigQuery?▼

Create an AWS IAM role with a placeholder trust policy, attach Glue and S3 read permissions, then create a federated BigLake catalog passing your AWS account ID as the glue-warehouse. Finally update the trust policy with the catalog's Google service account ID and enable background refresh.

Which GCP region should I pair with my AWS region for lakehouse federation?▼

Choose GCP regions with low-latency dedicated or partner cross-cloud interconnect to your AWS region. For example, AWS us-east-1 pairs best with GCP us-east4, and AWS eu-west-2 pairs with GCP europe-west2.

Why does my federated catalog fail to list namespaces after creation?▼

Common causes include the Secret Manager secret being in a different region than the catalog, missing secretAccessor permissions for the catalog service account, or an AWS trust policy that does not reference the correct biglake-service-account-id. Verify each of these and consult the cross-cloud lakehouse troubleshooting documentation.

Can this approach run Databricks SQL or manage Databricks clusters?▼

No. This federation only makes remote catalog metadata and Iceberg table data queryable from GCP engines like BigQuery and Spark. It does not support direct remote SQL execution on Databricks or managing remote clusters and AWS Glue jobs.

Related Skills

Explore More Data & Analytics Skills→
gemini-cli-extensionsgemini-cli-extensions
161

federate-lakehouse-catalog

Federate Google Cloud Lakehouse catalogs to remote Iceberg catalogs.

Official
Advanced
1919114514yasenpai-maker1919114514yasenpai-maker

federate-lakehouse-catalog

Configures Google Cloud Lakehouse federated catalogs connecting BigQuery to Databricks Unity and AWS Glue.

Community
Advanced
fuzzy-dynamicsfuzzy-dynamics
1

market-research-reports

Generate 50+ page market research reports with LaTeX formatting and multi-framework analysis.

Official
Advanced
robomotioniorobomotionio
2

openweather

Fetch current weather, forecasts, air quality, and historical data from OpenWeatherMap.

Official
Advanced