skills.rest
obispobruno
obispobrunoCOMMUNITY
Explore all skills by this author →
GitHub

creating-data-lake-table

Create managed Iceberg tables on Amazon S3 Tables with Glue catalog registration and IAM access control.

1|Updated Aug 1, 2024
Search Tags:#aws#iceberg#athena#data lake#table creation#glue catalog#s3 tables
One-click install
npx skills add https://github.com/obispobruno/dotfiles --skill creating-data-lake-table-obispobruno
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: creating-data-lake-table
Source: https://github.com/obispobruno/dotfiles/tree/main/dot_agents/skills/creating-data-lake-table
Command: npx skills add https://github.com/obispobruno/dotfiles --skill creating-data-lake-table-obispobruno

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve? Setting up analytics-ready data lake tables on AWS requires coordinating table buckets, namespaces, Iceberg schemas, Glue catalog federation, and IAM permissions across multiple services, where small mistakes like mixed-case names or wrong ARN patterns cause opaque failures. ## Core Features & Use Cases - Managed Iceberg Table Creation: Creates table buckets, namespaces, and Iceberg tables via the S3 Tables API with partitioning and schema validation. - Glue Catalog Integration: Registers the federated s3tablescatalog so tables are queryable from Athena and other Iceberg-compatible engines. - Access Control Setup: Configures least-privilege s3tables and glue IAM permissions with correct resource ARN scoping. - Use Case: A data engineer needs a partitioned customer_orders table queryable in Athena. The skill checks for existing tables, creates the bucket and namespace, registers the Glue catalog, applies IAM policies, and verifies the table with a DESCRIBE query. ## Quick Start Ask the AI to create an S3 Tables Iceberg table named customer_orders with order_date, customer_id, and amount columns partitioned by month in your AWS account.

Frequently Asked Questions about creating-data-lake-table

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I create an Iceberg table with Amazon S3 Tables?▼

Create a table bucket with aws s3tables create-table-bucket, add a namespace, then run aws s3tables create-table with Iceberg schema metadata JSON. Register the s3tablescatalog federated catalog in Glue so the table is queryable from Athena.

S3 Tables API vs Athena DDL for creating Iceberg tables?▼

The S3 Tables API is the default path and works for any user. Athena DDL is an alternative when you specifically want SQL statements or need ALTER TABLE schema evolution later. Both require the s3tablescatalog Glue catalog to exist first.

Why does my S3 Tables table not appear in Athena?▼

The table is invisible in Athena when the s3tablescatalog federated catalog is missing from Glue. Verify with aws glue get-catalog --catalog-id s3tablescatalog, and confirm queries pass the catalog in --query-execution-context rather than in the SQL statement.

Why do I get AccessDeniedException with an s3:* IAM policy?▼

S3 Tables uses its own s3tables:* IAM action namespace, not s3:*. The querying principal needs actions like s3tables:GetTable and s3tables:GetTableData on the table bucket ARN, plus glue:GetCatalog, glue:GetDatabase, and glue:GetTable.

Can I create S3 Tables tables from a Glue ETL Spark job?▼

Yes, using Glue 5.1 or higher with the --datalake-formats iceberg job argument. All spark.sql.catalog configuration must go in the --conf job argument because Glue 5.x treats catalog configs as static, and spark.conf.set() throws an AnalysisException.

What naming rules apply to S3 Tables buckets and tables?▼

All names must be lowercase because Glue rejects mixed case with GENERIC_INTERNAL_ERROR. Table buckets allow hyphens and must be 3-63 characters, while namespaces and table names must use lowercase with underscores and no hyphens.

Related Skills

Explore More Data & Analytics Skills→
awsaws
2.5k

finding-data-lake-assets

Resolves data lake asset references across Glue Data Catalog, S3, S3 Tables, and Redshift.

Official
Advanced
awsaws
2.5k

ingesting-into-data-lake

Import data from S3, JDBC databases, Snowflake, BigQuery, and DynamoDB into AWS data lake tables.

Official
Advanced
subhamay-bhattacharyyasubhamay-bhattacharyya

aws-config-iam-policies

Create and validate AWS IAM policies for Snowflake data lake access.

Official
Intermediate
mreferremreferre

creating-data-lake-table

Create managed Iceberg tables on Amazon S3 Tables with Glue catalog registration.

Community
Advanced