skills.rest
tottenjordan
tottenjordanCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

bigquery-data-transfer-service

Discovers and inspects BigQuery Data Transfer Service configurations for ingestion pipelines.

1|Updated Mar 19, 2026
Search Tags:#gcp#data pipeline#data ingestion#etl#gcloud#bigquery#data transfer service
One-click install
npx skills add https://github.com/tottenjordan/me-skittles --skill bigquery-data-transfer-service-tottenjordan
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: bigquery-data-transfer-service
Source: https://github.com/tottenjordan/me-skittles/tree/main/gemini/bigquery-data-transfer-service
Command: npx skills add https://github.com/tottenjordan/me-skittles --skill bigquery-data-transfer-service-tottenjordan

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve? When building data pipelines on Google Cloud, teams often don't know whether a BigQuery Data Transfer Service (DTS) transfer already exists for their data source, or which parameters a new transfer requires. This Skill discovers existing DTS configurations, extracts transfer metadata, and guides the declarative provisioning of new transfers. ## Core Features & Use Cases - Transfer Discovery: Lists existing transfer configs in a project and region, checks run history, and identifies disabled or duplicate transfers. - Data Source Parameter Inspection: Uses a REST API script to enumerate available data source IDs and retrieve required parameters for any data source, including OAuth authorization URL generation for Google data sources. - Declarative Provisioning Enforcement: Ensures all new transfers are created through a deployment.yaml resource provisioning framework rather than imperative CLI commands, with mandatory user confirmation of parameters. - Use Case: A user asks to ingest Google Ads data into BigQuery. The Skill checks for an existing transfer, discovers the required data source parameters, collects user-confirmed values and OAuth authorization, and verifies a successful transfer run before downstream pipeline work. ## Quick Start Ask the agent to check whether a BigQuery Data Transfer Service transfer already exists for your data source in your GCP project and region.

Frequently Asked Questions about bigquery-data-transfer-service

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I check for existing BigQuery Data Transfer Service transfers?▼

Run bq ls --transfer_config with the target transfer location and project ID to list existing transfers. Then use bq ls --transfer_run on a specific config to verify it has at least one successful run before reusing it.

How do I find required parameters for a BigQuery DTS data source?▼

Run the bigquery_dts.py discovery script with your project ID, data source ID, and region. It calls the BigQuery Data Transfer REST API and returns the full parameter specification, and can list available data source IDs when none is given.

Can I create BigQuery DTS transfers with bq mk or gcloud commands?▼

No. New transfer configurations must be provisioned declaratively through the gcp pipeline resource provisioning framework using a generated deployment.yaml. CLI commands are permitted only for discovery, listing, showing, and triggering manual runs.

Does BigQuery DTS support Google Ads and YouTube data sources?▼

Yes, Google ecosystem data sources like Google Ads and YouTube are supported, but they require OAuth authorization when using end user credentials. The discovery script generates an authorization URL, and the returned versionInfo code goes into deployment.yaml.

What should I do if a BigQuery DTS transfer run takes too long?▼

Poll the run status every 30-60 seconds for up to 5 minutes. If it is still in progress after that, stop and ask the user to confirm once ingestion finishes before continuing to build the rest of the pipeline.

Related Skills

Explore More Data & Analytics Skills→
interflownexinterflownex
2

bigquery-data-transfer-service

List and inspect BigQuery Data Transfer Service configurations across GCP projects.

Community
Advanced
1919114514yasenpai-maker1919114514yasenpai-maker

data-autocleaning

Profiles and cleans BigQuery and GCS data using Dataplex scans and SQL transformations.

Community
Advanced
gemini-cli-extensionsgemini-cli-extensions
161

gcp-pipeline-resource-provisioning

Provision GCP data pipeline resources from a deployment.yaml.

Official
Advanced
1919114514yasenpai-maker1919114514yasenpai-maker

bigquery-data-transfer-service

Discovers and inspects BigQuery Data Transfer Service configurations for data ingestion pipelines.

Community
Advanced