datalad

Retrieve, version, and publish scientific datasets with DataLad and git-annex provenance tracking.

46.6k|4.2k|Updated Oct 19, 2025
One-click install
npx skills add https://github.com/K-Dense-AI/claude-scientific-skills --skill datalad-k-dense-ai
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: datalad
Source: https://github.com/K-Dense-AI/claude-scientific-skills/tree/main/skills/datalad
Command: npx skills add https://github.com/K-Dense-AI/claude-scientific-skills --skill datalad-k-dense-ai

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires datalad, git-annex, datalad-container, and includes references (resource) components.

What problem does it solve? Large scientific datasets cannot live in plain Git, and analyses often lack a machine-readable record of how outputs were produced. This Skill manages dataset retrieval, versioning, publication, and computational provenance using DataLad and git-annex. ## Core Features & Use Cases - On-demand data retrieval: Clone datasets from OpenNeuro, DANDI, or datasets.datalad.org in seconds and fetch file content selectively with datalad get, since clones contain pointers rather than data. - Computational provenance: Record commands, inputs, and outputs with datalad run, re-execute them with datalad rerun, and capture software environments with datalad containers-run. - Two-target publishing: Publish Git history to hosting siblings and annexed content to storage remotes such as S3, with --publish-depends preventing broken publications. - Use Case: Clone a BIDS dataset from OpenNeuro, fetch only the subjects you need, run an FSL brain extraction inside a registered container, and publish the derivatives with a complete re-executable provenance record. ## Quick Start Use the datalad skill to clone the OpenNeuro dataset ds000001, fetch the anatomical files for sub-01, and record a brain-mask extraction as a provenance-tracked run.

Frequently Asked Questions about datalad

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I download data from a DataLad dataset after cloning?▼

Run datalad get on the paths you need, since a fresh clone contains only symlinks or pointer files, not content. Use datalad get -r . for everything including subdatasets, or datalad get -n -r . to install subdataset structure without file content.

When should I use DataLad instead of plain Git?▼

Use DataLad when files are too large for Git, data lives in multiple locations, analyses must be re-executable, or you consume published datasets from OpenNeuro or DANDI. Use plain Git for code-and-text repositories where everything fits comfortably and no partial checkouts are needed.

Why does a file in my cloned dataset read as empty or a broken symlink?▼

The file is a git-annex pointer, not corrupted data; its content has not been retrieved yet. Run datalad get on the path before reading it, and treat file existence as insufficient evidence that content is present locally.

How do I make a DataLad analysis reproducible with containers?▼

Register an image once with datalad containers-add using a docker:// or shub:// URL, then execute with datalad containers-run declaring inputs and outputs. The image is tracked in the dataset, so datalad rerun can re-execute the exact recorded command and environment.

Why do collaborators fail to get files after I push my dataset?▼

The Git history was published without the annexed content, or the default --data auto-if-wanted transferred nothing because the remote has no preferred-content settings. Create a storage sibling with git annex initremote, set --publish-depends on the Git sibling, and push with --data anything.

Why does datalad run refuse to start?▼

datalad run refuses when the dataset has unsaved modifications, because a record built on an unknown starting state is unreliable. Save or discard changes first with datalad save, or pass --explicit to declare the listed inputs and outputs as the complete scope.