skills.rest
methodic-research
methodic-researchOFFICIAL
Explore all skills by this author →
GitHub

chronicle-checkpoint-resume

Snapshots training environments and uploads resumable checkpoints to Chronicle runs.

1|1|Updated May 9, 2026
Search Tags:#model training#checkpoint#reproducibility#experiment tracking#chronicle#environment snapshot#resume training
One-click install
npx skills add https://github.com/methodic-research/skills --skill chronicle-checkpoint-resume-methodic-research
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: chronicle-checkpoint-resume
Source: https://github.com/methodic-research/skills/tree/main/plugins/chronicle/skills/chronicle-checkpoint-resume
Command: npx skills add https://github.com/methodic-research/skills --skill chronicle-checkpoint-resume-methodic-research

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires methodic-research.

What problem does it solve? Training runs that crash or get preempted lose hours of compute because checkpoints and environment state were never durably recorded. This Skill makes a Chronicle run reproducible and resumable by capturing an environment snapshot at start, streaming checkpoints to Chronicle during training, and discovering the latest checkpoint to resume from after a restart. ## Core Features & Use Cases - Environment Snapshotting: Records Python version, git SHA, installed packages, GPU/CUDA details, and custom config state as JSON at run start via run.upload_environment. - Checkpoint Upload During Training: Uploads checkpoint directories asynchronously with run.upload_directory_async, tracking state in a local SQLite UploadTracker for crash recovery. - Resume Discovery: Finds the latest ready checkpoint across all runs of a variation with run.latest_output("checkpoint") (or the MCP chronicle.list_outputs fallback) and downloads it so training continues instead of restarting. - Use Case: A long PyTorch training run crashes at step 40,000. On restart, the Skill locates the last uploaded checkpoint from the prior run, downloads it, and your load_state call resumes training exactly where it stopped. ## Quick Start Ask the agent to snapshot the environment, upload checkpoints as training progresses, and resume this Chronicle run from the latest checkpoint.

Frequently Asked Questions about chronicle-checkpoint-resume

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I resume training from the latest checkpoint in Chronicle?▼

Call run.latest_output("checkpoint") to find the newest ready checkpoint across all runs of the variation, then run.download_asset(id, resume_dir) and load it with your framework's load_state. On a first run it returns None and training starts from scratch.

How do I upload checkpoints during training with the methodic SDK?▼

Write your checkpoint as a directory, then call run.upload_directory_async(dir, asset_type="checkpoint", upload_tracker=tracker). The SDK presigns URLs, uploads files directly to cloud storage, and tracks state in SQLite for crash recovery.

Does checkpoint resume work with HuggingFace Accelerate or raw PyTorch?▼

Yes, the Skill is training-agnostic. Use accelerator.save_state/load_state for HuggingFace Accelerate, or torch.save/torch.load for raw PyTorch; only those two lines are framework-specific.

Why does latest_output return None when a checkpoint exists?▼

The lookup was likely scoped to the current run instead of the variation, or the prior upload never finalized to the ready state. List outputs with across_runs=True and check each asset's state field.

What is the difference between checkpoint and snapshot asset types?▼

checkpoint marks resumable mid-training state that resume discovery reads, while snapshot marks the final saved model. Marking a final model as checkpoint causes resume to load a non-resumable artifact.

Related Skills

Explore More Education & Research Skills→
DifanaDAPDifanaDAP

weights-and-biases

Automate ML experiment tracking and hyperparameter sweeps via the Python API.

Community
Advanced
EvoClawEvoClaw
11

anti-cherry-pick

Enforce complete reporting of all seeds, results, and failures in experiments.

Official
Advanced
InverterNetworkInverterNetwork

weights-and-biases

Integrate Weights & Biases for experiment tracking and hyperparameter optimization.

Official
Advanced
HolobiomicsLabHolobiomicsLab
4

lamindb

Track data provenance for biological workflows with lineage and ontology annotations.

Official
Advanced