skills.rest
flagos-ai
flagos-aiOFFICIAL
Explore all skills by this author →
WebsiteGitHub

cuda-op-integration

Generates CUDA boxing operators from PyTorch schemas for the torch_fl flagos backend.

12|18|Updated Apr 7, 2026
Search Tags:#code generation#PyTorch#cuda#libtorch#operator dispatch#torchgen#backend porting
One-click install
npx skills add https://github.com/flagos-ai/Torch-FL --skill cuda-op-integration-flagos-ai
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: cuda-op-integration
Source: https://github.com/flagos-ai/Torch-FL/tree/main/.claude/skills/cuda-op-integration
Command: npx skills add https://github.com/flagos-ai/Torch-FL --skill cuda-op-integration-flagos-ai

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Porting the torch_fl NVIDIA backend to a new PyTorch version normally means hand-writing and fixing dozens of CUDA kernels, and import torch_fl crashes with opaque "Mismatch in kernel C++ signatures" errors when dispatcher signatures drift between torch versions. This Skill replaces hand-written kernels with torchgen-based code generation and provides a tested procedure for adapting the CUDA-boxing backend to any torch branch. ## Core Features & Use Cases - Schema-driven operator generation: Runs scripts/codegen/codegen_ops.py against native_functions.yaml to emit all boxing kernels, dispatcher glue, and registration code into csrc/aten/generated/. - External libtorch_cuda.so scheme: Keeps the pip environment on CPU-only torch while resolving CUDA symbols at runtime via LD_PRELOAD, so the backend builds with g++ only and no nvcc. - Version-adaptation troubleshooting: Documents the per-operator IListRef/ArrayRef signature split, factory-op recursion fixes, LD_PRELOAD timing constraints, and test-marker pitfalls. - Use Case: When bumping torch_fl from torch 2.12 to 2.13, follow the documented loop — generate, build with FlagGems off, smoke-test through the wrapper script, and iterate on signature mismatches until import is clean. ## Quick Start Ask the AI to adapt the torch_fl CUDA backend to a new PyTorch version by running the codegen script, rebuilding, and fixing any kernel signature mismatches until import succeeds.

Frequently Asked Questions about cuda-op-integration

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I port a PyTorch PrivateUse1 backend to a new torch version?▼

Run the torchgen-based codegen script against the target version's native_functions.yaml, rebuild with FlagGems disabled, then smoke-test through the LD_PRELOAD wrapper. Iterate on any dispatcher signature mismatches until import is clean.

How to fix "Mismatch in kernel C++ signatures" when importing a PyTorch plugin?▼

The error means your registered kernel's TensorList type differs from what another dispatch key registered. Read both kernels in the message: if PyTorch wants ArrayRef, add the op to the ARRAYREF_OPS set; if it wants IListRef, remove it, then regenerate and rebuild.

Can I build a CUDA PyTorch extension without nvcc or CUDA torch installed?▼

Yes, using the external libtorch_cuda.so scheme: install CPU-only pip torch, download the matching cu130 wheel to extract libtorch_cuda.so, and LD_PRELOAD it before importing torch. The extension compiles with g++ only and links against torch_cpu_library.

Why does my PyTorch factory op like arange cause infinite recursion?▼

Factory ops that compute values dispatch back into your own PrivateUse1 kernel when the device defaults to it, causing a stack overflow. Build the tensor on the CUDA device so it hits the external kernel, then unbox the result back to flagos.

Why must libtorch_cuda.so load before importing torch?▼

PyTorch caches stub CUDAHooks at the first import torch call, so loading libtorch_cuda.so afterward causes "Cannot initialize CUDA without ATen_cuda library" errors. Use an LD_PRELOAD wrapper script rather than a late ctypes.CDLL call.

What are the limitations of the external libtorch CUDA approach?▼

The libtorch_cuda.so version must match the installed torch bit-for-bit or you get ABI corruption. Also, the first CUDA op in a fresh process may hit an uninitialized caching allocator error on out-variant ops, affecting a couple of dispatch-log tests.

Related Skills

Explore More Software Engineering Skills→
Fudan-SMI-labFudan-SMI-lab
39

torch-npu-venv-setup-cpu-base

Pin CPU-base PyTorch and install matching torch-npu and torchvision.

Official
Intermediate
TarunTeja44TarunTeja44
1

ship

Automates merging, testing, versioning, and pull request creation for Git branches.

Community
Advanced
Wyl-cmdWyl-cmd
6

worktree-status

Audit git worktrees for dirty state and merge status before cleanup.

Community
Intermediate
mindspore-aimindspore-ai
258

triton-cuda-examples-torch

Integrate Triton CUDA kernels with PyTorch models for custom operators.

Official
Intermediate