machine-learning

Define the training step as a pure function and decorate it with jax.jit, replacing Python loops with jax.lax.scan or jax.lax.fori_loop. Avoid side effects and Python control flow inside jitted code, since they break tracing or silently produce wrong results.

JAX uses a functional random API where you explicitly create and split PRNG keys instead of relying on global state. Split keys before each use and never reuse a key, otherwise random values become correlated or identical.

Both Flax and Haiku provide neural network layers on top of JAX while keeping models as pure functions with explicit parameter pytrees. Choose based on your team's existing codebase, since both integrate with jax.jit, jax.vmap, and jax.grad.

Python control flow is evaluated once at trace time, so data-dependent branches are frozen into the compiled graph. Replace conditionals with jax.lax.cond and loops with jax.lax.scan or jax.lax.fori_loop for correct dynamic behavior.

Use gradient checkpointing to trade compute for memory, choose smaller dtypes where acceptable, and batch operations to improve utilization. Profile with the JAX profiler to find hot paths before optimizing.