mhc

What problem does it solve?

Deep Transformer residual connections can become unstable in very deep architectures; mHC uses manifold-constrained doubling to preserve signal flow and improve training stability.

Core Features & Use Cases

• Applies a learned pre- and post-mapping to mix streams while preserving stable residuals.
• Uses Sinkhorn normalization to constrain the residual mapping, enabling scalable deep models.
• Use cases include training very deep Transformer models where gradient vanishing and representation collapse are concerns.

Quick Start

Apply mHC to stabilize very deep transformer networks by projecting the residual mapping onto the doubly stochastic manifold and using pre/post streams for controlled mixing.