#mixture-of-depths

Mixture-of-Depths: What Happens When a Transformer Skips Its Own Layers

Learn how Mixture-of-Depths lets transformers skip layers per token, cutting inference FLOPs. Compare MoD vs. MoE and early exit, plus routing and capacity tips.