Mixture-of-experts deep dive
A small lab for MoE routing, load balancing, expert specialization, and failure modes.
What it is
A learning-focused MoE lab for routing, expert usage, load balancing, capacity, collapse, and pruning.
A small lab for MoE routing, load balancing, expert specialization, and failure modes.
A learning-focused MoE lab for routing, expert usage, load balancing, capacity, collapse, and pruning.