https://iseekyan.github.io/tags/moe/Recent content in Moe on Yan BaiHugoen-usMon, 18 May 2026 21:50:00 +0800https://iseekyan.github.io/posts/qwen35-long-sequence-moe-rl/Mon, 18 May 2026 21:50:00 +0800https://iseekyan.github.io/posts/qwen35-long-sequence-moe-rl/<p>Long-context training is no longer a pretraining-only problem. In RL, prompts, rollouts, rewards, tool calls, and multi-turn interactions all push context length upward. The question many teams care about has also changed: can they start with a modest number of GPUs and reliably run 128K, 256K, or even longer-context algorithm experiments?</p> <p>RL uses resources differently from pretraining. Pretraining usually optimizes for large-scale, stable, long-running throughput. RL often needs many experiments in parallel. When users have more GPUs, they usually want to run several reward, rollout, data-mixture, or hyperparameter experiments at the same time, instead of putting every GPU into one training job. A long-context RL training system first needs a low-friction, reusable starting point with few tuning requirements.</p>