Kuaishou Technology’s large language model team, in collaboration with the University of Chinese Academy of Sciences, has open-sourced GoLongRL, a comprehensive post-training framework designed to solve critical performance degradation in artificial intelligence models processing exceptionally long text sequences.
Current long-context reinforcement learning techniques suffer from highly homogenous training data that focuses almost exclusively on locating specific data points within long essays. This narrow approach leaves models unequipped to handle complex structural text duties like sorting, abstract summaries, or multi-hop logical reasoning. To address this limitation, the Chinese research team released a fully open-source system that includes a high-utility dataset of nearly 23,000 heterogeneous samples, complete training source codes, and a specialized machine learning optimization algorithm.
The dataset features 22,965 precisely cataloged samples structured across nine distinct core task types to comprehensively train long-context comprehension. Rather than relying on synthetic templated inputs, which often teach models to rely on superficial paragraph boundaries, the dataset prioritizes genuine source materials, including literature from Project Gutenberg, academic preprints, legal documents, and corporate financial filings. For domains lacking labeled data, the pipeline synthesizes only the question-and-answer pairs based on raw inputs, ensuring high-fidelity data integrity.
To maximize learning efficiency across varied tasks like sorting, data extraction, and abstract generation, the researchers abandoned traditional single-metric reward functions in favor of localized evaluation scripts. Since text summaries rely on semantic overlaps while ordering sequences depends on ranking coefficients, the system assigns unique evaluation criteria tailored to each task's structural objective.
Managing these varied reward functions introduced complex numerical scaling variances during standard training runs. To stabilize the optimization process, the researchers developed an algorithm named TMN-Reweight. This math-based framework decouples numerical reward scaling from task difficulty corrections, preventing high-variance signals from disrupting model training.
The framework demonstrated immediate performance gains during empirical evaluation. When applied to a small four-billion-parameter base, the data and algorithm configuration surpassed specialized competing long-context models by a notable margin.
Scaling the framework up to a larger thirty-billion-parameter architecture yielded even more substantial results. The resulting model achieved a top evaluation metric score, outperforming elite, closed-source foundation models including DeepSeek-R1, Alibaba’s large-scale Qwen reasoning systems, and Google’s Gemini Flash framework.
Importantly, the reinforcement learning process did not trigger negative capabilities transfer regarding general analytical reasoning. The models demonstrated minor, steady improvements on mainstream intelligence benchmarks while displaying strong capabilities transfer into entirely novel fields, such as agentic memory and multi-turn conversational dialogue recollection.
The framework also displayed significant sequence length extrapolation capabilities. Although the model was trained on a hard maximum limit of 160,000 text tokens, its core synthesis and data retrieval capabilities successfully generalized out to processing blocks containing up to one million text tokens, proving that the learned processing techniques are length-agnostic.