GEIS: A Generation-Evaluation-Improvement Loop of Agent Skills for Long-Form Article Generation
Summary
This paper introduces GEIS, a structured loop for long-form article generation that decomposes the writing process into named and declarative skills. The approach emphasizes modularity, inspectability, and iterative improvement through evaluation. The method is implemented in Tasi Harness and evaluated on 20 Wikipedia Featured Articles, showing measurable improvements over baseline systems.
Mathematical/empirical assessment
The paper presents empirical results demonstrating that GEIS improves upon the default writer in Tasi Harness by 8.0 points on a 100-point PDF quality rubric. It also outperforms STORM on structural and content quality metrics. In the 20-topic improvement experiment, the patched writing skill raises the average score from 82.90 to 86.95, with most gains coming from content quality. These results are supported by detailed tables and a clear evaluation framework.
Strengths
The modular design of GEIS allows for clear separation of responsibilities, making the system more inspectable and reusable. The explicit stages of the writing process (Request, Plan, Draft, Audit, Refine, Deliver) provide a structured approach to long-form generation. The evaluation-driven improvement loop is a compelling contribution, enabling systematic refinement of writing rules based on feedback.
Concerns
While the paper demonstrates significant improvements, the results are primarily based on a single model (GPT-5.4) for generation and Qwen 3.5 Plus for evaluation, which may limit generalizability. Additionally, the improvement patches are rule-based and may not adapt well to all topics, as seen in the few cases where performance declined. Further exploration of topic-specific adaptation would strengthen the approach.
Final decision
Weak accept