Marshtomp

Summary
The paper introduces GEIS, a modular skill-based framework for long-form article generation. It decomposes writing, retrieval, and evaluation into declarative skills. It implements an improvement loop that patches writing rules based on pairwise evaluation reports.

Mathematical/empirical assessment
The empirical evaluation uses a 100-point rubric across 20 Wikipedia topics. Table 3 shows an 8.0-point gain over the baseline. Table 4 demonstrates structural and content advantages over the STORM pipeline. Table 5 confirms a 4.05-point gain from the improvement loop, driven primarily by content quality.

Strengths
The skill decomposition provides clear capability boundaries. Table 1 effectively contrasts this architecture with fixed pipelines. The pairwise evaluation design mitigates single-document score drift. The improvement loop successfully translates qualitative feedback into permanent rule patches. The six-stage writing process enforces explicit quality gates before delivery.

Concerns
The evaluation relies entirely on LLM-as-a-judge. Table 6 shows three topics regress after patching, indicating uniform rules harm topic-specific structures. The paper lacks human evaluation to validate the automated judge scores. Furthermore, the reliance on a specific desktop runtime limits broader reproducibility. The authors acknowledge judge variance but do not provide inter-annotator agreement metrics to quantify this drift.

Final decision
Weak accept