Daily AI Digest — 2026-08-28
arXiv Highlights
PAWBench: How Far Are We from Probabilistically Aligned World Modeling?
Problem
Video generators are increasingly pitched as world models, but standard evaluations grade single rollouts on visual plausibility. For any stochastic process — a coin toss, a die roll, a ball bouncing off a corner — a world model that always produces the same terminal state is wrong in a way single-sample metrics cannot detect. The paper formalizes this failure mode as a lack of probabilistic alignment: given initial observation x and action a, the model’s conditional P_M(\tau \mid x,a) should match the true distribution over futures both in support and in mass.
Method
PAWBench evaluates a model M by drawing K=50 i.i.d. rollouts under a fixed (x,a) pair, mapping each rollout to a terminal outcome in a discrete set \mathcal{Y}, and forming an empirical distribution \hat p_M over \mathcal{Y}. Fixing both the source image and the action prompt is what makes rollout variance attributable to the model’s induced distribution.
The 50 scenarios split into two tracks (Fig. 2):
- PAW-Calibration (25 scenarios): tosses, rotations, routing, and drawing scenarios where a reference distribution q over \mathcal{Y} is analytically specified (uniform for symmetric dice, prescribed for weighted spinners, etc.). Scored by total variation distance \mathrm{TVD}(\hat p_M, q).
- PAW-Coverage (25 scenarios): collisions, stability, agent interactions, material transitions. Valid outcomes are enumerable but exact probabilities are not; scored by fraction of \mathcal{Y} that appears at least once across K rollouts.
Separating the two tracks is the key design choice: calibration measures misallocated mass (every die roll looks plausible, but 4 comes up 80% of the time), while coverage measures missing modes (the die never lands on 5 or 6). A single scalar cannot distinguish these.
Outcomes are extracted by PAWEval, a rubric-based VLM judge that reads each rollout and emits an in-schema label (e.g., Head / Tail). A scene passes the outcome-readout gate — Scene Pass Rate (SPR) — only if at least 20 of 50 rollouts yield a readable label; the SPR statistic prevents a model from scoring well by producing unparseable videos.
Results
Eleven current systems — HappyHorse, Veo 3.1 Fast, Kling 3 Std., Seedance 2, Wan 2.2/2.7, LTX-2.3/2.5, Cosmos 3 Super I2V, LingBot-Video-MoE, MiniMax H3 — are evaluated under default inference settings.
No system consistently matches reference probabilities while covering the valid support. Reading off Table 3 (base rows):
- Wan2.2: Calibration TVD \times 100 = 26.3 (SPR 64%); Coverage 63.4% (SPR 92%).
- Cosmos 3 Super I2V: TVD 20.5 (SPR 80%); Coverage 55.2%.
- MiniMax H3: TVD 24.2 (SPR 68%); Coverage 48.7%.
- LTX-2.5: TVD 30.2 (SPR 60%); Coverage 57.4%.
For calibrated scenarios, a uniform-random baseline over the outcome set gives \mathrm{TVD}\times 100 = 0 against a uniform reference; models sit at 20–40, meaning they concentrate mass sharply on one or two futures. Coverage in the 50–65% range indicates that between one-third and one-half of qualitatively distinct valid outcomes never appear in 50 rollouts.
Interventions: prompt, noise, weights
The paper probes three levers for improving alignment. On the prompt side, five VLMs are queried directly (no video) as distribution samplers over \mathcal{Y} (Table 2). GLM-5V Turbo achieves TVD 34.8 on calibration and 39.9% coverage; GPT-5.5 sits at 42.3 / 34.3. VLMs themselves are miscalibrated over physical outcomes, so using them as steering controllers has a ceiling.
Table 3 makes this concrete. Attaching GPT-5.5 as a PE controller that names a target outcome per rollout gives mixed results: Wan2.2 calibration TVD goes from 26.3 to 27.1 (worse); Cosmos degrades from 20.5 to 31.6; MiniMax H3 from 24.2 to 38.9. Coverage sometimes improves (Cosmos 55.2 → 64.9) but the controller cannot fix distributional structure the base model does not natively produce.
The Oracle PE condition (feeding the ground-truth reference distribution to the controller) is the informative upper bound. Wan2.2 drops to TVD 15.3 and coverage 76.2%; Cosmos to 12.8 / 87.3%; MiniMax H3 to 10.8 / 82.9%. So when told exactly what mixture to sample, generators can hit roughly half the residual TVD gap, but even oracle-guided prompting leaves 10–20 TVD points and imperfect coverage. This isolates the remaining error as generation-side — the model cannot realize the requested future faithfully — rather than controller-side.
Limitations
Terminal-outcome labels collapse trajectory information; a model that produces the right final state via impossible dynamics still scores well. The outcome-readout gate discards scenes with too many unreadable rollouts, so SPR and TVD/Coverage must be read jointly. PAW-Coverage lacks reference probabilities, so a model that emits every outcome once but wildly miscalibrates mass still gets full coverage credit. K=50 is small for outcome sets with |\mathcal{Y}| > 6, which affects coverage estimates at the tail.
Why this matters
If video generators are to serve as world models for planning or counterfactual reasoning, matching the marginal distribution over futures is a prerequisite, not a nice-to-have. PAWBench provides the first controlled measurement showing that current systems — proprietary and open — fail this test by 20–40 TVD points and miss 35–50% of valid modes, and that inference-time prompt steering does not close the gap even under oracle supervision.
Source: https://arxiv.org/abs/2608.27345
TTPO: Test-Time Policy Optimization
Problem
Post-training pipelines for math reasoning (RL with verifiable rewards, on-policy self-distillation) require ground-truth labels, which precludes their use at test time. The obvious label-free substitute — majority-vote pseudo-labels over K sampled trajectories — is brittle: whenever the plurality answer is wrong, every trajectory agreeing with the vote is treated as a positive teacher signal, and the model is pushed toward a corrupted target token-by-token. On hard competition benchmarks (AIME, HMMT, BRUMO), pseudo-label accuracy is low enough that this failure is common, not rare, so a naive TTT loop degrades the model.
TTPO builds on an empirical asymmetry: while the positive set (rollouts matching \hat{a}) is only as reliable as the vote itself, the negative set (rollouts disagreeing with \hat{a}) is almost always genuinely wrong — even when \hat{a} is wrong, most disagreeing rollouts pick some other wrong answer.
The left panel of the figure quantifies this: during Qwen3-1.7B TTT on AIME 2026, disagreeing rollouts are wrong at a rate that dominates the pseudo-label error rate. This licenses an asymmetric objective that trusts negatives strongly and positives weakly.
Method
For each unlabeled test problem x, sample K trajectories \{y_k\} \sim \pi_\theta(\cdot\mid x), extract a_k = \operatorname{Extract}(y_k), cluster by mathematical equivalence, and take the largest cluster’s answer as \hat{a} with consensus count c. Partition into
\mathcal{P} = \{k : a_k \equiv \hat{a}\}, \qquad \mathcal{N} = \{k : a_k \not\equiv \hat{a}\}.
TTPO applies two different losses to the two branches.
Positive branch — OPSD with token weighting. Trajectories in \mathcal{P} are used as pseudo-teachers via a per-token forward KL from a frozen teacher (a snapshot of \pi_\theta) to \pi_\theta:
\mathcal{L}_{\text{OPSD}} = \sum_{k\in\mathcal{P}} \sum_{t} w_{k,t}\, \mathrm{KL}\!\left(\pi_{\text{teacher}}(\cdot\mid y_{k,<t},x)\,\|\,\pi_\theta(\cdot\mid y_{k,<t},x)\right).
The token weight w_{k,t} down-weights positions where the student is already confident and agrees with the teacher — i.e., “converged” tokens — so that if the pseudo-label is wrong, the loss touches mostly the still-uncertain tokens rather than reinforcing every position of a bad trajectory. This limits the damage of an incorrect \hat{a}.
Negative branch — Grouped RL with token masking. Trajectories in \mathcal{N} receive a GRPO-style penalty using group-relative advantages computed within the batch of K rollouts. Crucially, the negative gradient is masked to anomalous tokens: positions where \pi_\theta was confidently wrong (high probability on a token that lies on a divergent-from-consensus reasoning path). Non-confident tokens on a wrong trajectory are not penalized, since they represent exploratory uncertainty rather than a learned error.
Combining the two, the overall test-time update is
\mathcal{L}_{\text{TTPO}} = \mathcal{L}_{\text{OPSD}}^{\mathcal{P}} + \lambda \, \mathcal{L}_{\text{GRPO}}^{\mathcal{N}},
with \lambda balancing distillation and rejection. Two properties make this stable under noisy \hat{a}: (i) the positive branch’s token weighting means agreement-with-a-wrong-vote only nudges uncertain tokens; (ii) the negative branch is well-grounded because the “wrong-answer set” is reliably wrong regardless of \hat{a}’s correctness. As training progresses, majority voting becomes sharper (the model concentrates mass on fewer candidates), which tightens the self-supervision signal — a positive feedback loop that does not require external labels.
Setup and Results
Experiments use Qwen3-1.7B / 4B / 8B with LoRA (r=64, \alpha=128) on all linear layers. Two regimes are evaluated:
- OpenThoughts setting: models trained on labeled data; TTPO ignores labels and is compared to label-supervised OPSD.
- TTT setting: training happens directly on the (unlabeled) test set.
The headline claim is that TTPO, without any labels, matches label-supervised OPSD across five competition-level benchmarks. The rightmost panel of the figure shows Qwen3-1.7B averaged over AIME 2026, HMMT 2026, and BRUMO 2025: TTPO closes the gap to the labeled OPSD upper bound and improves substantially over the majority-vote-only baseline, which — consistent with the motivation — can regress below the base model when pseudo-labels are poor.
Limitations and open questions
- The asymmetry argument depends on the plurality being an anti-concentrated distractor set. On problems where a specific wrong answer is systematically appealing (adversarial distractors, common arithmetic slips), the negative branch could penalize correct-but-minority rollouts.
- The method requires K rollouts per test problem plus gradient updates, so wall-clock cost at inference is substantially higher than standard decoding; the paper does not report a compute-matched comparison against simply increasing K for self-consistency.
- All results use Qwen3 with LoRA on math benchmarks; whether the same asymmetry holds for code, multi-step tool use, or open-ended generation is untested.
- The token-weighting schemes (down-weight converged positives, mask non-confident negatives) are heuristic; a principled derivation from a noise model of pseudo-labels is missing.
Why this matters
TTPO turns the standard failure mode of unsupervised post-training — pseudo-label noise — into a design principle by exploiting that “wrong rollouts are reliably wrong” even when “right rollouts are only pseudo-right.” That gives a label-free objective that is competitive with label-supervised distillation on hard math benchmarks, suggesting test-time RL/OPSD hybrids can be viable when consensus is unreliable but disagreement is informative.
Source: https://arxiv.org/abs/2608.27448
Self-OPD: On-Policy Distillation for Flow Matching Models without Teacher
Problem
On-policy distillation (OPD) has become a strong recipe for aligning flow matching (FM) generators to task rewards: a specialized teacher provides dense, step-wise velocity targets that the student regresses against. This is expensive — one teacher per objective — and suffers compounding error when the student’s on-policy distribution drifts from the teacher’s training distribution. On the other side, trajectory-level RL methods (Flow-GRPO, GRPO-Guard, DiffusionNFT) avoid teachers but backpropagate only sparse terminal-reward signals through long ODE rollouts.
Self-OPD asks whether the student itself can generate the step-wise supervision that OPD normally imports from a teacher, retaining dense per-step targets while eliminating the teacher dependency.
Method
The setup is standard rectified flow: probability path x_t=(1-t)x_0+t\epsilon, velocity field v_\theta trained via \mathcal{L}_{\mathrm{FM}}=\mathbb{E}[\|v_\theta(x_t,t)-(\epsilon-x_0)\|^2], integrated in reverse from t{=}1 to t{\approx}0 along a discrete schedule \{t_j\}. To enable local exploration, the ODE step is converted to Euler–Maruyama SDE steps
x_{t_{j+1}} = x_{t_{j+1},\theta} + \sigma_{t_j}\sqrt{|\Delta t_j|}\,z_j,\quad \sigma_{t_j}=\eta\sqrt{t_j/(1-t_j)},
with \eta=0.7 in experiments. The score identity \nabla_{x_t}\log p_t(x_t)\approx -\bigl(x_t+(1-t)v_\theta\bigr)/t links the drift back to the velocity field, so perturbations in x-space correspond to well-defined perturbations in the velocity target.
At each training timestep, the student branches its deterministic next state into K{=}8 stochastic SDE candidates, then finishes each branch with an ODE rollout to a final image. Each rollout is scored to give r_k; a deterministic (noise-free) rollout gives the self-reference baseline r^{\mathrm{ref}}. Normalized advantages
A_k = \frac{r_k - r^{\mathrm{ref}}}{\mathrm{std}(\{r_k\})+\varepsilon}
partition branches into positives (A_k>0, velocities v_+) and negatives (A_k<0, velocities v_-).
The loss is an all-branch pull–push on the velocity field at the branching timestep:
- Pull term: MSE from v_\theta toward advantage-weighted positive velocities.
- Push term: repulsion from negative velocities, but modulated by a direction-aware coefficient d_k that gates the push whenever it would oppose the pull direction, plus SDE-variance normalization to keep the magnitude of stochastic perturbations comparable across timesteps.
Concretely, the update looks like a signed weighted regression v_\theta \leftarrow v_\theta - \nabla_\theta \sum_k w_k\|v_\theta - v_k\|^2 where w_k carries the sign of A_k, the variance normalization, and the gating d_k. Because supervision is on the velocity itself (not on the trajectory return), gradients are dense and step-wise as in teacher-based OPD, but the “teacher velocity” is replaced by a reward-weighted mixture of the student’s own branched velocities.
Training uses SD3.5-Medium at 512×512 with LoRA on the transformer, AdamW at 3\times 10^{-4}, two training timesteps per optimization step.
Results
Single-reward, teacher-free comparison against Flow-GRPO and GRPO-Guard (Table 1): Self-OPD reaches GenEval strict 0.9536 vs 0.9155 (GRPO-Guard) and 0.9005 (Flow-GRPO), from a 0.5222 base. On OCR, Self-OPD is 0.9745 vs 0.9348 / 0.9253. On PickScore, 24.47 vs 23.98 / 23.57. On HPSv2, 0.4099 vs 0.3453 / 0.3396. Improvements are consistent across all four reward models, with the OCR and HPSv2 gains being the largest relative deltas.
The trade-off known from single-reward RL fine-tuning persists: optimizing OCR degrades GenEval to 0.4186 and PickScore’s HPSv2 to 0.2719, i.e., per-reward specialists over-fit to their objective. The mixed-reward regime (Table 2) is where Self-OPD is most notable: a single teacher-free model reaches GenEval strict 0.9521, OCR 0.9597, PickScore 23.87, HPSv2 0.3214, beating teacher-based Flow-OPD (0.8594 / 0.9392 / 23.43 / 0.3042) and DiffusionOPD (0.9150 / 0.9464 / 22.72 / 0.2676) on GenEval and OCR while matching or exceeding them on preference metrics, and beating the teacher-free DiffusionNFT baseline (0.8888 / 0.9229 / 23.67 / 0.3206) on essentially every axis.
Limitations and open questions
- Cost per step: K{=}8 branches each requiring a full ODE rollout to score means the effective per-update compute is high; the paper does not report wall-clock comparisons vs Flow-GRPO/DiffusionNFT.
- Self-reference baseline can collapse when all K branches are near r^{\mathrm{ref}}; the variance normalization mitigates but does not resolve low-signal timesteps.
- The direction-aware gating d_k is a heuristic; a principled derivation (e.g., trust-region on velocity space) is absent.
- Reward hacking is visible in single-reward runs (OCR trades against GenEval); the mixed-reward result depends on hand-fused scorers.
- Evaluation is at 512×512 with LoRA on one backbone (SD3.5-Medium); scaling behavior at higher resolution or on video FM models is untested.
Why this matters
Self-OPD closes a practical gap: it delivers the dense, step-wise supervision quality of teacher-based OPD without training a teacher per objective, and does so while outperforming both teacher-based OPD and trajectory-level RL under a single mixed-reward model. If the compute overhead of K branched rollouts can be amortized, this becomes a natural default for reward fine-tuning of flow matching generators.
Source: https://arxiv.org/abs/2608.26872
GameWAM: A World Action Model for Video Games
Problem
Two families of interactive game models sit on either side of a gap. Behavior-cloned agents (VPT-style) map observations and instructions to native actions but carry no explicit forward model, so they cannot reason about visual futures. Interactive world models (Genie, GameNGen, and successors) synthesize plausible next frames conditioned on supplied actions but do not produce a policy — they must be driven by an external controller. World-Action Models (WAMs) aim to unify the two objectives, but existing WAM work has largely targeted robotics or short-horizon manipulation. Video games raise distinct difficulties: first-person perception with rapid visual change, persistent world state across minutes, and heterogeneous native controls that mix continuous mouse deltas with discrete key events, plus mode switches between free-look gameplay and modal GUI screens (inventories, crafting tables). GameWAM is presented as the first WAM built for this native closed-loop regime.
Method
GameWAM factorizes rollouts into world-action blocks B_j = (\mathcal{V}_j, \mathcal{A}_j), where \mathcal{V}_j are video latents and \mathcal{A}_j = [\mathcal{A}_j^{\text{cont}}; \mathcal{A}_j^{\text{disc}}] is the native keyboard-mouse vector. The conditioning context at block k of cycle c is
\Gamma_{c,k} = (C_{c,k}, \ell, H_c, s_{c,k}),
with clean within-cycle observations C_{c,k}, instruction \ell, persistent cross-cycle visual history H_c, and optional proprioception s_{c,k}. Over an R-block plan the model factorizes block-causally:
p_\theta(B_{k:k+R-1}\mid\Gamma_{c,k}) = \prod_{r=0}^{R-1} p_\theta(B_{k+r}\mid\Gamma_{c,k}, B_{k:k+r-1}).
Both modalities are trained with flow matching. For m\in\{v,a\}, the interpolation and target are
X_{\sigma_m}^m = (1-\sigma_m)X_0^m + \sigma_m \epsilon^m,\quad U^m = \epsilon^m - X_0^m,
and parallel Video-DiT and Action-DiT branches predict (\widehat{U}_\theta^v, \widehat{U}_\theta^a) from the noisy modalities plus context. At inference an ODE is integrated from \sigma=1 to 0 starting from Gaussian samples.
The critical design choice is a Fast-WAM-style modality-decoupled attention mask. For target block j, both branches see the same clean visual prefix \mathcal{P}_{c,j} = H_c \cup C_{c,j}, but their noisy variables do not attend to each other:
\operatorname{Vis}(\mathcal{A}_j) = \mathcal{P}_{c,j} \cup \mathcal{A}_j,\qquad \operatorname{Vis}(\mathcal{V}_j) = \mathcal{P}_{c,j} \cup \mathcal{V}_j.
The Video DiT encodes \mathcal{P}_{c,j} into layer-wise K/V that both branches share, so video supervision shapes the context used for action denoising while online action-only rollout does not have to denoise a future video. An ablation against joint within-block video-action attention is discussed in §D.2.
To handle heterogeneous native control, GameWAM adds a per-timestep router. A logit \rho_\tau decides gameplay vs. GUI mode; the predicted flow field for the whole native action vector is
\widehat{U}_\tau^a = (1-\hat{r}_\tau)\widehat{U}_\tau^{\text{game}} + \hat{r}_\tau \widehat{U}_\tau^{\text{gui}},\quad \hat{r}_\tau = [\sigma(\rho_\tau) > 1/2].
Continuous and discrete coordinates remain in one native vector during flow generation; discrete channels are thresholded into binary decisions only after sampling. Continuous-action normalization (mouse deltas) is applied per mode.
For long-horizon interaction, block-cycle control predicts R blocks ahead but commits only a short prefix before re-planning. Hierarchical memory pairs a transient cycle-local K/V cache with persistent cross-cycle visual history H_c, so long-range state (biome, inventory contents, mission progress) survives cycle boundaries without recomputing attention over the full trajectory.
Data
Joint world-action learning requires aligned (o_t, a_t) streams with mode labels. In Minecraft the authors combine three streams standardized on a common interaction timeline: (i) regular VPT trajectories for broad naturalistic coverage and long temporal context; (ii) an Event-Anchored VPT dataset in which MineStudio-style interaction events are extracted from state transitions, yielding instruction-aligned event-centered clips; and (iii) scripted MineStudio GUI trajectories for interface coverage. Clip sampling within the event-anchored stream is dense near the event anchor and sparse away from it, on the argument that traversal and incidental camera motion carry weak supervision compared with the moments immediately surrounding annotated events.
Results
Evaluation uses two closed-loop benchmarks: Minecraft Universe (MCU), which measures long-horizon task success and executed environment actions under native control, and a four-map ViZDoom suite for fast visual dynamics evaluated by average episode reward. The paper compares against published baselines under their reported protocols. The provided sections do not enumerate the numerical table entries; specific per-task success rates and reward numbers are reported in the experiments section and appendices of the released paper.
Limitations and open questions
Several points are worth flagging. The modality-decoupled mask deliberately blocks the noisy action tokens from attending to the noisy video and vice versa, so the “joint” nature of generation is mediated only through the shared clean prefix K/V — how much the video branch actually improves policy quality beyond an auxiliary loss is an empirical question the §D.2 ablation is meant to address. The router is a hard binary switch; failures in mode prediction discard the entire mode-specific action vector for that timestep. Event-centered dense sampling risks distribution shift away from the traversal segments that dominate real play. Finally, evaluation is confined to Minecraft and ViZDoom; transfer to games with fundamentally different control surfaces (twin-stick, rhythm, strategy) is untested.
Why this matters
GameWAM is a concrete instantiation of a WAM under native, heterogeneous, long-horizon interaction, and it demonstrates that flow-matching DiTs with block-causal decoupled attention can serve as both a forward model and a policy for GUI-plus-gameplay control. If the released weights and data hold up, it provides a reusable substrate for studying whether visual future prediction genuinely helps action selection in interactive environments.
Source: https://arxiv.org/abs/2608.26200
PILOT in the Loop: Live Self-Improvement for Long-Horizon Agents
Problem
Long-horizon agent runs (multi-hour shell sessions, repository-level repairs) accumulate experience that is typically only harvested after the episode terminates: trajectory summaries feed offline fine-tuning, retrieval memory updates, or prompt tweaks. Two failure modes follow. First, the active run cannot be redirected once it commits to a dead-end plan — a ReAct worker asked to critique itself blends execution and assessment in one context, and self-critique degrades as the trajectory grows. Second, lessons visible mid-run (e.g., a package manager quirk that just cost 20 tool calls) are not applied to sibling tasks running concurrently or to workers spawned minutes later. The authors argue self-improvement should be live: emerging trajectory evidence should both steer the current worker and mutate the persistent harness in-flight.
Method
PILOT is a supervisor–worker harness. Model parameters \theta stay frozen; what evolves is a harness H = (\mathcal{K}, \mathcal{M}) consisting of a skill library \mathcal{K} and memory \mathcal{M}. For a task \tau in environment \mathcal{E}, the supervisor spawns workers W_j \leftarrow \textsc{Spawn}(\theta, \tau_j, H), possibly concurrently. Each worker runs in an isolated context and emits a trajectory \xi_j. The supervisor reads slices of \xi_j only on demand, keeping its own context focused on goal state, recent events, and recurring failure patterns rather than verbose tool output.
Two coupled loops run over this structure:
- Live steering: the supervisor can redirect or abort an active worker mid-execution. This is architecturally distinct from subagent delegation (where the parent only observes final output) and from single-agent self-correction (where the same context does both jobs).
- Live self-evolution: during supervision, the supervisor distils procedures and failure modes from \xi_j into new or revised entries in \mathcal{K} and \mathcal{M}. Crucially these updates happen before the verifier outcome is available — a candidate skill is content created from the live trajectory and environment feedback, not a post-hoc summary conditioned on pass/fail. Workers spawned after a harness mutation load the updated H.
Both supervisor and worker are filled with the same frozen backbone in the experiments, isolating orchestration gains from any capability gap between the roles.
Results
Three benchmarks, two backbones (GLM-5.1, Kimi-K2.6), four to five harness baselines (Pi, OpenCode, Terminus-2, Hermes, Mini-SWE-Agent).
Terminal-Bench 2.0 (89 tasks). PILOT reaches 71.9% (GLM-5.1) and 71.3% (Kimi-K2.6), averaging 71.6% — versus the next-best Pi at 66.3%. On Hard tasks specifically, PILOT scores 55.0% on both backbones, against 50.0% / 48.3% for Pi. The paper reports up to 9.8 percentage points over counterpart harnesses.
SWE-bench Multilingual / Pro. On Multilingual, PILOT averages 72.7% (essentially tied with Pi at 72.9%). On the harder SWE-bench Pro, PILOT averages 59.9% versus Pi 55.5%, driven by a strong Kimi-K2.6 result of 65.1% (vs. Pi 59.1%). PILOT ranks first in five of six configurations.
Iterative self-evolution on Terminal-Bench 2.0. Running iterated sweeps where H_i persists across runs but the environment is reset per task:
- GLM-5.1 peaks at iteration 14, Kimi-K2.6 at iteration 13.
- Additional passes vs. iteration 0 concentrate on Hard: GLM-5.1 gains (+2, +6, +8) across Easy/Medium/Hard; Kimi-K2.6 gains (+1, +7, +12).
- Skill library grows 62→83 (GLM-5.1) and 50→81 (Kimi-K2.6).
- Mean output tokens per evaluated task drop 28.5K→16.3K (42.9% reduction) on GLM-5.1 and 41.9K→22.1K (47.4%) on Kimi-K2.6, in five-iteration windows.
- Successful evaluations per million output tokens: peak improves 110.3% (GLM-5.1) and 134.0% (Kimi-K2.6) over iteration 0.
The concentration of gains on Hard tasks — those needing specific recovery procedures the backbone cannot reconstruct de novo — is the cleanest evidence that \mathcal{K} is capturing genuinely reusable structure rather than memorising trivia.
Limitations and open questions
- Supervisor and worker share the frozen backbone; whether a weaker supervisor still steers effectively (or a stronger one dominates) is not tested. This matters because supervisor calls are the marginal cost of PILOT versus a single-agent harness, and cheap-supervisor deployments are the practical regime.
- Skill/memory updates are made without verifier signal, but the iterated Terminal-Bench 2.0 protocol still evaluates on the same task distribution used to accumulate H. Cross-benchmark transfer of \mathcal{K} (e.g., Terminal-Bench-derived skills applied to SWE-bench Pro) is not reported.
- No ablation isolating live steering from live self-evolution in the one-shot vs. self-improvement settings is shown in the supplied sections; the 9.8 pp headline number bundles both mechanisms.
- The skill library grows monotonically over 13–14 iterations; there is no analysis of skill decay, contradiction handling, or retrieval scalability past ~80 skills.
- Live mutation of H while sibling workers are running raises consistency questions (which H_i does an in-flight worker see?) that the excerpts do not address.
Why this matters
PILOT reframes agent self-improvement from a post-hoc training/memory update into a control-plane problem: a separate supervisor edits both the active trajectory and the persistent harness during execution. The 42–47% token reductions with simultaneous accuracy gains suggest that a lot of what looks like model weakness on long-horizon tasks is actually harness amnesia — the same discoveries being re-derived every run.
Source: https://arxiv.org/abs/2608.26530
Zero-WAM: In-Context World-Action Modeling from Human Videos for Open-Ended Task Generalization
Problem
Zero-shot cross-task manipulation asks a policy to execute tasks that never appeared in training, with no parameter update at deployment. Language instructions underspecify manipulation: “put the red block on the shelf” leaves grasp affordance, approach direction, and sub-goal decomposition ambiguous, and language-conditioned policies tend to memorize task-to-trajectory shortcuts from the training distribution. Zero-WAM proposes that the natural task specification for manipulation is a human video demonstration of the same task, and casts cross-task generalization as an in-context learning (ICL) problem: given a video prompt of a human doing task T, predict the robot’s future video-action chunks for T.
The core obstacle is data. Paired human-robot demonstrations across thousands of tasks do not exist at scale, and naive language-conditioned pretraining on public robot data over-samples repeated tele-operations of a few tasks.
Method
HumanGen data pipeline. The authors re-partition five public robot datasets (AgiBot, InternData-A1, Open-X-Embodiment, RoboCOIN, RoboMIND) into tasks defined by (action, object) pairs and cap trajectories per task, yielding roughly 6,000 tasks and ~400K trajectories per epoch of task-balanced VA pretraining. On top of this, they synthesize matched human videos from task-sampled robot videos, producing 74.2K human-robot ICL pairs across 8.6K tasks, split into Pre-train ICL (External/In-house), Simulation ICL, and Real-world ICL.
Causal video-action model. Following LingBot-VA, each trajectory is chunked as \tau=\{(\mathbf{x}^i,\mathbf{a}^i)\}_{i=1}^N with aligned video and action chunks. At step i the model factorizes
p_\theta(\mathbf{x}^{i+1},\mathbf{a}^{i+1}\mid\mathbf{x}^{\le i},\mathbf{a}^{\le i},c) = p_\theta^{\mathrm{vid}}(\mathbf{x}^{i+1}\mid\cdot)\, p_\theta^{\mathrm{act}}(\mathbf{a}^{i+1}\mid\mathbf{x}^{i+1},\cdot),
where p_\theta^{\mathrm{act}} acts as an inverse-dynamics head that decodes actions from the predicted next-video chunk. Video prediction uses flow matching: for clean chunk \mathbf{x}_0, noise \bm\epsilon, and t\in[0,1],
\mathbf{x}_t=(1-t)\mathbf{x}_0+t\bm\epsilon,\quad \mathbf{v}_t^\star=\bm\epsilon-\mathbf{x}_0,\quad \mathcal{L}_{\mathrm{fm}}=\mathbb{E}\big[\|\mathbf{v}_\theta(\mathbf{x}_t,t,c)-\mathbf{v}_t^\star\|_2^2\big].
The condition is c=\ell (language) for Task-diverse VA data and c=\{\mathbf{h},\ell\} for HumanGen ICL data, where \mathbf{h} is the human video.
Architecture. Zero-WAM is initialized from Wan-2.2-TI2V-5B and refactored into a Mixture-of-Transformers with two streams of hidden dim d_v=d_a=3072, 30 layers, RoPE, with a height-axis RoPE offset \Delta_H=32 separating the ICL human video from the robot video tokens. Videos (human and robot) pass through the Wan-2.2 VAE; language uses T5.
In-context Future Prediction (IFP). The critical training objective for ICL: rather than only predicting the next chunk, the model predicts K=4 future robot chunks with temporal stride s=2, weighted (w_1,\dots,w_4)=(0.5,0.25,0.15,0.15). Because near-future prediction can be solved from the recent trajectory prefix alone (a shortcut for seen tasks), forcing farther-horizon prediction requires drawing task-progression information from the human video prompt. This is the mechanism that suppresses task-identity shortcuts learned during pretraining.
Training regime. Task-diverse VA and HumanGen are mixed 1:5. Language dropout is raised from Wan-2.2’s default 0.1 to 0.4 on ICL samples, ICL human-video latent is dropped with probability 0.1, and non-ICL language dropout is 0.1. Chunk size during pretraining is uniformly 1-4; inference chunk size is 2. AdamW at lr 10^{-4}, weight decay 0.01, 15,360 GPU-hours. Token-packing keeps per-GPU sequences within 160K tokens.
Inference. Two modes: language-only (video CFG=5, no human prompt) or ICL (human video prompt, language disabled, ICL CFG=5, action CFG=1.0).
Results
On seven unseen tasks in RoboTwin 2.0 — spanning unseen objects, articulated objects, bimanual, and long-horizon manipulation — Zero-WAM achieves 47.0% average success, an absolute +29.5 pp over the strongest video-action baseline (implying the baseline sits at 17.5%).
In real-world evaluation, human video prompts drive generalization to multi-object scenes, long-horizon sequences, and fine-grained insertion configurations not present in training.
Limitations and open questions
- The paper’s headline gain relies on a synthetic human video generator; the fidelity and bias of that generator likely bounds ICL quality, and the abstract does not report an ablation isolating “synthetic human video prompt” vs. “real human video prompt” at test time.
- The 47.0% absolute success rate leaves majority-failure on unseen tasks; long-horizon and fine-manipulation failure modes are not decomposed here.
- IFP weights and stride (K=4,s=2) are hand-tuned. Whether the shortcut-suppression argument holds without this schedule, or how it scales with longer horizons, is open.
- Two-mode inference (language-only vs. ICL) with disabled language in ICL mode suggests the model has not fully unified the two conditioning channels; compositional prompts (language + video) are not evaluated in the excerpt.
- Compute (15,360 GPU-hours from a 5B base) makes independent replication expensive, and generalization to embodiments outside the pretraining datasets is untested here.
Why this matters
Zero-WAM operationalizes ICL for manipulation with a concrete recipe: task-balanced VA pretraining, automatically-generated human video prompts, and a future-chunk prediction loss that structurally penalizes task-identity shortcuts. If the +29.5 pp gap on unseen tasks holds under stricter ablations, video prompting becomes a viable alternative to language-conditioned generalist policies where the task specification itself, not model capacity, is the bottleneck.
Source: https://arxiv.org/abs/2608.26103
Understanding Evolution Strategies for LLM Reasoning: Broader Reasoning Coverage than GRPO
Evolution Strategies (ES) have re-emerged as a memory-efficient alternative to policy-gradient post-training for LLM reasoning: no backprop through the sampler, only forward evaluations of perturbed parameters. This paper asks what ES actually optimizes differently from GRPO, and whether its qualitative advantages (or drawbacks) are principled or incidental. The authors organize the analysis around three questions: (1) does ES avoid GRPO’s entropy collapse and preserve Pass@K? (2) does its large parameter drift imply catastrophic forgetting? (3) which hyperparameters and estimators make ES scalable?
Population diversity as a mechanism for Pass@K
The central theoretical claim is that ES’s parameter noise induces a controlled amount of policy-space diversity that transfers into higher Pass@K. Let s_\theta(y\mid x)=\nabla_\theta\log\pi_\theta(y\mid x) and prompt-conditioned Fisher information
\mathcal{I}_x(\theta)=\mathbb{E}_{Y\sim\pi_\theta(\cdot\mid x)}[s_\theta s_\theta^\top].
A local perturbation \delta=\sigma\epsilon produces D_{\mathrm{KL}}(\pi_{\theta+\delta}^x\|\pi_\theta^x)=\tfrac{1}{2}\delta^\top\mathcal{I}_x(\theta)\delta+o(\|\delta\|^2). Lemma 1 then quantifies the population-averaged Jensen–Shannon spread of an N-member ES population:
\mathbb{E}_{\epsilon_{1:N}}[\mathrm{JS}^{\mathrm{pol}}_N(x)] = \frac{\sigma^2}{2}\Big(1-\frac{1}{N}\Big)\operatorname{tr}\mathcal{I}_x(\theta) + O(\sigma^4).
The mechanism is thus explicit: perturbation scale \sigma and population size N dial the JS diversity across the population, and the paper argues (formalized under standard support/smoothness conditions) that when the verifier v(x,y) projects this diversity onto the correctness event \mathcal{C}(x), sampling across heterogeneous \pi_i raises the probability that at least one member produces a correct response — the Pass@K quantity. Reward weighting in the ES update further biases the recombined center toward the higher-p_\pi(x) members, so the diversity advantage can survive aggregation rather than being washed out.
Empirically (Panel (a)), GRPO exhibits the familiar entropy collapse: Pass@1 improves but Pass@K at large K degrades. ES improves Pass@1 while attaining higher Pass@K than GRPO. The authors then propose a sequential GRPO→ES schedule that recovers GRPO’s Pass@1 sharpening and layers ES’s coverage gains on top.
Large drift, sparse functional updates
RQ2 addresses the standard worry about ES: parameter movement is enormous relative to gradient methods, which suggests catastrophic forgetting. Measuring relative L_2 drift D_{\mathrm{rel}}(\theta,\theta_0)=\|\theta-\theta_0\|_2/\|\theta_0\|_2 (Table 4), Full ES moves 40.7–44.1× farther than GRPO across Qwen2.5-1.5B/7B, Llama-3.2-3B, and DeepSeek-R1-Distill-Qwen-1.5B (e.g., 0.0475\to 1.933 for Qwen2.5-1.5B; 0.0954\to 4.185 for Llama-3.2-3B).
But the update magnitude distribution is heavily skewed toward small changes. At threshold \tau=2.0\times 10^{-3}, 97.57%–98.11% of nonzero updates fall in (0,\tau] for the three instruct models; only DeepSeek-R1-Distill-Qwen-1.5B is less sparse (87.29%). This is consistent with Hoy et al.’s decomposition of ES into reward-aligned progress plus a high-dimensional random walk along flat directions: most of the drift is diffusion, and reward-relevant change concentrates in a sparse subset of larger-magnitude updates. Panel (b) shows that magnitude-thresholding — keeping only the larger updates — preserves target-task performance and, importantly, held-out Maj@32, indicating that catastrophic forgetting is not an intrinsic property of ES but a property of the diffusion component.
Estimator and scaling knobs
RQ3 isolates which choices actually matter. Z-score reward normalization within the population outperforms no normalization throughout training (Panel (c), item (1)); this matches the theoretical picture, since ES uses rewards as weights on perturbation directions and unnormalized rewards drift with task difficulty. Population-size scaling on GSM8K (Table 6) shows the point of diminishing returns: for Qwen2.5-1.5B and 3B, N=32 is within 0.01 of the N=64 reference at every measured update (e.g., 1.5B at update 300: 0.8449 vs. 0.8438; 3B: 0.9004 vs. 0.9031). For the 0.5B model, larger N still helps (0.4421 at N=8 vs. 0.5287 at N=64 at update 300), suggesting the required diversity to hit correctness grows as base model competence drops — coherent with \operatorname{tr}\mathcal{I}_x(\theta) interacting with N in Lemma 1.
Limitations and open questions
The Pass@K analysis is local (\sigma\to 0) and conditional on realized populations; the transfer of population JS diversity to the ES-updated center under reward weighting is characterized under conditions but not tightly bounded for practical \sigma and N. The sparsity result is descriptive: the paper does not yet give a training-time procedure that suppresses diffusion updates without harming reward-aligned progress (magnitude thresholding is post-hoc). Finally, GRPO→ES is shown to combine strengths, but the interaction between GRPO’s entropy contraction and ES’s diversity injection is not analyzed.
Why this matters
If Pass@K coverage is what downstream verifiers or best-of-N pipelines consume, then GRPO-style entropy collapse is actively harmful and ES provides a principled, Fisher-information-controlled way to preserve coverage. The functional-sparsity finding also reframes ES’s “huge parameter drift” from a liability into a signal that most updates are prunable, which points toward cheaper deployment (sparse deltas) and hybrid pipelines that use GRPO for Pass@1 and ES for reasoning breadth.
Source: https://arxiv.org/abs/2608.27351
Hacker News Signals
Terminal-Bench-Science: Evaluating AI agents on scientific research workflows
Terminal-Bench-Science is a benchmark targeting AI agent evaluation on end-to-end scientific research tasks executed inside a terminal environment. Unlike most existing benchmarks that test narrow question-answering or isolated code generation, this one chains together the full research loop: literature search, data acquisition, experimental design, code execution, and result interpretation — all within a bash-accessible environment with real tools (Python, wget, git, standard scientific libraries).
Each task is structured as a goal specification given to the agent (e.g., “reproduce Figure 3 from this paper using the provided dataset”) and scored on verifiable outputs: correct numerical results, passing test suites, or artifact existence checks. The benchmark covers domains including bioinformatics, computational chemistry, and physics simulations. Tasks are graded automatically where possible, with a subset requiring human review for partial credit.
The design philosophy is that terminal access is the minimal interface for real scientific work — agents must chain shell commands, manage file system state, install packages, and debug runtime errors, which surfaces failure modes invisible in sandboxed code-execution benchmarks. The leaderboard results show a wide capability spread: top frontier models complete roughly 30-40% of tasks fully, while smaller models stall on multi-step dependency chains and error recovery.
One notable design decision is the inclusion of “noisy environment” tasks where upstream data sources are slightly malformed or dependencies have version conflicts, testing robustness and self-debugging. This is closer to actual research workflows than clean-room benchmarks.
Key open questions: scoring scientific correctness at scale without domain expert review is unsolved; the benchmark currently sidesteps this for most tasks by requiring exact numerical matches rather than judging scientific validity. Coverage is also skewed toward computation-heavy sciences; wet-lab planning tasks are absent.
Source: https://www.terminal-bench-science.ai/announcement
Show HN: The load-bearing vocabulary of Claude
This project reverse-engineers which tokens in Claude’s vocabulary carry disproportionate influence over its outputs by probing activation patterns and output distributions across a large prompt corpus. The core idea: not all tokens are equal in how they steer model behavior. Some tokens — often structurally mundane-looking ones — act as pivots that shift the model’s completion distribution dramatically when present or absent.
The methodology appears to involve computing something analogous to a per-token importance score by measuring KL divergence or output entropy shifts when specific tokens are masked or substituted. The resulting “load-bearing vocabulary” list is striking because it skews toward punctuation, whitespace patterns, XML-like delimiters (Claude is heavily trained with structured prompts), and specific imperative verbs, rather than obviously semantic content words.
Practically this is relevant for prompt engineering and adversarial prompting research. If a small set of tokens dominates output steering, then prompt optimization (e.g., AutoDAN, GCG-style attacks, or benign soft-prompt search) should concentrate budget on those tokens rather than searching uniformly across the vocabulary. It also suggests that Claude’s instruction-following behavior is structurally brittle in specific lexical locations.
The visualization on the project page maps token importance across vocabulary rank, showing a heavy-tailed distribution — a small number of tokens have outsized impact, consistent with what one would expect from RLHF fine-tuning over structured prompt templates. The analysis is Claude-specific because the token selection correlates with Anthropic’s known training prompt formats (XML tags, <thinking>, Human:, Assistant: delimiters).
Limitations: the methodology is somewhat opaque — it is not entirely clear whether importance scores are derived from logit attribution, attention weight aggregation, or behavioral probing. Replication requires access to the model’s outputs at scale, not its weights, so this is black-box reverse engineering.
Source: https://louisabraham.github.io/load-bearing/
Asahi Linux Progress Report: Linux 7.2
The Asahi Linux project covers upstream integration work landing in Linux 7.2, targeting Apple Silicon (M-series ARM SoCs). The headline items are continued DCP (Display Coprocessor) driver maturation, PCIe power management improvements, and NVMe controller fixes affecting M3-class hardware.
The DCP driver, which handles display output through Apple’s firmware-managed coprocessor, has received improvements to HDMI/DisplayPort hotplug handling and color management. The Apple GPU driver (AGX) continues incremental upstream stabilization, with OpenGL conformance fixes that close gaps versus the downstream Asahi Mesa fork. The Rust-written Apple SoC pinctrl driver has been further refined — this is one of the more concrete examples of production Rust code in the mainline kernel managing real hardware.
PCIe power management was a significant regression source in earlier kernels on M2/M3 Macs; 7.2 includes patches addressing ASPM (Active State Power Management) behavior that caused NVMe latency spikes and occasional controller resets. The thunderbolt/USB4 stack also received fixes specific to Apple’s implementation.
The report notes that the SMC (System Management Controller) driver gained new sensor support, improving thermal and battery reporting accuracy in userspace tools. CPU frequency scaling via the Apple-specific cpufreq driver has been tuned for better energy efficiency on the efficiency cores.
From an engineering standpoint, the most interesting ongoing work is the DCP reverse engineering pipeline: Apple’s display firmware is closed-source, so the driver is built from protocol traces and educated inference about firmware state machines. The project maintains a parallel RTKit (Apple’s embedded RTOS) abstraction layer that other Apple peripheral drivers also depend on. Upstream acceptance velocity for this driver class has improved, suggesting the kernel maintainers are now comfortable with the architecture.
Source: https://asahilinux.org/2026/08/progress-report-7-2/
Z.ai confirms Ox Alpha is a new GLM-series model and will release its weights
Z.ai (formerly Zhipu AI) has confirmed that Ox Alpha is an extension of the GLM (General Language Model) architecture lineage, the same family as GLM-4 and ChatGLM. The confirmation that weights will be released publicly is the substantive news — it means researchers will be able to inspect and fine-tune a model reportedly competitive with DeepSeek-class performance.
The GLM architecture diverges from standard GPT-style transformers in its pretraining objective: it uses autoregressive blank infilling rather than pure left-to-right language modeling, treating spans of masked tokens as generation targets while conditioning on bidirectional context elsewhere. This was the original GLM innovation; later GLM-4 variants moved closer to conventional decoder-only architectures but retained some of the pretraining methodology.
Ox Alpha is reported to be a large-scale model (exact parameter count unconfirmed) trained with enhanced long-context capability and improved instruction following. The Bloomberg piece notes it was benchmarked internally against DeepSeek-V3 and GPT-4o-class models on Chinese-language and code tasks, with competitive results.
The weight release is significant for several reasons. GLM-series models have historically had permissive licenses that allow commercial use, which has made them widely deployed in Chinese enterprise contexts. Open weights at frontier capability would continue that pattern and provide a concrete alternative to the DeepSeek weight releases that dominated open-model discourse earlier in 2025-2026.
Open questions: the exact architecture details (MoE vs. dense, context window, tokenizer changes) are not yet public. The benchmark methodology used for the DeepSeek comparison is unverified. The release timeline is not specified.
Gemini-3.5-Transcribe
Gemini-3.5-Transcribe is a dedicated speech-to-text model from Google, positioned as a replacement for Whisper-class models in high-accuracy transcription pipelines. The key differentiator versus general-purpose Gemini multimodal models is that this variant is specialized and optimized specifically for transcription throughput and accuracy, rather than treating audio as one modality among many.
Technically, the model handles long-form audio natively (the post mentions multi-hour transcription without chunking artifacts, which is a known pain point with Whisper’s 30-second windowing). It supports over 100 languages and includes automatic language detection. Speaker diarization is integrated rather than requiring a separate post-processing step.
The architecture specifics are not disclosed, but the framing suggests an encoder-decoder structure where the audio encoder benefits from Gemini’s pretraining data and the decoder is fine-tuned on large-scale transcription corpora with speaker metadata. Timestamp generation at the word and sentence level is supported, which matters for subtitle generation and downstream alignment tasks.
Accuracy benchmarks cited show improvements over Whisper Large-v3 on multilingual test sets, particularly for low-resource languages and accented speech, and Word Error Rate (WER) reductions in the 15-25% relative range on named entity-heavy domains (medical, legal). Latency is presented as competitive for streaming use cases, though specifics on real-time factor are sparse.
The API pricing model is per-minute of audio, and there is a batch mode for offline processing. The main practical limitation is that it is a closed API — there is no weight release, so users cannot self-host, fine-tune on private data, or audit behavior on sensitive transcription domains. For enterprise use cases with data residency requirements, this is a hard blocker.
Source: https://blog.google/innovation-and-ai/models-and-research/gemini-models/gemini-3-5-transcribe/
Gemini Omni 1.1 Flash
Gemini Omni 1.1 Flash is a multimodal model handling text, image, audio, and video input/output within a single unified architecture, targeting latency-sensitive production deployments. The “Flash” designation continues Google’s naming convention for distilled/optimized variants intended for high-throughput serving rather than maximum capability.
The 1.1 update over the original Gemini Omni Flash focuses on three areas: improved instruction following in interleaved multimodal contexts, better audio output quality (the model can generate speech directly rather than routing through a separate TTS system), and expanded video understanding with longer effective context over video frames. Native audio output is architecturally interesting — it implies the token vocabulary or output heads include audio tokens in addition to text, similar to the approach used in GPT-4o’s reported architecture.
For developers, the key capabilities are real-time voice conversation (the model handles turn-taking and interruption), image-conditioned dialogue, and video summarization. Latency figures cited suggest time-to-first-audio-token in the low hundreds of milliseconds, which is in the range required for natural voice interaction.
The HN discussion focuses on API behavior differences from the previous version — several commenters note regression in specific structured output tasks and changes in refusal behavior. These are common friction points with model updates in production pipelines and suggest the 1.1 fine-tuning changed the instruction-following distribution in ways that are not purely additive.
Context window is 1M tokens, consistent with the Gemini 1.5/2.0 line. Pricing follows a tiered structure with input/output token costs differentiated by modality. The model is not open-weight; the architecture is undisclosed beyond Google’s public technical reports.
Source: https://blog.google/innovation-and-ai/technology/developers-tools/build-with-gemini-omni-1-1-flash/
Show HN: LatticeDB – Like SQLite but for graph databases
LatticeDB is an embedded graph database written in Go, targeting the same deployment niche as SQLite — a single-file, zero-configuration store with no server process — but with a property graph data model and a graph query interface instead of relational SQL.
The storage layer uses a custom B-tree implementation backed by a memory-mapped file. Vertices and edges are stored as typed entities with arbitrary key-value properties. The query interface appears to be a path-expression language rather than a full Cypher or Gremlin implementation, supporting pattern matching over adjacency structure with predicate filters on properties.
The architectural decision to embed rather than serve is motivated by the same tradeoffs as SQLite: eliminating serialization overhead, simplifying deployment in CLI tools and mobile apps, and allowing transactional access without network latency. The current implementation uses a WAL (Write-Ahead Log) for crash safety and supports ACID transactions at the single-writer/multiple-reader level.
Graph databases at this scale face a structural challenge that relational embedded stores do not: graph traversal access patterns are highly irregular and pointer-chasing, which is hostile to B-tree locality. LatticeDB addresses this by storing adjacency lists for each vertex as sorted arrays within the vertex record, which makes degree-1 neighbor lookup a single page read but makes dense traversal more expensive than purpose-built adjacency list formats (e.g., CSR).
Current limitations noted in the README: no full-text index, no distributed mode (by design), query planner is rudimentary (no cost-based optimization), and the query language is not compatible with any standard (Cypher, GQL). The last point is a significant adoption barrier — there is no migration path from Neo4j or similar. The project is early-stage and the HN comments include several feature requests around Cypher compatibility and Python bindings.
Source: https://github.com/jeffhajewski/latticedb
Value Classes Still Need Compiler Sympathy
This post examines a persistent performance problem with value types (stack-allocated structs without heap indirection) in systems languages: compilers frequently fail to keep value-class instances in registers across function call boundaries and through non-trivial control flow, forcing spurious stack spills that eliminate the intended performance benefit.
The author demonstrates the problem with concrete benchmarks in C++ and a language with explicit value class semantics. The core issue is that the ABI for passing struct arguments is architecture-specific and often pessimistic: the System V AMD64 ABI passes structs up to 16 bytes in two integer registers, but structs slightly above threshold are passed by pointer, and the calling convention interacts badly with inlining decisions. When the compiler decides not to inline a function containing a value-class argument, it materializes the value on the stack and passes a pointer, negating any benefit from the value semantics.
The post includes assembly inspection showing specific cases where a two-field struct (16 bytes) is register-allocated successfully but a three-field variant (24 bytes) triggers the stack-spill path. It also covers LLVM’s byval vs. inreg attribute interaction and how mis-annotation of struct passing convention leads to correctness issues distinct from the performance problem.
The proposed mitigations are: (1) explicit [[gnu::regparm]] or equivalent annotations where the ABI permits, (2) relying on whole-program optimization / LTO to enable cross-translation-unit inlining that bypasses ABI constraints, (3) using std::span or thin pointer wrappers when the struct exceeds the register threshold, and (4) profiling call-site assembly before assuming value semantics deliver their theoretical benefit.
The broader point — that language-level abstractions for zero-cost value semantics depend on specific compiler behaviors that are not guaranteed and degrade silently — is important for systems programmers who rely on intuitions from benchmark microtests that happen to inline.
Source: https://johan-sjolen.github.io/post/compiler-sympathy/compiler-sympathy/
Noteworthy New Repositories
deeplethe/utopia
Self-described as an open-source “enterprise world model,” Utopia targets the gap between narrow task-specific models and general-purpose simulation of business environments. The core idea is to maintain a structured, updatable world state — encoding entities, relationships, and temporal dynamics — that an LLM backbone can query and update rather than relying purely on in-context representations. The architecture separates state management from inference, which is architecturally similar to neurosymbolic or model-based RL designs but applied to enterprise knowledge graphs. The intent is to support multi-step planning, causal reasoning over organizational data, and persistent memory across agent sessions. For practitioners building enterprise agents that need coherent long-horizon behavior — supply chain planning, CRM workflows, financial projections — a world model substrate is more tractable than raw retrieval. The repository is early-stage, so the “enterprise” framing is aspirational, but the structural separation of world state from the LLM is the technically interesting commitment. Worth watching for how they handle state consistency and conflict resolution when multiple agents write to shared world state simultaneously.
Source: https://github.com/deeplethe/utopia
Spielewoy/autoprompt-skill
Autoprompt is a prompt-engineering layer designed as a drop-in skill for coding agents, claiming a 45% reduction in task failures on agentic coding benchmarks. The mechanism is dynamic prompt construction: rather than static system prompts, the skill inspects the current task context — file types, error traces, prior tool call outputs — and selects or assembles prompt fragments from a structured library. This is closer to retrieval-augmented prompting than to learned soft prompts. The 45% failure reduction figure is internally benchmarked on agentic coding tasks (the exact eval harness matters here and should be scrutinized), but the directional result is plausible given how much coding agent failures stem from under-specified context. The skill interface is designed to compose with existing agent frameworks — the repo shows integration patterns for tool-calling loops where the skill intercepts the prompt assembly stage. The practical value is that it externalizes prompt logic from agent code, making it testable and versioned. Limitation: effectiveness is highly sensitive to the quality of the fragment library, which requires domain-specific curation.
Source: https://github.com/Spielewoy/autoprompt-skill
Kylin010/tcpfit
tcpfit is a TCP tuning tool that derives per-machine parameters empirically rather than applying fixed textbook values. The standard approach to TCP tuning — setting rmem/wmem, tcp_congestion_control, and buffer sizes from generic guidelines — ignores actual network conditions. tcpfit measures the bandwidth-delay product (BDP) on the target interface and probes for rate-limiter inflection points (the knee in throughput vs. buffer-size curves) to derive machine-specific socket buffer recommendations. The BDP measurement is straightforward: \text{BDP} = \text{bandwidth} \times \text{RTT}, but the inflection-point detection requires iterating over send buffer sizes under load, which the tool automates. The output is a set of sysctl recommendations derived from actual measurements rather than rules of thumb. This is particularly valuable in heterogeneous environments — cloud VMs, mixed 1G/10G/100G hosts, hosts behind traffic shapers — where a single tuning template performs poorly. The tool is written in a systems-scripting style and requires root for sysctl writes. Limitation: the probing itself generates non-trivial traffic, so it should not be run on production interfaces under load.
Source: https://github.com/Kylin010/tcpfit
calmrocks/ai-engineer-notebooks
A structured curriculum of Colab notebooks covering the practical AI engineering stack, framework-free and runnable on the free Groq API tier. The technical scope is broad but coherent: model API integration, structured output (JSON schema enforcement, function signatures), tool calling, retrieval-augmented generation (chunking, embedding, reranking), and evaluation as a first-class concern rather than an afterthought. The agent section covers building a tool-calling loop from scratch — useful precisely because it avoids framework magic — and addresses guardrails, prompt injection, and MCP (model context protocol) integration. Fine-tuning coverage distinguishes full fine-tuning from LoRA, which is the correct pedagogical separation. The “evals-as-the-spine” framing is the strongest design choice: evaluation infrastructure is introduced early and used throughout rather than bolted on at the end. The framework-free constraint forces explicit understanding of what higher-level libraries abstract away. Suitable for engineers transitioning into applied AI roles who need to reason about the stack rather than just call library APIs. The Groq dependency keeps costs at zero for experimentation.
Source: https://github.com/calmrocks/ai-engineer-notebooks
genspark-ai/genoffice
GenOffice is a cross-platform (macOS, Windows, Linux) open-source office suite with embedded AI agents, targeting the .docx, .xlsx, .pptx, PDF, and Markdown formats. The technical differentiator relative to LibreOffice or OnlyOffice is that AI capabilities are first-class in the editing model rather than add-on plugins — the agent layer can interpret and modify document structure (tables, charts, slide layouts) rather than just appending text. The suite parses and emits the Open XML formats natively, which is the correct choice for compatibility. The AI integration architecture appears to route document-context to an LLM for instruction following — “reformat this table as a bar chart,” “summarize this section” — with the agent having structured access to the document object model rather than operating on raw text. At 3,800+ stars shortly after release, there is clear demand for an open alternative to proprietary AI-augmented office tools. Key open questions: offline/local model support, fidelity of complex .xlsx formula handling, and whether the agent has write-back access to full document semantics or only text regions.
Source: https://github.com/genspark-ai/genoffice
ShawnPana/phone-harness
phone-harness provides an agent control harness for Android and iOS devices, exposing phone UI as an action/observation space for LLM-based agents. The architecture wraps ADB (Android Debug Bridge) and iOS instruments to provide screenshot capture, touch/swipe action execution, and UI hierarchy extraction (via accessibility trees). The agent receives either a pixel-space observation or a structured XML accessibility dump, and emits actions — tap coordinates, swipe vectors, text input — that the harness translates to device commands. This is the standard approach for GUI agents (similar to AppAgent, MobileAgent), but phone-harness frames itself as a composable skill/tool rather than a monolithic pipeline, making it easier to slot into existing agent frameworks. The 2,000+ star count reflects genuine interest in phone automation as an agent capability. Practical uses include automated mobile testing, personal automation tasks, and research on GUI agents. Limitations: coordinate-space actions are brittle across device resolutions; accessibility tree quality varies by app; and iOS requires a connected Mac with developer mode, restricting deployability.
Source: https://github.com/ShawnPana/phone-harness
cristicretu/diri
diri is a native macOS orchestrator for parallel coding agents, solving the concrete problem that running multiple LLM coding agents (Claude Code, OpenAI Codex, Cursor, Gemini) simultaneously on the same codebase produces conflicting file modifications. The solution is git worktrees: each agent session operates in an isolated worktree branched from the same repository, preventing interference. diri manages worktree lifecycle — creation, agent assignment, monitoring, and merge — through a native macOS UI. It also supports remote host dispatch, so agents can run on separate machines while diri provides a unified control plane. The technical value is in the orchestration layer: task routing to specific agents, session state tracking, and the merge/diff workflow that surfaces conflicts between parallel agent branches. This is an under-explored operational problem as coding agents move from single-session tools to multi-agent parallelism. Built as a native macOS app (Swift/SwiftUI inferred), which limits portability but provides tight OS integration. The remote host support suggests it is designed for developers running agents on cloud GPU instances or beefy workstations while using a laptop as the control surface.
Source: https://github.com/cristicretu/diri
tt-a1i/simplify-codebase
simplify-codebase addresses accidental complexity in codebases: dead code, redundant abstractions, over-engineered indirection that accumulated without deliberate design. The approach is behavioral equivalence verification — it attempts to prove that a proposed simplification does not change observable behavior before applying it. The technical core relies on static analysis to identify candidates (unreachable code paths, unused parameters, functions called only once that could be inlined) combined with either formal lightweight verification or test-suite coverage checks to validate that removal is safe. This is adjacent to compiler dead code elimination but operates at a higher semantic level — targeting design-level complexity rather than just bytecode. The “prove” framing is the strongest and most contentious claim: full behavioral equivalence proof for arbitrary code is undecidable, so the implementation necessarily relies on bounded verification (test coverage, symbolic execution of bounded depth, or property-based checks). The practical value is real — codebases accumulate complexity faster than teams manually refactor — but users should verify what “proof” actually means in the implementation before relying on it for safety-critical refactors.