Daily AI Digest — 2026-09-24

Published

September 24, 2026

arXiv Highlights

HappyWorld-Bench

Existing world-model evaluations conflate frame-level visual fidelity with the harder question of whether a generated world remains internally consistent when an agent probes, revisits, or edits it. HappyWorld-Bench targets that gap by defining a hierarchical capability axis W1–W6 and instantiating it across three tracks — video, spatial, and embodied — with a shared scoring rubric and human A/B Elo arena.

Capability hierarchy

The W-axis is the organizing abstraction. W1 covers passive generative construction (natural scene evolution with no control), W2 adds atomic action response, W3 multi-step rollouts with persistent state, W4 counterfactual edits to actions or physical rules under a shared initial state, and W5–W6 multi-agent control and unified world modeling. Levels are qualitative transitions, not just increased difficulty: each new level introduces a distinct consistency requirement (temporal continuity, state persistence, counterfactual invariance of unaffected content, per-subject controllability).

The benchmark comprises 1,138 video prompts, 300 spatial scenes, and 254 embodied test cases, evaluating 14 video, 9 spatial, and 8 embodied systems. All three tracks feed into HappyWorld-Arena, which collects pairwise human A/B comparisons and derives model-level Elo ratings that complement automated behavioral metrics.

Video track

Each case is a tuple (I_0, \text{text}, c_{1:T}): a first frame, a scene/subject description, and an optional time-indexed control sequence. Annotations record target W level, dimension, application domain, image source, and viewpoint. Control specifications are tied to capability level:

W1: no actions; only visual continuity and scene evolution are scored.
W2/W3: control sequences give directions and time intervals rather than absolute target poses, since identical inputs produce different displacements across models — a nontrivial design decision that makes cross-model comparison well-defined without privileging any model’s action calibration.
W4: natural-language instructions for actions, interactions, and environmental-rule edits.
W5/W6: subject-indexed controls so multiple entities can be steered in the same scene.

First frames combine real imagery, rendered environments, and generated fills. The pipeline (Figure 3) enforces a capability matrix over (W level × dimension × domain × viewpoint), which is what allows behavioral gaps to be localized rather than aggregated away.

The reported failure mode of the 14 video models is degradation over extended rollouts and revisits — i.e., temporal consistency and loop closure, not per-frame quality. This is the empirical claim the W3-heavy design is built to surface.

Spatial track

The spatial track evaluates renderable environments (Gaussian splats, meshes, or other explicit representations) built from images and/or descriptions, including hallucinated unobserved regions. Beyond visual quality, it scores whether the environment supports exploration, provides physical support, and preserves existing content under editing or expansion.

Data construction reuses filtered video-track first frames, restricted to static scenes because no current method achieves spatiotemporal world modeling. Manual review excludes scenes containing people or body parts (hands, legs), and descriptions are rewritten to strip first-/third-person phrasing that would leak agent context into a scene-level generator. The abstract notes spatial models “achieve at be[st]” — the reliability ceiling on preservation-under-edit is the headline result for this track.

Embodied track

Embodied cases are formulated as image-conditioned, prompt-controlled video generation from an egocentric robot viewpoint:

Y_{1:T} = G_\theta(I_0, p),

where p specifies acting subject, target, action type, direction, and — for multi-step tasks — ordered stages with time intervals. The current interface is natural-language, with a stated extension path to discrete robot actions.

Only W2–W4 are operationalized. W2 tests atomic actions (movement, manipulation); W3 tests ordered multi-step rollouts with persistent state across movement-only, manipulation-only, and combined regimes; W4 tests paired rollouts from a shared I_0 under either a changed action or a changed physical rule, targeting counterfactual invariance of unaffected content.

The scoring axis has four dimensions kept deliberately separate:

Perception: scene/robot/objects recognizable under the prescribed input.
Consistency: appearance, geometry, state, temporal continuity across rollout.
Causality: contact events, ordering, and state changes have plausible physical/semantic causes.
Controllability: the specified action is faithfully executed, including intermediate process and terminal state.

The authors motivate the decomposition explicitly: a rollout can reach a plausible terminal state while omitting the grasp, moving the object in the wrong direction, or changing the scene with no visible trigger — failures a holistic judgment collapses. This is the strongest methodological argument in the paper: it makes controllability failures distinguishable from causal-plausibility failures, which matters because they imply different fixes (action grounding vs. physics prior).

Limitations and open questions

W1, W5, W6 are not operationalized in the embodied track and remain definitional. The spatial track restricts to static scenes. Action interfaces are natural-language rather than low-level control, so results couple prompt-following capacity with world-model fidelity — a confound the paper acknowledges but does not disentangle. The abstract is truncated at reported spatial numbers, so exact Elo/behavioral scores per model are not reproducible from the excerpt provided. It is also unclear how inter-annotator agreement is handled in Arena Elo computation, or how many pairwise comparisons back each rating.

Why this matters

World-model progress has been measured largely by per-frame video quality, which is uninformative about the properties that matter for downstream agents: state persistence, counterfactual locality, and action fidelity. HappyWorld-Bench operationalizes those properties across three modalities under one capability hierarchy, and its four-way scoring decomposition is directly actionable for diagnosing where a given model fails.

Source: https://arxiv.org/abs/2609.24308

The Past Frames the Future: Memory for Autoregressive Video Generation

Autoregressive (AR) video generation has become the dominant paradigm for long-horizon rollouts, interactive world models, and streaming visual environments. The core bottleneck is not fidelity per frame but persistence across frames: causal rollouts must extend indefinitely, yet transformers, diffusion backbones, and their KV caches operate under strictly bounded context windows. Entity identities, object permanence under occlusion, scene layouts, and the causal consequences of past interventions routinely fall out of the active window well before their influence on future generation is exhausted. This survey formalizes the resulting “memory problem” and organizes the rapidly growing literature into a coherent taxonomy.

Operational definition of memory

The authors define memory operationally rather than architecturally: memory is persistent historical information maintained across outer AR steps, capable of influencing future generation after the originating evidence has left the local context. This distinction — outer AR rollout vs. inner (per-frame or per-chunk) generation — is important because it separates memory (cross-step persistence) from intra-step attention or denoising state. The unified framing in Figure 2 shows that discrete-token AR and continuous frame-/chunk-wise AR (e.g. diffusion forcing, rolling diffusion) share the same causal outer loop, and thus admit the same memory formalism despite very different inner generators.

Overview of AR video generation and memory-conditioned generation.

Four carrier types

The survey’s central organizing axis is the carrier — the computational object in which history is retained. Four families are distinguished (Figure 3):

Four memory carrier families and representative realizations.

Visual memory preserves past frames, clips, or observation-aligned latents. Formally, \mathbf{M}_{n}^{\mathrm{vis}}=\{\mathbf{m}_{i}^{\mathrm{vis}}=E_{\mathrm{vis}}(\mathbf{y}_{i})\mid i\in\mathcal{I}_{n}^{\mathrm{vis}}\},\quad \mathcal{I}_{n}^{\mathrm{vis}}\subseteq\{1,\ldots,n-1\}, where E_{\mathrm{vis}} is the identity (pixel-space) or a VAE encoder. The defining property is traceability to a specific historical observation. This preserves appearance fidelity and spatial detail with minimal abstraction, at the cost of storage that scales with retained frames and of redundancy across visually similar observations.
Implicit state memory stores history in model-native latent states — KV caches, recurrent hidden states, compressed latent tokens — without imposed semantic or geometric structure. These are compact and directly consumable by the backbone but opaque and hard to edit.
Explicit state memory represents history through structured, interpretable variables: entity slots, object-centric embeddings, scene graphs, 3D layouts, occupancy fields, event tuples. These support compositional edits and causal interventions but require reliable structure extraction and impose an ontology.
Adaptive parametric memory encodes sequence-, instance-, or experience-specific information in weights that are updated online — LoRA adapters, fast weights, test-time-trained modules, or continually fine-tuned parameters. Persistence lives in the parameters themselves rather than in an auxiliary state.

The first three are non-parametric runtime carriers; the fourth relocates persistence into parameters. Crucially, the taxonomy is defined by form, not by function: a single pipeline commonly mixes visual anchors (e.g. a keyframe cache), a compressed implicit summary, structured entity slots, and a LoRA-based user/scene adapter.

Cross-cutting properties and the four-axis landscape

Beyond form, the survey characterizes carriers along orthogonal properties: storage growth (bounded vs. unbounded in n), unit structure (token, patch, slot, frame, graph), access key (positional, content-based, semantic), interpretability, editability, and model coupling (frozen vs. jointly trained backbone). These properties predict downstream trade-offs — e.g., unbounded visual caches maximize fidelity but preclude infinite rollout without retrieval; bounded implicit summaries enable constant-cost inference but risk irreversible information loss.

The full landscape (Figure 1) crosses this Forms axis with three others: Functions (what the memory should preserve — identity, geometry, dynamics, causality), Operations (write, read, update, forget, retrieve — the lifecycle under bounded budgets), and Learning (how memory behavior is trained: end-to-end, auxiliary objectives, retrieval-augmented pretraining, RL-based gating, or test-time adaptation).

Landscape of memory mechanisms across Forms, Functions, Operations, and Learning.

This four-axis decomposition is the survey’s main contribution: it lets otherwise incomparable systems — a KV-compression world model, a slot-attention scene tracker, and a LoRA-personalized video diffusion model — be placed in a common design space and compared on the axes that actually matter for long-horizon behavior.

Limitations and open questions

The survey is descriptive rather than empirical: it does not benchmark carriers against one another on shared long-horizon tasks, and there is no unified evaluation protocol for memory-specific failure modes (identity drift, object permanence, causal consistency after intervention). Several tensions remain open. How should hybrid systems allocate budget across carriers — when does an explicit entity slot outperform a visual keyframe cache of the same footprint? What are the right benchmarks for causal persistence, as opposed to visual quality metrics like FVD? Can adaptive parametric memory be updated online without catastrophic drift in long rollouts? And how does memory interact with test-time compute scaling in diffusion-based inner generators?

Why this matters

Long-horizon AR video generation is bottlenecked not by per-frame quality but by what the model can carry forward under bounded compute. A carrier-centric taxonomy with explicit operations and learning axes gives a principled design vocabulary for systems that must remain coherent over thousands of frames — a prerequisite for usable world models, interactive simulators, and streaming video agents.

Source: https://arxiv.org/abs/2609.28466

PACT: From Credit Assignment to Critic Alignment

The axiomatic characterization of token-level credit

The paper’s starting point is that “token-level credit” is used constantly in LLM RL but never mathematically pinned down. The authors impose three regularity conditions on a credit assignment \{C_i\} derived from a trajectory Y=(q,T_1,O_1,\dots,T_\tau,O_\tau) with terminal reward R=\mathcal{R}(Y):

Completeness: \sum_i C_i = R - \mathbb{E}[R\mid q].
Prefix Consistency: C_i is \mathcal{F}_i-measurable and its conditional expectation given the past collapses correctly.
Neutrality: no dependence on tokens that carry no incremental information about R.

Under these, the unique solution is the Doob-martingale increment

C_i^\pi = V_i^\pi - V_{i-1}^\pi,\qquad V_i^\pi = \mathbb{E}_\pi[R\mid \mathcal{F}_i].

This is a clean way to say that token-level credit is not a design choice — it is fully determined once you fix the policy and the outcome reward. Coarser schemes (turn-level, response-level) are aggregations of this object.

Reinterpreting OPD, RLOO, and GAE

Using this representation, the paper reads several existing algorithms as approximations to C_i^\pi.

On-policy distillation as an implicit critic. For sampled-token OPD with teacher q_t and student p_t, the per-token signal is A_t^{\mathrm{OPD}}(q_t) = \log q_t(T_t)/p_t(T_t). Defining the ideal teacher as the KL-regularized reward-improvement solution

q_t^\star \in \arg\max_{q\in\Delta(\mathcal V)} \Big\{ \mathbb{E}_{a\sim q}[Q_t^\pi(a)] - \beta D_{\mathrm{KL}}(q\|p_t)\Big\},

one gets q_t^\star(a) \propto p_t(a)\exp(Q_t^\pi(a)/\beta), so \log q_t^\star/p_t = Q_t^\pi/\beta - \log Z_t. The expected score-function update under this signal is proportional to the policy gradient induced by the unique token-level credit. In other words, an ideal OPD teacher plays the role of a critic without ever instantiating a value head.

RLOO matches in expectation despite coarser granularity. Response-level REINFORCE-Leave-One-Out uses one scalar advantage per trajectory, yet its expected gradient contribution equals that of \sum_i C_i^\pi. The finer resolution of C_i^\pi only reduces variance; it does not change the target.

GAE suffers under credit sparsity. The paper further argues (Theorem 4) that when R is bounded, most C_i^\pi are approximately zero over long horizons — credit is sparse in a well-defined sense. Since C_i^\pi = V_i^\pi - V_{i-1}^\pi is a difference of two nearby values, even modest critic error \varepsilon can be of the same order as the true signal, and GAE’s bias–variance tradeoff via \lambda does not resolve this.

The core observation: policy–critic lag

Section 4 identifies a specific failure mode of PPO-style training. In iteration k, trajectories \mathcal D_k\sim\pi_k are scored with a critic \widehat V_{\phi_{k-1}}\approx V^{\pi_{k-1}} trained on the previous batch. The critic is then updated on \mathcal D_k, but that update is not used to recompute advantages for the current actor step. Because C_i is a difference of adjacent values, this one-step lag between actor and critic can dominate the signal — exactly the regime Theorem 4 flags as dangerous.

PACT

Policy Aligned Critic Training reorders the update: for each rollout batch, first fit the critic on \mathcal D_k until \widehat V_{\phi_k}\approx V^{\pi_k}, then compute advantages with this synchronized critic, then update the actor. Importance sampling handles the off-policy nature of the intra-batch minibatches; critic training masks samples whose importance ratio falls outside [0,6], and the actor uses either DIS (mathematical reasoning) or PPO clipping (coding). The intent is to eliminate the \pi_{k-1}\to\pi_k lag so that \widehat V_{\phi_k}(\mathcal F_i) - \widehat V_{\phi_k}(\mathcal F_{i-1}) approximates C_i^{\pi_k} rather than a stale mixture.

Results

On mathematical reasoning with Qwen3.5-4B trained on a 3.2k-problem subset of DAPO-Math-17k (Avg@16 accuracy):

Method	AIME 2025	AIME 2026	BeyondAIME	HMMT Nov 2025	Average
Base	46.67	51.25	28.63	37.50	41.01
PPO (\lambda=0.95)	32.33	29.38	17.86	25.67	26.31
PPO (\lambda=1.0)	66.04	73.96	40.31	58.54	59.71
GRPO (\epsilon_{\mathrm{high}}=0.28)	76.50	74.78	41.81	63.19	64.07
SAO	51.25	63.33	36.63	53.33	51.14
PACT w/o IS	76.04	82.50	51.38	61.04	67.74
PACT	83.12	85.21	51.69	71.46	72.87

The 26.6-point gap between PPO with \lambda=0.95 and \lambda=1.0 is itself evidence for the credit-sparsity concern: pushing \lambda\to 1 downweights the biased intermediate critic values that GAE mixes in. PACT beats GRPO by 8.8 average points and PPO (\lambda=1.0) by 13.2, and the ablation “PACT w/o IS” shows that policy–critic synchronization alone contributes most of the gain (+8.0 over GRPO), with importance sampling adding another 5.1.

Limitations

The theorems assume bounded R, full-support p_t, and a fixed ideal teacher per update; real distilled teachers do not solve the KL-regularized argmax exactly. The credit-sparsity result is asymptotic and does not quantify how the constants scale with vocabulary size or context length. Empirically, evaluation is limited to two domains (math reasoning, agentic coding via OpenSWE — coding numbers not shown in the excerpt), one model family (Qwen3.5-4B, Qwen3.6-35B-A3B), and pass@1-style outcome rewards; whether PACT’s advantage persists with process rewards, verifier ensembles, or dense environmental feedback is untested. The critic-first schedule roughly doubles per-iteration critic compute.

Why this matters

The paper turns a folk concept — “token-level credit” — into a uniquely determined object, then uses that object to diagnose a concrete, fixable bug in the standard PPO loop: critics used for advantage computation are always one policy step stale, and this stale gap is on the same order as the signal you are trying to extract. PACT is a small change to the training schedule with a large, reproducible effect on math-reasoning benchmarks.

Source: https://arxiv.org/abs/2609.26355

InternW0: A Foundational Physical World Model for Efficient Real-World Interactions

Problem

World-action models (WAMs) that couple future visual prediction with action generation face a structural tension: high-capacity video prediction is too slow to run at control frequency, but conditioning the policy on stale predictions degrades closed-loop behavior as the environment drifts from its predicted trajectory. Existing WAMs either regenerate the full plan at every control step (expensive) or freeze it across an action chunk (stale). InternW0, the first release in Shanghai AI Lab’s InternW series, targets this tension explicitly, while also supporting heterogeneous embodiments (single-arm, dual-arm, mobile, dexterous hand) and contact-rich modalities (force, tactile).

Method

InternW0 is a mixture-of-transformers (MoT) with two modality-specific experts sharing joint layers: a high-capacity Video-DiT for future visual dynamics and a lightweight Action-DiT for flow-matching-based control. Each joint layer contains separate parameters and token streams, coupled through an observation-conditioned interface.

Architecture of InternW0 showing video and action experts coupled through an observation-conditioned chunk K/V editor.

The pipeline is asymmetric. The video expert consumes past frames (encoded by a frozen Wan VAE) and language embeddings, producing a shared predictive representation without seeing robot-specific action tokens or domain identities. The action expert reads (i) DINOv3 features of the current chunk-level observation, (ii) proprioception, (iii) language, (iv) embodiment-specific interfaces and soft prompts, and (v) the video context via cross-attention.

The central mechanism is asynchronous duplex inference with an observation-conditioned chunk K/V editor. Rather than rerunning the video expert per control update, InternW0 caches its layerwise K/V tensors and edits them conditioned on the newly observed chunk state before the action expert attends to them. Formally, for each layer the cached video context (K_v, V_v) is refreshed as

(\tilde{K}_v, \tilde{V}_v) = \mathrm{Edit}\big((K_v, V_v)\,;\, o_t\big),

where o_t is the latest observation embedding. The Action-DiT then denoises a short action chunk a_{t:t+H} via flow matching, cross-attending to (\tilde{K}_v, \tilde{V}_v). The Video-DiT is refreshed on a slower schedule, keeping expensive prediction off the critical control path.

Actions are represented in a unified 37-dimensional space covering single-arm, dual-arm, and mobile morphologies; unused dimensions are masked. Contact-aware post-training introduces force and tactile histories as additional action-side observations and jointly predicts future contact signals, without modifying the video backbone.

Data

Pretraining uses ~7,233 hours across 811,969 episodes from four real-robot datasets (AgibotWorld, Galaxea, RoboCOIN, MolmoAct), two simulated datasets (InternData-A1, RoboDojo), and 275.4 hours of egocentric wet-lab video (EgoLab, 3,192 episodes) collected in-house. Datasets are sampled proportional to \sqrt{n_{\text{anchors}}}. Cleaning removes static trajectories, head/waist-motion intervals, gripper micro-pulses, and out-of-range values via validity masks on each dataset’s native timeline. Twenty-five training domains and 24 statistics groups are configured; EgoLab contributes only video supervision.

Results

LIBERO (full setting, 40 tasks, 50 rollouts each). InternW0 reaches 98.6% average success across Spatial/Object/Goal/Long, edging out LingBot-VA (98.5) and Fast-WAM (97.6). Per-suite: Spatial 99.4, Object 99.4, Goal 98.6, Long 97.0. The gain is largest on Spatial (best in table) while Long is slightly behind LingBot-VA (98.5) and Motus (97.6).

RoboTwin 2.0 is used to contrast Full vs Clean2Random regimes, isolating pretraining-driven generalization when only clean demonstrations are available for adaptation but randomization appears at test time. The paper positions this as evidence that the pretrained physical representation transfers beyond the adaptation distribution (specific numbers omitted in provided excerpt).

Real-robot. Five tasks, four dual-arm (Make Sandwich, Pick Industrial Parts, Sort Tubes, MoF Experiments) plus one dexterous-hand task (Quantitative Pipetting). MoF Experiments comprises 15 sequential subtasks reproducing metal-organic-framework solution preparation—pour, funnel removal, transfer, stopper insertion, stirrer activation—requiring precise insertion and progress tracking across visually similar stages.

Efficiency. On an RTX 5090D, InternW0’s critical-path action latency is 60.73 ms, yielding a maximum policy update rate of 16.47 Hz. This is a 3.13\times speedup over Fast-WAM. Critically, this excludes Video-DiT inference, which runs on the asynchronous predictive path; observation encoding, chunk K/V editing, and Action-DiT denoising are what block the next action.

Limitations and open questions

The reported 16.47 Hz measures model-side latency only; end-to-end closed-loop stability also depends on how stale (\tilde{K}_v, \tilde{V}_v) can become before the K/V editor’s local adaptation fails. The paper does not quantify degradation as a function of the video-refresh period.
Dexterous-hand data was excluded from pretraining, yet dexterous pipetting is evaluated—transfer here rests entirely on post-training and the shared representation.
The 37-D unified action space masks unused dimensions; how this scales to substantially different morphologies (humanoids, soft robots) is unaddressed.
No ablation is shown (in the provided sections) isolating the contribution of the chunk K/V editor versus simply caching an unedited video context, nor the marginal value of EgoLab-only video supervision.
Contact-aware post-training is described but tactile/force benchmark numbers are not surfaced in the excerpt.

Why this matters

Decoupling predictive video modeling from control via a cached, observation-edited K/V interface is a clean architectural answer to the freshness/cost tradeoff that has bottlenecked WAMs. If the editor’s local correction is faithful enough, this pattern generalizes to any prediction-conditioned policy where the predictor is an order of magnitude slower than the controller—which describes most emerging embodied foundation models.

Source: https://arxiv.org/abs/2609.27656

Spatial-Interactor: Learning Spatial Reasoning through Interaction with the Observable Physical World

Problem

Vision-language models trained on static spatial QA — “is the chair left of the table?”, “what color is the mug?” — internalize snapshot relations but not the transitions between them. In embodied settings, an observation O_t is a joint function of world state W_t and observer pose E_t, and reasoning correctness hinges on attributing an observed change to the right source (object motion vs. viewpoint shift) and integrating many such attributions across a trajectory. The authors show diagnostically that current VLMs are weakly sensitive to frame order and degrade sharply on long-horizon reasoning without explicit transition traces.

State-transition diagnostics: frame shuffling exposes weak temporal-order sensitivity; explicit local-transition traces improve long-horizon reasoning.

The gap this paper targets is that static QA does not provide direct supervision for the map (O_t, a_t) \mapsto O_{t+1}, whereas interaction trajectories do.

Method

The core object is an interaction trajectory decomposed into local records:

\mathcal{T} = (\delta_1, \ldots, \delta_{T-1}), \quad \delta_t = (O_t, a_t, O_{t+1}),

where a_t can be an object manipulation, an environmental event, or a camera motion (translation/rotation). Each \delta_t is verifiable against interaction metadata: given the recorded action, the transition between O_t and O_{t+1} has a ground-truth attribution (which entity changed, how the pose moved), so it can be converted into a QA/target pair without human annotation. Composing \delta_t into \mathcal{T} then generates long-horizon targets that require chaining local inferences.

Training is organized as a three-level curriculum (see overview):

L1 — passive world-state transitions: W_t changes, E_t fixed. The model must localize and describe what object-level change occurred.
L2 — active self-state transitions: E_t changes (camera translates/rotates), W_t fixed. The model must recover the viewpoint delta.
L3 — long-horizon trajectories: chained \delta_t where both may change, and the model must maintain a coherent running spatial state.

Overview of Spatial-Interactor: three-level curriculum and two-stage training.

The corresponding dataset, LSI-108K, is constructed from executed interactions (for L1/L2 attribution) and geometric camera trajectories (for L2/L3 pose deltas), which are automatically converted into verifiable transition targets and QA pairs.

Optimization proceeds in two stages:

SFT on 82,596 L1–L2 LSI examples plus ~80K public spatial QA (VSI-590K, MindCube, VSTI-Bench). This installs local-transition competence.
Observable Process Distillation (OPD) on 10,712 L3 RoomTour and 10,783 long-horizon VSTI-Bench examples, starting from the SFT checkpoint. OPD extends a GRPO-style RL loop with (a) privileged conditioning — a teacher rollout has access to ground-truth process traces (the sequence of verified \delta_t attributions) — and (b) process distillation — the student is rewarded not only on the final answer but on matching the teacher’s intermediate transition trace. The matched GRPO baseline is identical (init, data, prompts, rollout groups, answer rewards, schedule) except for these two additions, isolating the contribution of process-level supervision.

Both stages update the LM and the multimodal projector; the visual encoder is frozen. The framework is instantiated on Qwen2.5-VL-3B/7B and Qwen3-VL-4B/8B. Training and evaluation splits are disjoint at both scene and video granularity, which matters because interaction datasets otherwise leak spatial layout.

Conventional spatial QA retains isolated evidence; Spatial-Interactor composes local transitions into a coherent spatial state.

Results

The provided sections quantify the training regime rather than benchmark deltas: SFT uses 82,596 L1–L2 LSI examples plus ~80K public spatial QA; OPD uses 10,712 L3 RoomTour examples and 10,783 long-horizon VSTI-Bench examples. The diagnostic figure establishes two empirical claims that motivate the design: (i) frame-shuffling barely changes baseline VLM outputs, indicating that current models do not exploit temporal order, and (ii) supplying explicit local-transition traces at inference time improves long-horizon reasoning, indicating that the bottleneck is process representation rather than final-answer capacity. Controlled ablations in the paper isolate the contributions of local-transition supervision (SFT stage) and OPD (RL stage), and closed-loop evaluations test whether the model handles cases in which its own action changes subsequent observations — the setting where trajectory-level composition is unavoidable.

Limitations and open questions

The verifiability of \delta_t depends on interaction metadata being clean; real robot logs contain noisy actions and partial observability, which would degrade automatic target construction. OPD requires a privileged teacher with access to ground-truth process traces, so the method is currently tied to simulated or heavily annotated environments. The curriculum treats “world change” and “self change” as clean categories, but many natural interactions (grasping while walking) mix them; whether the L1/L2 factorization holds up under simultaneous changes is not addressed in the excerpts. Finally, the visual encoder is frozen, so any perceptual bottleneck in attributing subtle geometric changes propagates unfixed.

Why this matters

Spatial reasoning benchmarks have converged on static relations, but embodied deployment requires modeling how observations transform under actions. Spatial-Interactor operationalizes this by turning verifiable interaction records into direct supervision for the transition operator, and by distilling process traces rather than only final answers — a template that generalizes beyond spatial QA to any domain where intermediate state is auditable.

Source: https://arxiv.org/abs/2609.23038

RewardVerse: Rubric-Guided Policy Optimization for Video Reward Modeling

Problem: scalar drift in video reward models

Video reward models trained to emit a single scalar S(y\mid q)\in[1,5] for a generated video y under query q tend to collapse in two distinctive ways that the authors formalize as scalar drift:

Score-range collapse. Pointwise scores cluster in a narrow high band, so the gap between preferred and dispreferred videos shrinks. Downstream RL sees near-flat advantages.
High variance under context shift. Paraphrases of the query or reordered inputs shift the effective scale, producing inconsistent scores for identical content.

This is a real bottleneck: preference-based RL fine-tuning of diffusion/video models depends on a reward with (a) resolution across the video distribution and (b) invariance to prompt phrasing. Direct scalar heads on MLLMs satisfy neither.

Comparison of unconstrained scoring vs. rubric-anchored scoring.

Method: dynamic rubric as an intermediate representation

RewardVerse decomposes scoring into two learned steps sharing one policy \pi_\theta (built on Qwen2.5-VL-7B):

Given query q=(d,p) — a dimension d\in\mathcal{D} and the text prompt p — the policy first emits a dynamic rubric r\sim\pi_\theta(\cdot\mid q): an explicit set of evaluation criteria (themes, tips) adapted to the target dimension and the prompt content.
Conditioned on (q,r,y), the same policy emits a scalar score s\sim\pi_\theta(\cdot\mid q,r,y).

The rubric serves as a semantic anchor: because the score is now conditioned on explicit criteria, the mapping from video content to [1,5] has a stable meaning that survives paraphrases of q. This is the mechanistic claim behind Figure 1 — inserting r short-circuits the drift path.

RGPO: two-stage optimization

Training uses Rubric-Guided Policy Optimization (RGPO), a two-stage scheme summarized in Figure 3.

RGPO stages: scorer warm-up on seed rubrics, then joint rubric-generator training with continued calibration.

Stage 1 — scorer warm-up. Fix rubrics to self-evolving seed rubrics r_{\text{seed}} (bootstrapped, then iteratively refined) and train the scorer with:

a preference reward enforcing pairwise ordering consistent with human ratings,
a format reward enforcing structured score outputs,
a calibration loss aligning pointwise scores with human labels (mitigating range collapse).

Stage 2 — joint optimization. Unfreeze the rubric generator. The policy now samples r\sim\pi_\theta(\cdot\mid q) and s\sim\pi_\theta(\cdot\mid q,r,y), and receives a scalar reward from the calibrated preference/format objectives. The scorer branch continues under the calibration loss. This is what turns the rubric from a static prompt scaffold into a query-adaptive intermediate representation.

Videos are scored independently rather than in pairs, sidestepping the pairwise position bias the authors document in their appendix analysis.

Results

Evaluation covers 16 secondary dimensions on EvalVerse (pointwise PLCC/SRCC vs. human ratings) and VGRB (26.5K pairs) for pairwise transfer.

On EvalVerse (macro-average over the 10 dimensions supported by all baselines):

RewardVerse (Joint): PLCC 0.554, SRCC 0.446.
Best prior model, Q-Scorer: PLCC 0.467, SRCC 0.354.
VideoScore2: 0.433 / 0.427; VisionReward: 0.408 / 0.351; VideoReward: 0.384 / 0.399; UnifiedReward: 0.334 / 0.328; VideoScore-v1.1: −0.033 / 0.132.

So RewardVerse improves PLCC by roughly 0.087 absolute (≈19% relative) over the strongest external RM, and SRCC by ≈0.05.

Ablating the pipeline is where the argument sharpens:

Raw-seed Rubric (zero-shot scoring with seed rubric, no training): PLCC 0.311, SRCC 0.275.
Raw-rubric (dynamically generated but uncalibrated rubrics, zero-shot scoring): PLCC 0.379, SRCC 0.293.
Joint (full RGPO): PLCC 0.554, SRCC 0.446.

Dynamic rubrics alone give a modest PLCC lift over seed rubrics (+0.068) but SRCC is essentially flat. Joint calibration is what closes the gap — the scorer must be trained to use the rubric, not just receive it. Notable per-dimension jumps under Joint include Expression 0.566 (from 0.144/0.277), Pace 0.471 (from 0.079/0.075), and Progression 0.510 (from 0.225/0.183) in PLCC.

Qualitatively, the learned rubric generator adapts to the query. For a Visual Quality query, themes and tips remain generic (chromatic fidelity, lens behavior). For a Consistency query, the tips ground onto specific entities named in the prompt.

Dynamic rubric samples: abstract themes for Visual Quality vs. entity-grounded tips for Consistency.

Limitations and open questions

Reported gains ride on a single 7B backbone (Qwen2.5-VL-7B); it is unclear how much of the benefit depends on the base MLLM’s instruction-following. A smaller backbone may lack the capacity to both generate useful rubrics and score.
The rubric introduces additional inference cost and latency (generate r, then score); the paper does not quantify the RL wall-clock impact when RewardVerse is plugged in as a reward.
SRCC on some dimensions remains low (Character 0.146, Compositional-related 0.243), suggesting rank order is still noisy where the rubric provides weak discriminative signal.
Absolute PLCC ≈0.55 is a meaningful gain but far from ceiling; human agreement upper bounds are not reported here.
Robustness to adversarial paraphrases of q — the phenomenon that motivated the work — is argued mechanistically but not directly quantified in the reported results.

Why this matters

Rewards for video generation RL have been the weakest link: scalar heads on MLLMs drift and squash. RewardVerse shows that treating the rubric as a learned intermediate variable, and co-training it with the scorer under a calibration objective, converts an MLLM into a reward model with meaningful pointwise resolution — a prerequisite for stable RL fine-tuning of video generators.

Source: https://arxiv.org/abs/2609.22947

Just-in-Time Memory: Learning to Curate Task-Adaptive Memory for LLM Agents

Problem

Existing agentic memory systems — reflections (Reflexion), workflows (MemP), skills (SkillOS), reasoning strategies (ReasoningBank) — commit to a fixed abstraction at write time. Once a trajectory is completed, it is distilled into a query-independent artifact and retrieved later by similarity. Two structural problems follow:

Irreversible information loss. The distillation must anticipate all possible future queries, so any detail the summarizer discards is gone. Different future tasks would benefit from different projections of the same raw trace.
Long-horizon credit assignment. The utility of a write-time decision only manifests when a semantically related task arrives, possibly many steps later. Training a write-time curator therefore requires either delayed reward propagation or hand-crafted task groupings.

JitMem inverts the schedule: keep raw trajectories, curate at read time, when x_t is known.

Method

The system has four components; only the curator \pi_\phi is trained. The executor \pi_L and retriever \mathcal{R} are frozen.

Overview of JitMem. Retriever fetches raw trajectories; curator distills them conditioned on the current task into a task-adaptive payload injected into the executor context; successful trajectories are gated back to the memory bank.

For each task x_t in a stream \{x_1,\ldots,x_T\}:

Retrieve. \hat{\bm{\xi}}_t = \mathcal{R}(x_t, \mathcal{M}_t): top-k raw trajectories via BM25 over task descriptions only (not trajectory bodies — cheap and length-independent).
Curate. p_t = \pi_\phi(x_t, \hat{\bm{\xi}}_t): a compact natural-language briefing conditioned on the current task.
Execute. (\xi_t, r_t) = \pi_L(x_t, p_t) with r_t \in [0,1] from the environment.
Update. Append \xi_t to \mathcal{M} if an LLM-as-judge (the executor itself) rules the trajectory successful. The payload p_t is ephemeral and never stored.

The objective is straightforward: \max_\phi \; \mathbb{E}\!\left[\sum_{t=1}^T r_t\right].

Because p_t is consumed on the same task that generated the reward, credit assignment collapses to a one-step problem — the curator can be trained by policy-gradient-style RL directly against immediate task success, with no need to attribute a storage decision to a distant future query or to artificially cluster related tasks. This is the main structural advantage of deferred curation.

The memory bank stores unabstracted (x, o_1, a_1, \ldots, o_n, a_n) tuples. Preserving raw traces is essential: the same trajectory can be projected differently for different x_t. Gating by executor-as-judge (rather than storing all trajectories with success/failure labels) is ablated in Section 4.2; positive-exemplar-only storage wins.

Results

Evaluation is on ALFWorld (SR and Score) and WebShop (SR), across three frozen executors (Qwen3-8B, Gemini-2.5-Pro, and a third block truncated in the excerpt). Numbers are mean over 3 seeds with different task orderings. “-base” denotes a prompted (untrained) curator.

Qwen3-8B executor. No-memory baseline: 47.9 / 33.3 / 9.8 (ALFWorld-SR / ALFWorld-Score / WebShop-SR). The strongest prior-art baseline is SkillOS at 61.2 / 40.6 / 16.5. JitMem-base (prompted only) is roughly on par or worse: 60.5 / 32.5 / 11.7 — importantly, on WebShop the untrained curator underperforms the no-memory baseline (32.5 vs 33.3 Score), showing that read-time synthesis without training is not automatically helpful. RL-trained JitMem reaches 77.4 / 61.1 / 32.8, absolute gains of +16.2, +20.5, +16.3 over SkillOS. The WebShop jump from 16.5 to 32.8 SR (roughly doubling) is the most striking; it also converts the negative delta of the prompted curator (−0.8 Score) into a +20.5 gain, indicating that essentially all of the value comes from training \pi_\phi against r_t.

Gemini-2.5-Pro executor. No-memory: 66.4 / 48.6 / 38.4. Best baseline SkillOS: 80.2 / 56.0 / 41.3. JitMem-base already reaches 80.0 on ALFWorld-SR, on par with the strongest baseline before training. (The excerpt truncates the trained-JitMem row here.) That the prompted curator matches SkillOS on a stronger executor is consistent with the idea that a capable executor can exploit read-time task-conditioned summaries even without a learned curator; the training gain then compounds on top.

A secondary observation from the table: swapping curator LLMs (Qwen3-8B vs Gemini-2.5-Pro) inside baselines like ReasoningBank, MemP, and SkillOS shifts scores by only a few points (e.g., ReasoningBank 71.4 → 78.6 on ALFWorld-SR with Gemini). The JitMem gap over the best baseline is substantially larger than any curator-model swap, arguing the improvement is not just from a stronger writer.

Limitations and open questions

Retrieval uses BM25 over task descriptions only; the curator never influences what raw traces it sees. A jointly trained retriever could either help (better coverage of relevant strategies) or hurt (harder credit assignment).
Storage cost grows with raw trajectories. The paper does not (in this excerpt) discuss bank size limits or eviction; long deployments will require compaction — reintroducing a write-time decision.
LLM-as-judge gating with the executor as judge risks systematic bias: trajectories the executor thinks succeeded (but did not) will be stored as positives, and vice versa.
The one-step credit assignment is clean only because p_t is not stored. Any cross-task payload reuse would reintroduce the delayed-utility problem the paper avoids.
Both benchmarks (ALFWorld, WebShop) have short trajectories and narrow task distributions. Behavior at longer horizons or with heterogeneous task streams is untested here.

Why this matters

Deferring memory curation from write time to read time removes the long-horizon credit-assignment problem that has quietly bottlenecked learned memory systems, letting the curator be trained end-to-end from immediate task reward. The empirical gap (e.g., WebShop SR nearly doubling on Qwen3-8B) suggests that the write-time inductive bias — reflections, skills, workflows — has been leaving substantial signal on the table.

Source: https://arxiv.org/abs/2609.27334

Hacker News Signals

Claude discovers a novel enzyme system with CRISPR-like repeats

Anthropic reports that Claude identified a previously uncharacterized enzyme system containing CRISPR-like repeat sequences while assisting with genomic data analysis. The core technical claim is that the model, operating over large biological sequence datasets, flagged a pattern of clustered repeats associated with novel protein-coding regions that do not match known CRISPR-Cas architectures in curated databases. The putative system involves repeat arrays flanking what appear to be effector proteins with no close homologs in UniProt or NCBI nr, suggesting either horizontal gene transfer from an unsequenced lineage or convergent evolution of a repeat-based defense or editing mechanism.

The methodological detail worth noting: Claude was used iteratively — researchers would pose bioinformatic queries, Claude would suggest sequence alignment strategies and flag anomalies, and wet-lab follow-up was used to validate expression and activity. This is not autonomous discovery in the strong sense; it is AI-assisted hypothesis generation with human experimental closure. The enzyme’s biochemical activity has not been fully characterized at the time of the announcement, so calling it “CRISPR-like” is a structural observation, not a functional equivalence claim.

Open questions are significant: whether the repeat arrays actually mediate any adaptive immunity function, whether the associated proteins have nuclease activity, and whether the system is broadly distributed across prokaryotes or is a narrow lineage artifact. The absence of a preprint or peer-reviewed paper accompanying the announcement makes independent scrutiny impossible at this stage.

This kind of AI-in-the-loop genomic mining is technically interesting because metagenomics databases have grown faster than human bandwidth to analyze them, and pattern-recognition over sequence space is a plausible genuine use case — but extraordinary claims about novel biology require the standard extraordinary evidence.

Source: https://www.anthropic.com/news/claude-discovers-novel-enzyme-system

OpenAI GPT-6 Astra breaks Enigma message that has resisted solution since 2005

The MVUEH message is one of the last unsolved wartime Enigma intercepts, a ciphertext that has resisted cryptanalysis for two decades due to a combination of an unknown wheel order, unknown plugboard settings, and unusually short message length — all of which make brute-force exhaustion expensive and statistical attacks underpowered. The standard attack surface for Enigma is index-of-coincidence analysis and known-plaintext cribs; short messages lack the statistical mass for reliable IC-based rotor identification, and without a crib the plugboard search space (26! minus symmetry constraints) is intractable without significant pruning.

The cryptocellar writeup describes GPT-6 Astra contributing to a break by combining heuristic search over rotor configurations with iterative refinement of candidate plaintexts — effectively using the model as a scored hypothesis generator that can assess partial-decryption plausibility against German language models and known Naval/Wehrmacht message conventions. This is a hybrid approach: constraint propagation narrows the key space, and the LLM scores candidate outputs by linguistic coherence, replacing a pure hillclimbing optimizer with a semantics-aware scoring function.

The technical substance of the break hinges on whether the final plaintext is verified through corroborating historical records or remains a best-candidate plaintext with high model confidence. The writeup identifies the rotor settings and claims a coherent German plaintext, which is the practical standard of proof for these breaks — there is no ground truth key surviving in archives.

This is notable for the cryptanalysis methodology rather than for what it says about LLM capability per se. Using a language model as a scoring oracle for candidate plaintexts in a search over a structured combinatorial space is a clean and replicable technique applicable to other historical cipher problems.

Source: https://www.cryptocellar.org/bgac/the-mvueh-break.html

GPT-6 Astra has gained the ability to drive a car

DrivingBench appears to be a benchmark and evaluation harness for end-to-end autonomous driving tasks, with GPT-6 Astra being evaluated as a vision-language-action model capable of issuing low-level control commands — steering angle, throttle, brake — from camera inputs and natural language route instructions. The technical claim is that the model generalizes across scenarios without scenario-specific fine-tuning, handling urban intersections, highway merging, and adverse weather in simulation.

The architecture implied is a multimodal transformer that ingests tokenized image patches alongside a route description and outputs discrete or continuous action tokens — similar in spirit to RT-2 or DriveVLA approaches, where the pretrained VLM backbone provides semantic scene understanding and the action head is either fine-tuned or prompted into a driving policy. The key engineering question is whether this is prompt-based zero-shot transfer or whether there was supervised fine-tuning on driving demonstrations; the benchmark page does not make this fully explicit.

DrivingBench’s evaluation metrics include collision rate, route completion percentage, infraction counts (running red lights, lane violations), and a composite driving score adapted from CARLA’s leaderboard methodology. Claimed numbers place GPT-6 Astra above prior state-of-the-art on route completion in the most complex urban scenarios, but without the underlying model weights or a reproducible CARLA checkpoint, independent validation is not possible.

Limitations are the standard ones for sim-to-real driving benchmarks: CARLA’s renderer does not capture real sensor noise, edge-case distributions in simulation differ substantially from real traffic, and latency of a large VLM inference loop has not been reconciled with the real-time control requirements of a physical vehicle. The “ability to drive a car” framing overstates what is demonstrated — this is strong benchmark performance in a specific simulator.

Source: https://drivingbench.com/

Once Claude can measure something, it can make it faster

This post from the Claude.dev engineering blog describes a performance optimization loop applied to Claude’s own inference and tooling infrastructure. The central thesis is that latency improvements became tractable once instrumentation was in place — before measurement, engineers were optimizing by intuition; after, they could target specific bottlenecks with quantified impact.

The technical substance covers several areas. First, tracing was added at the granularity of individual tool calls, context window assembly, and network round-trips within the agentic scaffolding. This revealed that a significant fraction of wall-clock latency was in context serialization and prompt construction rather than model forward passes — a result that is not obvious a priori and that would not have been found without fine-grained profiling. Second, caching of repeated context prefixes was extended: system prompts and static tool definitions that are identical across turns were pinned in the KV cache, reducing effective prompt processing time for multi-turn sessions. Third, parallelism in tool execution was increased: where tool calls within a single model turn had no data dependency, they were issued concurrently rather than serially.

Quantitative claims include a roughly 2x reduction in median latency for multi-step coding tasks and a reduction in p95 latency that was disproportionately large relative to the median improvement — indicating that the tail was dominated by specific serialization pathologies that the profiling exposed.

The engineering lesson is not novel in principle — measure before optimizing is standard — but the specific instantiation in an LLM agent context is useful: the dominant costs in agentic loops are often scaffolding overhead, not model compute, and that has real architectural implications for anyone building similar systems.

Source: https://claude.dev/blog/how-we-made-claude-ai-faster/

Claude Code reads AGENTS.md only when telemetry is on [fixed]

This is a behavioral bug report with security-relevant implications. The researcher found that Claude Code, Anthropic’s CLI coding agent, would parse and act on instructions in an AGENTS.md file in the project root — a convention for providing agent-level behavioral configuration — but only when usage telemetry was enabled. With telemetry disabled, the file was silently ignored.

The technical cause, as reconstructed in the writeup, appears to be a code path dependency: the telemetry initialization routine was inadvertently gating a configuration-loading function. The AGENTS.md parser was called within a branch that was only reached when the telemetry client was initialized, meaning the conditional if telemetry_enabled: init_telemetry(); load_agent_config() structure (or equivalent) caused the config load to be skipped for privacy-conscious users who had opted out.

The security implication is a trust and predictability issue rather than a direct vulnerability: users who deliberately disabled telemetry — often for confidentiality reasons on sensitive codebases — were operating under the assumption that their agent configuration was active when it was not. Any security policies, scope restrictions, or workflow constraints encoded in AGENTS.md were being silently bypassed. This is the kind of subtle bug that is difficult to catch in testing because the telemetry-off path is not the default and behavioral divergence across configuration states is rarely part of QA checklists for agent tooling.

Anthropic acknowledged and fixed the issue. The broader lesson is that agent configuration files with security-relevant content need explicit test coverage across all runtime configuration states, and that initialization ordering bugs in complex CLI tooling can have non-obvious behavioral consequences.

Source: https://blog.szypowi.cz/p/claude-code-reads-agents.md-only-when-telemetry-is-on/

Gemini 3.8 text-to-speech

Google’s Gemini 3.8 TTS release describes a system built on the same multimodal foundation model rather than a separate dedicated TTS pipeline, which is the architecturally interesting design choice. Traditional industrial TTS stacks (Tacotron/VITS/StyleTTS lineage) use specialized acoustic models with explicit prosody representations; the claim here is that a general-purpose language model backbone, when fine-tuned or prompted appropriately, can produce competitive speech synthesis without these specialized components.

The system outputs audio tokens that are decoded by a vocoder into waveforms. The token representation is key: rather than predicting mel-spectrogram frames, the model predicts discrete audio codec tokens (likely from a SoundStream or EnCodec-style codec), which allows the autoregressive LM training objective to apply directly. This is the same approach used in AudioPaLM, SoundStorm, and VoxtLM — the novelty in 3.8 is the scale and integration with the general Gemini model family.

Claimed capabilities include multilingual synthesis across 24 languages, fine-grained prosody control via natural language instructions (“speak slowly and with a formal tone”), and multi-speaker dialogue synthesis in a single inference pass. The multi-speaker single-pass capability is technically nontrivial: it requires the model to track speaker identity tokens across turns and modulate voice characteristics accordingly without resetting the acoustic context.

Evaluation metrics cited include MOS scores competitive with human speech on clean studio recordings and strong preference scores on naturalness in noisy/spontaneous domains. Robustness to unusual proper nouns, code-switching, and numeric/symbol-heavy text — historically weak spots for TTS — is claimed to be improved but not exhaustively benchmarked in the public post.

Source: https://blog.google/innovation-and-ai/models-and-research/gemini-models/gemini-3-8-text-to-speech/

ArXiv receives multiyear commitments to support it as an independent nonprofit

ArXiv has secured multi-year funding commitments that will sustain its operation as an independent nonprofit under Cornell University stewardship. This is an infrastructure story, not a research story, but it matters for the entire ML research ecosystem: ArXiv hosts effectively all ML/AI preprints and operates as critical shared infrastructure with no paywalls, no submission fees, and moderation that is light relative to journals.

The financial model has historically been a membership program where universities and research institutions pay tiered annual fees, supplemented by a Simons Foundation grant that is now ending or transitioning. The new commitments appear to extend and diversify this base, providing runway for technical infrastructure upgrades — the ArXiv codebase and submission pipeline are aging, and there have been longstanding complaints about the submission UI, metadata handling, LaTeX compilation failures, and the difficulty of the moderation queue.

The technical challenges ArXiv faces are non-trivial: ingesting increasingly large submissions (papers with embedded datasets, supplementary videos, interactive figures), scaling search and recommendation to a corpus now exceeding 2.3 million papers, maintaining LaTeX-to-HTML compilation that correctly renders modern ML paper conventions, and managing the moderation workload as submission volume has grown roughly 20% year-over-year in CS/ML categories.

The independence framing is significant: there have been periodic concerns about whether a large commercial entity might acquire or subsume ArXiv’s function. Securing independent nonprofit funding directly addresses that governance risk. For researchers, the practical implication is continuity of free access and the open API that downstream tools — Semantic Scholar, Connected Papers, paper recommendation systems — depend on.

Source: https://blog.arxiv.org/2026/09/23/arxiv-receives-multiyear-investment/

Stripe’s Knowledge AI Platform

Stripe’s engineering blog describes an internal platform built to make institutional knowledge — documentation, runbooks, code conventions, incident postmortems, API specifications — queryable by employees and, selectively, by external developers via the Stripe developer portal. The system is a retrieval-augmented generation (RAG) platform, but the post goes into enough implementation detail to be worth examining.

The retrieval layer uses a hybrid approach: dense vector retrieval (embedding-based ANN search over a FAISS or similar index) combined with BM25 sparse retrieval, with a learned re-ranker merging the two result sets. The dual-path approach addresses the known failure mode of pure dense retrieval on keyword-heavy technical queries — a developer asking about a specific API parameter name or error code gets better recall from sparse retrieval than from semantic embeddings alone.

The indexing pipeline is non-trivial: Stripe’s internal documentation exists in Confluence, Notion, internal wikis, GitHub READMEs, and auto-generated API reference docs, all with different structure, freshness signals, and authority levels. The platform assigns source-level confidence weights and uses document freshness as a retrieval signal — stale runbooks are down-ranked, and the system surfaces freshness warnings when the top-retrieved document is older than a threshold.

For the generation step, Stripe uses citation grounding: every claim in the model’s response is linked to a source chunk, and the UI exposes these links to users. This addresses the hallucination problem not by eliminating it but by making the evidence chain auditable. The post notes that citation accuracy (whether the cited chunk actually supports the generated claim) required a separate verification model trained on Stripe’s own document domain.

Latency budgets and cost tradeoffs between retrieval index size and query latency are discussed but without specific numbers.

Source: https://stripe.dev/blog/meet-stripes-knowledge-ai-platform

Noteworthy New Repositories

okf-memory/okf-agent-memory

Git-native persistent memory layer for AI coding agents, implementing the Google Open Knowledge Format (OKF) v0.2 spec. The core design stores agent memory as structured files in a Git repository, giving you full version history and diff-ability with no external database dependency. The in-memory BM25 search index is built at load time and serves queries in under 300 microseconds, which is fast enough to run synchronously in an agent loop without adding latency overhead. An embedded MCP (Model Context Protocol) server exposes the memory to any MCP-compatible agent without requiring network round-trips to a separate process. The “progressive disclosure” mechanism surfaces only the most contextually relevant memory chunks to the model at each step, which the authors claim cuts token consumption by roughly 80% compared to naive full-context injection. The entire implementation is pure Go with zero external dependencies, making it trivially embeddable in other Go-based tooling or cross-compiled for different targets. The git-native storage also means memory can be shared, branched, or rolled back using standard git operations — useful for multi-agent setups or debugging agent behavior post-hoc. Worth evaluating if you want persistent agent memory without standing up a vector database or cloud service.

Source: https://github.com/okf-memory/okf-agent-memory

datawhalechina/zero-to-sglang

A structured course jointly produced by the SGLang team and Datawhale covering LLM inference from first principles through production-grade implementation. The curriculum is organized in three stages: (1) conceptual grounding in inference mechanics — KV caching, continuous batching, memory management, scheduling policies; (2) building a minimal SGLang-compatible runtime from scratch so learners internalize the architecture rather than just reading it; (3) reading the actual SGLang codebase and submitting a real upstream PR. This last step is unusually concrete — most educational projects stop before touching production code. The material is available in both English and Chinese, with the Chinese version likely the more complete given the Datawhale authorship. For researchers who use SGLang as a black box, this is a fast path to understanding where latency comes from and how to profile or modify the scheduler. For engineers, the build-from-scratch phase makes the RadixAttention and chunked-prefill optimizations legible before you encounter them in the real codebase. Stars-to-age ratio suggests rapid adoption, consistent with the SGLang project’s growth trajectory in the inference optimization space.

Source: https://github.com/datawhalechina/zero-to-sglang

Loopera-ai/loopera

An AI agent framework targeting fundamental factor research in quantitative finance — specifically the hypothesis generation and validation loop that quant researchers iterate through manually. The agent operates hypothesis-driven: it generates a factor hypothesis, designs a validation procedure, runs the empirical test, gates further development on evidence quality, and persists findings in a research memory store. The “evidence-gated validation” framing means the agent does not proceed to downstream steps (portfolio construction, paper writing, production deployment) unless the statistical validation clears configurable thresholds, reducing spurious factor proliferation. The research memory component accumulates findings across sessions, enabling the agent to avoid re-testing known dead ends and to compose more complex hypotheses from prior validated components. This is a narrower scope than general financial RL or end-to-end trading systems — it is explicitly focused on the factor discovery research workflow. Relevant for any team running systematic fundamental factor pipelines who want to automate the hypothesis-grind portion. The Chinese-first documentation reflects the primary target user base but the core technical design is readable from the code.

Source: https://github.com/Loopera-ai/loopera

muellerberndt/cadence

A machine learning library for building “consensus-style equilibrium world models” — the key idea being that model updates are governed by a consensus mechanism across multiple competing hypotheses rather than simple gradient descent toward a single loss. The library is designed for continual learning settings where the model must integrate new experience without catastrophic forgetting. The equilibrium framing draws an analogy to game-theoretic equilibria: the model state at any point represents a stable distribution across hypotheses consistent with observed data, and updates move toward a new equilibrium rather than a new point estimate. Practically, this is pitched at applications where the data distribution shifts continuously and you need the model to track that shift without full retraining. The library is early-stage with 272 stars, and the core algorithms are the interesting part to inspect — whether the consensus mechanism is implemented as an ensemble with weighted voting, as a variational objective, or as something more novel is worth examining directly in the source. If the equilibrium convergence guarantees hold empirically, this could be useful for long-running agent environments.

Source: https://github.com/muellerberndt/cadence

nokia-applied-research/AnyJev

Adapter layer that converts any LLM into a Jev-style decision model without fine-tuning. The Jev formalism requires decisions to be typed (the output schema is declared in advance) and to carry calibrated probabilities over alternatives — properties LLMs do not natively satisfy. AnyJev wraps arbitrary LLMs with a structured prompting and output parsing layer that enforces the typed decision interface and elicits explicit probability estimates via constrained generation or logit extraction. The “no training” claim means this works at inference time against any model with a compatible API. This is useful for decision-theoretic pipelines that need to compose LLM judgments with downstream probabilistic reasoning (e.g., Bayesian networks, influence diagrams) — the typed probability output is machine-readable and can be piped directly into such systems. Nokia Applied Research origin suggests this came out of a telecoms or network management use case where formal decision models have long been standard. The “(continue updating)” note in the description implies the codebase is actively evolving. Worth watching if you’re building hybrid neuro-symbolic decision systems and need a clean LLM interface that respects probability semantics.

Source: https://github.com/nokia-applied-research/AnyJev

mikehasa/golive-skill

Open-source deployment automation targeting agent-built applications — the scenario where a coding agent has produced a working app and needs to provision hosting, a database, a domain, email delivery, and payment processing on the user’s own accounts, not the tool vendor’s. The implementation is a two-part system: an Agent Skill (callable from within an agent framework) and a zero-dependency Node CLI. The workflow follows a strict pipeline: detect existing environment and what services are missing, generate a deployment plan, present it for human approval, apply changes, and verify the result. The human-in-the-loop approval step before applying is important — it means this is not fully autonomous infrastructure modification, which is the right default for production. Zero external dependencies and no telemetry/backend mean the CLI can be audited fully and run air-gapped. The “your own accounts” design avoids vendor lock-in to any particular GoLive service. The primary gap in existing agent deployment tooling is exactly this last-mile step from “agent wrote the code” to “code is running on real infrastructure owned by the user,” and this addresses that gap directly.

Source: https://github.com/mikehasa/golive-skill

qiz029/dscode

A harness around DeepSeek coding models that adds several capabilities missing from the base model API: a persistent shell session that survives across agent turns (so commands like cd, environment variable exports, and running processes stay live), “Ultra” subagents that the main agent can spawn for parallel subtasks, auto-approval of tool calls to remove human-in-the-loop latency during automated runs, a Chrome MCP integration for browser-based tasks, and per-session telemetry logging. The persistent shell is the most practically significant feature — most agent frameworks re-initialize the shell per turn, which breaks stateful workflows. The subagent architecture enables decomposing large coding tasks into parallel workstreams, each with their own shell context. Session telemetry provides a trace of exactly what commands were run and what the agent decided, which is valuable for debugging agent failures post-hoc. The Chrome MCP integration extends the agent to web scraping, form filling, and browser automation tasks beyond pure code generation. Targeted at users who want to run DeepSeek models for extended, multi-step software engineering tasks without babysitting each approval prompt.

Source: https://github.com/qiz029/dscode

xiaYuTian11/maskit

A local privacy-sanitization proxy (“data masking gateway”) designed specifically for LLM coding assistants and API-compatible tools such as Cursor, Claude Code, OpenAI Codex, and others that accept a configurable base URL. The proxy sits between the client tool and the upstream LLM API, intercepts outgoing requests, detects and redacts sensitive entities (PII, credentials, internal identifiers) before the request leaves the machine, and then re-injects the original values back into the streaming response on the way in — so the user sees coherent output referencing their actual variable names and data, while the upstream model never receives the raw sensitive content. The stream-aware response restoration is the technically interesting part: it must maintain a substitution map and correctly re-apply it to token-streamed responses without breaking the stream framing. Running locally means no sensitive data transits a third-party masking service. This addresses a real enterprise friction point — teams that want to use hosted LLM coding tools on codebases containing credentials, customer data references, or proprietary identifiers but cannot send those to external APIs under their data governance policies.

Source: https://github.com/xiaYuTian11/maskit