Daily AI Digest — 2026-09-21

Published

September 21, 2026

English · 日本語

arXiv Highlights

Grounded Skill Synthesis from Code at Scale for Agentic Intelligence

Reusable skills — procedural knowledge that transfers across tasks — are typically acquired either from agent trajectories (which require environment interaction and are narrow) or from documents (which lack executable grounding). This paper argues source code is a third, complementary substrate: it encodes procedures with executable evidence but does not presuppose any agent history. The authors build Code2Skill, an automated pipeline that mines GitHub repositories into a skill bank (CodeSkillBank) of ~1M records, and evaluate whether retrieval-augmenting agents with these records improves downstream performance.

The construction pipeline

A skill record must, per the authors’ formulation, specify (i) applicability conditions, (ii) target behavior, (iii) preconditions and invariants, (iv) failure handling, and (v) source spans that back these claims. This is operationalized through three quality axes: grounded (recoverable from implementation spans), transferable (project-specific identifiers abstracted away), and maintainable (provenance and construction status retained so records can be re-verified as code evolves).

Figure 1: The pipeline of Code2Skill. Source units are ranked, converted into atomic, composite, or recurring-pattern skill records, checked through source-body-blind reconstruction and source-aware comparison, and retained with evidence for retrieval.

The pipeline (Figure 1) proceeds as: (1) rank candidate source units — functions, methods, CLI entry points, or file-level components — along with their repository context; (2) convert each into an atomic, composite, or recurring-pattern skill record; (3) verify through two complementary checks:

  • Source-body-blind reconstruction: given only the abstracted skill description (no source body), can an LLM reconstruct code that matches the reference behavior? This tests whether the record actually captures the procedure.
  • Source-aware comparison: an equivalence judge compares reconstruction against the source with body visible, with adjudication on disagreement, and a value filter for retention.

Records that pass are kept with workflow, boundary, provenance, and source-evidence metadata. This dual verification is the paper’s central mechanism: the reconstruction check enforces that the abstract description alone is sufficient to reproduce the behavior — i.e., it isn’t a summary but an executable specification.

CodeSkillBank at scale

Applied to 19,769 popular, actively-maintained GitHub repositories, the pipeline yields 1,006,822 accepted skill records. The source pool is quality-filtered: median 3,133 stars and 82 merged PRs, with 78.3% of repos above 1,000 stars, 46.9% with ≥100 merged PRs, and 66.0% pushed within the prior year. The intent is that reusable conventions, boundary checks, and repair patterns are more likely to have accumulated in such codebases.

Human annotation of four sampled outcome pools (value-filter removals, direct acceptances, adjudicated acceptances, final rejections) gives the following: 92% of skill descriptions in the final bank are judged accurate, 80% of records are judged worth retaining, and among directly accepted records 84% support correct reconstruction. The rejection pool sits at 32% description accuracy and 28% retention value with no correct reconstructions — a large gap that validates the filter’s discriminative power.

Figure 13: Cross-label structure in CodeSkillBank. Software behavior types align with their characteristic problem archetypes, while the problem-reuse panel shows the many-to-many associations between target-side objectives and solution-side procedural structures.

Semantic profiling (Figure 13) shows records concentrate on data transformation, state updates, input understanding, validation, parameterization, state-machine control, and API usage. The many-to-many mapping between problem archetypes and procedural structures is what enables semantically targeted retrieval — a task can be matched against multiple axes rather than a single label.

Empirical evaluation

The main table covers 72 protocol-matched evaluations across nine model settings (including DS4-Flash, Qwen3.5/3.6, Gemini 2.5 Pro, GPT 5.2) and eight benchmarks spanning three regimes: programming and SWE (BigCodeBench, SWE-bench Verified), terminal and OS control (TerminalBench, LongCLI-Bench, AgentBench-OS), and reasoning (AIME 2026, HMMT 2025, GPQA). Models augmented with retrieved CodeSkillBank skills improve on average by 11.7% over matched baselines. (The abstract sentence is truncated at “outperform them in 5” — presumably 5 of the model settings dominate across all benchmarks or similar.)

The RQ decomposition is worth noting for anyone re-implementing: RQ3 asks where in the agent loop skills should enter (planning vs. execution stages), RQ4 tests whether compact skill representations preserve utility, RQ5 examines whether the injection point matters under RL fine-tuning, and RQ6 uses a 400-task LiveCodeBench subset to compare skills mined from human vs. AI-generated code — a provenance question relevant to whether such a bank can grow with synthetic implementations.

Limitations and open questions

The reconstruction-based verification measures whether the description regenerates behavior in a model’s decoding distribution, which conflates faithful abstraction with LLM prior alignment: a skill an LLM would produce anyway may pass reconstruction trivially. Reported rejection-pool statistics show the filter discriminates, but the ceiling on record novelty relative to base-model priors is not quantified. Second, the retrieval interface, embedding, and top-k are the actual channel through which improvements arrive; the paper’s ablations on retrieval quality determine how much of the +11.7% is attributable to the bank’s content vs. its indexing. Third, maintainability — regenerating records as code evolves — is a claim; longitudinal drift is not measured. Finally, the temporal setup (models cited as DS4-Flash, Qwen3.6, GPT 5.2, AIME 2026) suggests some benchmarks and models are post-training data of others, and contamination pathways through GitHub source are non-trivial to rule out for code-heavy benchmarks like BigCodeBench and SWE-bench.

Why this matters

Code as a substrate for skill mining sidesteps the environment-specificity of trajectory methods and the ungrounded nature of document methods, and the reconstruction-based verification is a concrete quality gate that could be reused elsewhere (e.g., for tool-use libraries or workflow templates). If the ~11.7% average improvement holds under strict retrieval and contamination controls, mining executable procedures at web scale becomes a viable third pillar for agent skill acquisition alongside RL trajectories and documentation.

Source: https://arxiv.org/abs/2609.05571

CodeMidas: Scaling Agentic Coding RL Environments from Code Itself

Problem

RL for coding agents is bottlenecked by environment supply: you need tasks with reliable, executable verifiers, and enough diversity to avoid overfitting to a narrow slice of software engineering. The dominant recipe mines development artifacts — GitHub issues, PRs, commits — and reconstructs verifiers from them (SWE-bench-style). This restricts tasks to what maintainers happened to document and to bug-fix-shaped work, and inherits noisy tests written for human review rather than automated grading. CodeMidas asks whether the source code alone, without issues or commits, is a sufficient substrate to generate diverse, verifier-backed RL tasks at scale.

Method

CodeMidas is an agentic pipeline where each stage — task selection, code adaptation, test synthesis, environment preparation, filtering — is delegated to an LLM agent operating on the codebase. A task consists of a natural-language statement, a containerized development environment given to the solver, and a hidden executable verifier used only at grading time to produce a binary reward r \in \{0,1\} for GRPO.

The pipeline has four mechanical stages:

  1. Task design and codebase adaptation (§3.1). An agent inspects build metadata and code structure to find implemented functionality with public entry points and observable outcomes — CLIs assessed via process output, library functions via return values, and stateful APIs via state changes across calls. It traces the entry point through shared dependencies to define scope, then removes the core implementation. The remaining code is edited so the repository still builds and other components still make sense; the removed reference implementation is retained separately as an oracle solution.

  2. Execution-grounded test construction (§3.2). Rather than hallucinating tests from the statement, the agent runs the original (still-implemented) code to produce oracle inputs, outputs, and state traces, and writes tests grounded in those executions.

  3. Environment preparation with execution consistency check (§3.3). Containers are built; the reference implementation is executed against the synthesized verifier to confirm it passes. Any task whose oracle solution fails its own verifier is dropped.

  4. Rollout-based filtering (§3.4). Candidate tasks are attempted by a solver policy multiple times; tasks that are unsolvable or trivially solved are removed, leaving those with informative pass-rate variance for GRPO.

The final dataset contains 5,545 training tasks from 3,185 codebases across 23 languages and 15 domains. During RL, MiMo-V2.5 is trained with GRPO, batch size 32, 32 rollouts per task, and the binary execution reward.

Results

Across five external benchmarks, RL on CodeMidas produces monotone improvements over the initial MiMo-V2.5 policy: DeepSWE v1.1 +11.7 percentage points, ProgramBench (Almost Solved, ≥95% of tests passing) +17 points, plus gains on SWE-bench Pro, RepoZero C2Rust, and Terminal-Bench v2.1. The benchmark span — issue repair, whole-program synthesis, cross-language translation, and terminal work — is important because the training distribution never uses issue text or commit diffs, yet transfer to SWE-bench Pro and DeepSWE (both issue-repair) still holds.

The scale/quality analysis in §5.1 is the more interesting empirical result. Training on random 1k, 3k, and full 5k subsets of the high-quality pool yields progressively better DeepSWE scores of 17.57 → 19.05 → 21.70 and CodeMidas Val scores of 41.30 → 43.22 → 44.73. On CodeMidas Val, the 1k pool peaks at step 30 (41.30) and the 3k pool at step 65 (43.22), while the full pool leads at every checkpoint from step 40 to 70 (44.73), indicating scale extends the useful training horizon rather than only lifting an early plateau.

Filtering quality dominates raw quantity: an unfiltered 8k sample — same tasks, no environment cleaning, no execution consistency check, no post-rollout filtering — is beaten by the 5k filtered pool by 0.59 / 4.59 / 4.49 points on SWE-bench Pro, DeepSWE, and CodeMidas Val respectively. Even the 3k filtered subset beats vanilla 8k on all three. The gap is largest on DeepSWE, where verifier reliability plausibly matters most because reward noise propagates directly into GRPO’s advantage estimates A_i = (r_i - \bar r)/\sigma_r.

Limitations and open questions

The paper does not report performance against strong closed models or against agents trained on issue-derived environments (e.g., SWE-Gym) at matched compute, so the comparative value of “source-only” versus “artifact-derived” tasks is undetermined. Binary execution rewards say nothing about code quality, efficiency, or idiomaticity — only that outputs match. The pipeline’s reliance on agents at every stage means construction cost per task is nontrivial and quality depends on the constructor model; failure modes such as verifiers that overfit the reference implementation’s incidental behavior are not quantified. Finally, 23 languages / 15 domains is a coarse breakdown; per-language transfer curves would clarify whether gains are broad or dominated by Python/JS-heavy tails.

Why this matters

CodeMidas shows that RL environments for coding agents can be produced from source alone, decoupling training-data supply from the accidental byproducts of open-source development, and that aggressive execution-based filtering is worth more than 60% more raw tasks. For groups building coding-agent RL pipelines, the takeaway is concrete: budget compute for verifier validation and rollout-based difficulty filtering before scaling task count.

Source: https://arxiv.org/abs/2609.22068

RecreationWorld: Scalable and Verifiable Environments for Hybrid Computer-Use Agents

Problem

Computer-use agent (CUA) research has bifurcated: one line targets GUI operation (clicks, keystrokes, accessibility trees), another targets code and shell interaction. Real digital workflows interleave the two — you inspect an application, write code to reproduce or extend it, launch the artifact, and visually confirm it behaves as intended. Benchmarks that either mandate a fixed workflow or evaluate on source similarity fail to measure this interleaving, and they cannot supply a scalable reward signal for training.

RecreationWorld reframes hybrid CUA evaluation and training around application recreation: the agent is given a running reference application A^\star and a prepared environment containing both GUI and coding tools, and must produce a source submission S whose built artifact \widehat{A} matches A^\star’s observable behavior. Only behavior is scored; architecture and source structure are free. This creates an execution-grounded, implementation-agnostic reward and, crucially, a limitless supply of tasks from open-source GitHub applications pinned to specific commits.

Framework and task contract

The task tuple is (\text{prompt}, A^\star, \text{env}). The agent freely alternates among reference operation, code editing, build/launch, and visual inspection of \widehat{A}. The delivery contract is platform-native (Table 1): source with build/launch entry points on Ubuntu/macOS/Windows, a Gradle project producing an APK on Android, and a pinned React stack producing a self-contained index.html on the web. Behavioral tests use each platform’s programmatic interface — AT-SPI (Ubuntu), AXUIElement (macOS), UI Automation (Windows), UiAutomator (Android), and DOM/ARIA (web) — supplemented on every platform by visual assertions.

An important asymmetry: desktop and Android are source-blind (the reference binary/APK is executable but its source is withheld); web cannot be source-blind because the client is what the server serves, so the protected material there is the ground-truth test suite. Android further restricts to applications lacking Internet permission to avoid remote-service nondeterminism. Web tasks preserve original path-based URLs (no hash-routing substitution) and are scored on both desktop and mobile viewports.

Data scaling and training

RecreationWorld mines high-quality open-source GUI applications across the same five platform families for training tasks, deduplicated against evaluation sets. Trajectories are generated with Qwen3.8-Max acting inside the same agentic harness (GUI + coding tools, concurrent isolated workers). Rejection sampling uses the task-specific behavioral verifiers as the filter; from the surviving pool, 7{,}000 trajectories per platform are selected, yielding a balanced 35{,}000-trajectory SFT mixture (balanced by count, not tokens).

Two initializations are fine-tuned on this same mixture: Qwen3.7-Plus (fully post-trained) and Qwen-Flash-CPT (an in-house Qwen-Flash after continual pretraining). The transfer claim is evaluated on five out-of-distribution coding and hybrid CUA benchmarks. Both training arms show consistent improvement across all five OOD suites, and averaged behavioral analyses show more frequent verification of rendered outputs — i.e., the recreation objective transfers not just as skill but as a habit of visually checking artifacts before submission.

Main benchmark results

Ten frontier models are evaluated on RecreationBench through a direct-MCP interface (the persistent programmable runtime is a separate ablation and excluded from headline scores). Aggregate ranking:

  • GPT-6 Astra: 58.06%
  • Claude Opus 5: 44.16%
  • GPT-5.6 Sol: 42.06%

GPT-6 Astra is the only model achieving full-suite passes on multiple platforms. Its 90% and 100% programmatic-coverage rates are 17.6% and 2.8% of tasks respectively, versus at most 5.5% and 0.8% for any other model — the gap widens sharply at the top of the coverage distribution, indicating that stricter behavioral fidelity is where frontier separation lives.

Trajectory analysis

The analysis correlates exploration behavior with score, though causality is not isolated (harness, runtime, and model change together). Claude Opus 5, Qwen3.8-Max-0902, and GPT-6 Astra receive a median of 53 images per trajectory, with 96.8% of trajectories including at least one image; GLM-5.3, restricted to structured GUI representations, cannot be compared on pixel-level observation.

Reference exploration breadth tracks final score. GPT-6 Astra covers 58.1% of platform-supported reference-side GUI interaction types, ahead of Qwen3.8-Max-0902 at 41.8%. On window-identity coverage, GPT-6 Astra observes 5.23 distinct reference windows per trajectory versus 2.21 for Qwen3.8-Max-0902. These two metrics — interaction diversity and observed interface-state diversity — agree in ordering, which is not tautological given they proxy different dimensions of exploration. GPT-6 Astra’s 13.9 pp lead over Claude Opus 5 co-occurs with this broader coverage.

Limitations and open questions

  • Source-blind protection is inherently weaker on web; the benchmark relies on withholding tests rather than binaries, and the “no-reload-of-reference” rule is a policing constraint rather than an information-theoretic guarantee.
  • The observable-behavior specification is bounded by each platform’s accessibility API surface (e.g., custom-rendered toolkits or macOS menu-bar popovers stretch what AT-SPI/AXUIElement can assert). Visual assertions patch some gaps but introduce their own brittleness.
  • The training pipeline uses a single generator (Qwen3.8-Max) and SFT only; no on-policy RL is reported despite the availability of executable, verifier-based rewards — an obvious next step.
  • Trajectory-analysis correlations mix model, harness, and runtime and therefore cannot isolate which behaviors (exploration breadth, verification frequency) causally drive score.
  • Balancing by trajectory count leaves token contributions uneven across platforms; whether reweighting by tokens or difficulty would change transfer is not tested.

Why this matters

RecreationWorld operationalizes hybrid computer use with an execution-grounded, implementation-agnostic reward that scales with the open-source ecosystem, and it shows non-trivial transfer of behavioral verification habits from recreation training to unrelated coding and CUA benchmarks. That combination — a reward signal cheap enough to farm at scale and a task family that forces GUI + code interleaving — is the missing substrate for training CUAs with RL rather than imitation.

Source: https://arxiv.org/abs/2609.22000

MintAct: A Unified Visual Agent for Digital Environments

Problem

Building a single VLM policy that handles UI grounding, multi-step navigation across mobile/desktop/web, and visual tool use has historically required per-domain specialists: grounding models trained on click-target datasets, navigation agents fine-tuned on AndroidControl or OSWorld trajectories, and tool-augmented VLMs trained on function-calling corpora. Each domain has different observation modalities (screenshots vs. accessibility trees vs. rendered tool outputs), different action spaces (pixel coordinates vs. API calls), and different reward signals. MintAct asks whether a unified policy at modest scales (2B/4B/8B) can match these specialists without capacity loss, and what environment and RL infrastructure that unification requires.

Figure 1: Overview of the diverse visual agentic tasks supported by MintAct.

Method

The policy \pi_\theta is defined over the standard agentic decomposition a_t \sim \pi_\theta(a_t \mid c, g, o_{\le t}, a_{<t}), where c is a system prompt, g is the instruction, o_{\le t} is the observation history (screenshots plus, where available, structured accessibility state), and a_{<t} contains prior actions and their chain-of-thought reasoning traces. Each a_t is emitted as a reasoning-then-action pair; the action string is parsed into either (a) a UI primitive (click at (x, y), type, scroll, key event) or (b) a tool call with structured arguments. Collapsing UI ops and tool calls into a single serialized action vocabulary is what allows one head to serve all three capability families.

The infrastructure side is where most of the engineering sits. MintAct runs hundreds of concurrent environment instances across heterogeneous backends (Android emulators, desktop VMs for OSWorld, browser sandboxes for web navigation, and tool-execution sandboxes for MM-ToolSandBox). These pools serve both offline trajectory collection and online RL.

Figure 3: Asynchronous RL infrastructure and trajectory group generation pipeline.

The RL loop is asynchronous: rollout workers pull the latest policy weights but continue generating with stale weights while trainers step, producing off-policy trajectories. The framework retains explicit control over the cross-domain sampling distribution — i.e., it does not let the empirically faster domains (grounding, short mobile tasks) dominate the gradient over the slow OSWorld/Weblica tasks that dominate wall-clock. Trajectory groups are formed per prompt to support group-relative advantages (a GRPO-style estimator is implied by the group generation pipeline in Figure 3, though the paper’s excerpt does not spell out the exact objective). Stability under noisy environment feedback — flaky emulators, non-deterministic web pages, tool timeouts — is handled at the infrastructure level rather than by discarding trajectories.

Max episode length is 30 steps for online navigation and 100 for visual tool use, reflecting the longer decomposition typical of multi-tool workflows.

Results

MintAct is evaluated across four capability clusters:

  • Grounding: MMBench-GUI, ScreenSpot-v2, UI-Vision, OSWorld-G.
  • Mobile navigation: AndroidControl, AndroidWorld.
  • Desktop navigation: OSWorld-Verified.
  • Web navigation: Weblica, Online-Mind2Web.
  • Visual tool use: MM-ToolSandBox.

The headline number is 48.9 on OSWorld-Verified, reported as state-of-the-art at comparable model scale. OSWorld-Verified is the harder, human-verified subset of OSWorld and is generally regarded as the stiffest test of long-horizon desktop agency; passing it requires reliable grounding, application-specific procedural knowledge, and recovery from execution errors. The abstract further claims MintAct “matches the performance of per-domain specialists across all of these capabilities,” meaning the unification does not degrade grounding accuracy relative to grounding-only baselines nor navigation success rates relative to navigation-only baselines — the usual failure mode when jointly training heterogeneous action spaces.

The scaling from 2B to 8B is offered as evidence that the recipe is not an artifact of a single capacity point, though the excerpt does not break down per-benchmark scaling curves.

Limitations and open questions

The excerpt does not disclose:

  1. The exact RL objective (PPO vs. GRPO vs. a bespoke group-relative estimator), the importance-sampling correction used to tolerate off-policy drift, or the KL regularization schedule.
  2. How the cross-domain sampling distribution is actually chosen — uniform per-domain, loss-weighted, or curriculum-based — and how sensitive final performance is to this choice.
  3. Reward shaping for OSWorld and Weblica, which are notoriously sparse. Whether MintAct uses process rewards, verifier models, or purely terminal success signals is not stated in the provided material.
  4. The composition of pretraining/SFT data before RL. Given that specialists are matched, the SFT mix likely carries most of the capability; the RL delta on top is not isolated in the excerpt.
  5. Generalization to unseen applications and websites — the benchmarks used (OSWorld, AndroidWorld, Online-Mind2Web) all have known task distributions that could leak through training-set curation.

An additional structural question: unifying UI actions and tool calls into one vocabulary is convenient but may cap performance on tool-heavy tasks where structured decoding (constrained JSON, typed arguments) would be strictly better than free-form generation.

Why this matters

Per-domain specialization has been the pragmatic choice for GUI/agent VLMs because joint training tends to trade off grounding precision against long-horizon planning. MintAct’s claim — that with the right async RL infrastructure and cross-domain distribution control, a single 8B model matches specialists and reaches 48.9 on OSWorld-Verified — suggests the specialization was a training-systems limitation, not a capacity one. If reproducible, this collapses a fragmented benchmark landscape into a single policy target.

Source: https://arxiv.org/abs/2609.22083

EvoOntology: A Self-Evolving Ontology Layer for Data Agents

Problem

Data agents that execute natural-language instructions against heterogeneous data (CSVs, files, relational DBs) face an agent-data gap: the underlying semantics — what a column really means, how tables link, which values are canonical — live outside the agent, while generic tools only surface surface-level metadata like column names and file paths. Two extant approaches fail to scale: (i) letting the agent explore raw sources burns context and produces brittle interpretations, and (ii) injecting a hand-authored semantic layer into the prompt does not adapt to workload or agent behavior and grows unboundedly with the data. EvoOntology asks whether an ontology can be built and continuously refined by agents themselves, exposed through a runtime API rather than dumped into the prompt.

Figure 1

Method

EvoOntology maintains a versioned ontology state \mathcal{L}_t = (\mathcal{S}_t, \Gamma_t, \mathcal{R}_t) at evolution round t, decomposed into three layers.

Content Layer \mathcal{S}_t. A typed semantic graph with four node families — Terms (domain concepts), Mappings (grounding to fields and join paths), Constraints (validity rules), Evidence (provenance) — and two edge families: Semantic Relations (association, hierarchy, composition, equivalence, derivation) between Terms, and Structural References linking Terms to Mappings and attaching Constraints/Evidence.

Schema Layer \Gamma_t. Declares the fields of the four node families, permitted relation types, and reference patterns. Because the schema is first-class, the ontology’s representational capacity can be extended without editing instantiated content.

Tool Layer \mathcal{R}_t. Exposes the ontology as an MCP server with two calls: - f_{\mathrm{browse}}(q, k, n) returns the top-n semantic matches of kind k for query q; - f_{\mathrm{resolve}}(\mathcal{I}, c) hydrates a set of node identifiers \mathcal{I} with linked objects at context depth c.

Only a compact session manifest — source overview and usage guidance — enters the prompt; detailed records are pulled on demand. This is the core reason the design scales to large ontologies: prompt cost is decoupled from ontology size.

Figure 2: Overview of EvoOntology’s typed content graph, schema, and MCP tool interface.

Two agents operate over \mathcal{L}_t. A builder agent produces the evidence-grounded initial state \mathcal{L}_0 from the training workload and raw sources. An evolution agent runs a self-evolution loop over historical trajectories: it attributes failures to missing/incorrect Terms, Mappings, Constraints, or Evidence, proposes typed edits (add/refine specific node families), and each candidate patch is admitted only after a backbone-conditional paired evaluation — the same evaluator backbone judges runs with vs. without the patch on the same tasks, so noise across backbones does not leak into acceptance.

The appendix case study illustrates a typical accepted patch: an initial ontology carries generic Card and Legality concepts but no notion of legality status; the patch adds a Legality Status Code Term with its Mapping, Evidence, and a Constraint tying the status value to the requested game format.

Figure 8: An accepted patch introduces a Legality Status Code Term, Mapping, Evidence, and Constraint (red dashed boxes).

Results

Evaluation covers three data-agent benchmarks under official protocols:

  • DDR-Bench (10-K scenario): Message-Wise and Trajectory-Wise accuracy on open-ended data research.
  • InsightBench: Insight and Summary scores for business analytics over CSVs.
  • BIRD (Oracle Knowledge setting): Execution Accuracy (EX) and Valid Efficiency Score (VES) for text-to-SQL.

Four backbones are tested: GPT-5.5, GPT-5.6-sol, Claude-Sonnet-5, Claude-Opus-4.8.

The most informative analysis tracks the deployed agent’s score across accepted evolution rounds. All four backbones improve monotonically from the initial builder-produced state through subsequent accepted patches on DDR-Bench Trajectory-Wise accuracy. GPT-5.6-sol reaches 93.5 Traj-Wise after five accepted rounds; Claude-Opus-4.8 reaches 92.3 after four. Curves flatten in the final two rounds, indicating convergence as recurrent cross-filing failure signatures are absorbed into the ontology. This is important: it shows the reported gains are not artifacts of a single lucky patch but the outcome of the four-step attribution → edit → paired gate → merge loop. Consistent monotone improvement is also observed on InsightBench (Insight) and BIRD (EX).

Limitations and open questions

  • The paired evaluation gate ties acceptance to a specific backbone; the paper does not fully characterize cross-backbone transfer of accepted patches, though improvements appear on all four backbones.
  • Convergence in “the last two rounds” suggests diminishing returns; the framework does not yet address how to detect distribution shift in the workload that would require re-opening evolution.
  • The MCP tool interface exposes semantics through f_{\mathrm{browse}} and f_{\mathrm{resolve}} only; complex multi-hop constraint reasoning may still require the agent to reconstruct chains in-context.
  • Cost accounting for the evolution loop (patch proposals, paired evaluations) versus the amortized inference savings is not the focus here and would clarify practical deployment.
  • Robustness to adversarial or noisy evidence attached during autonomous construction is not stress-tested.

Why this matters

EvoOntology reframes the semantic layer for data agents as a runtime-queryable, agent-maintained artifact rather than a prompt payload or a human-curated schema, and demonstrates that attribution-guided typed edits under a paired-evaluation gate produce monotone, converging quality gains across heterogeneous benchmarks and backbones. This is a concrete template for closing the agent-data gap without unbounded prompt growth or manual ontology engineering.

Source: https://arxiv.org/abs/2609.15779

IntBMoE: Integrating Block-Level Conditioning into Expert Composition for Full-Participation Mixture-of-Experts

The three-way tradeoff

The paper starts from a clean taxonomy that is worth internalizing before the method. For a single token, an MoE architecture involves three distinct quantities:

  • Participation P: number of experts whose knowledge contributes to the output.
  • Execution E: number of experts actually computed (FLOPs cost).
  • Materialization M: number of expert-sized parameter sets that must be built and stored (memory cost).

Existing MoE designs couple these. Sparse top-k routing (Switch, DeepSeek-V3 MoE, ReMoE, V-MoE) sets P=E=k and M=N, so participation collapses with sparsity. Dense output mixing (Soft MoE, \muMoE) restores P=N but forces E=N. Parameter-merging methods (SMEAR, Lory) synthesize a single composed expert per token so E=1, but each distinct routing decision produces a distinct materialized expert, so M scales with the number of tokens or contexts. IntBMoE’s claim is to decouple all three simultaneously.

Method

IntBMoE replaces the FFN in each Transformer layer with a block-conditioned MoE module. The design factors routing into two levels: block-level composition (which is token-independent) and token-level sparse execution.

Block-level composition. Each IntBMoE module maintains, per internal layer \ell \in \{1,\dots,L\}, a shared pool of N expert bases \{E^{(\ell)}_n\}_{n=1}^N. It also maintains a small learned codebook of K block embeddings \{c_k\}_{k=1}^K with K \ll N typically. A lightweight hypernetwork h_\phi maps each codebook entry to a set of composition coefficients

\alpha^{(\ell)}_k = \mathrm{softmax}(h_\phi(c_k, \ell)) \in \Delta^{N-1},

which merges the pool into a composed layer:

\tilde{E}^{(\ell)}_k = \sum_{n=1}^N \alpha^{(\ell)}_{k,n} E^{(\ell)}_n.

Stacking these across \ell yields K token-independent L-layer composed blocks. Crucially, because \alpha^{(\ell)}_k has full support over the pool, participation P = N for every composed block — each block draws on all bases.

Token-level sparse execution. For a token x, a router produces g(x) \in \mathbb{R}^K and selects the Top-k blocks; each selected block k applies block-conditioned feature filtering, then processes x sequentially through its L composed experts. The module output is

y = \mathrm{SwiGLU}_{\text{shared}}(x) + \sum_{k \in \mathrm{Top}_k(g(x))} p_k(x) \cdot B_k(x),

with p_k the normalized routing weights. Execution is therefore E = kL composed-expert layers per token, independent of N.

Materialization. Because composition is token-independent, the K composed blocks can be materialized once per forward pass (or once per batch) rather than once per token. Storage is dominated by the N pool bases plus the K codebook entries; materialization scales as M = N + K, not with the number of routing decisions. This is the decoupling: P = N, E = kL, M \sim N + K, chosen independently.

Empirical results

ImageNet-1K. All models use an 8-layer DeiT-Tiny backbone trained from scratch; means over three seeds. IntBMoE is compared to the dense backbone and to sparse-routing (Switch, DeepSeek-V3 MoE, ReMoE, V-MoE, Expert Choice, DynMoE, MASS) and dense-participation (Soft MoE, SMEAR, Lory, \muMoE) baselines. The abstract and setup indicate IntBMoE beats both families on Top-1/Top-5, but concrete numerical deltas are not reproduced in the sections quoted here — the paper reports them in the results tables (see Section 5.2 in the full text).

MiniPile language modeling. An 18-layer Llama-style causal Transformer with IntBMoE FFN sublayers is evaluated by token-level cross-entropy and perplexity.

IntTravel sequential recommendation. 162.8M users, 7.3M POIs, 4.13B interactions; the “Where” task predicts the next-journey destination POI, evaluated by HR@1, HR@5, and NDCG@5. This tests transfer of the same architecture to a very different token distribution (POI IDs, long user histories) and confirms the design is not vision-specific.

Limitations and open questions

  • The block-composition coefficients \alpha^{(\ell)}_k are token-independent. That is what buys constant materialization, but it also means adaptivity per token comes only from Top-k block selection and block-conditioned filtering — not from re-composition. This is a weaker form of input-conditioning than SMEAR/Lory, and it is unclear how it scales when N grows into the hundreds.
  • K and k jointly control expressivity vs. compute; the sections quoted do not report sensitivity curves for these hyperparameters.
  • Sequential execution through L composed layers per selected block increases the critical path relative to standard MoE FFNs, which is a wall-clock concern even when FLOPs are matched.
  • The scaling experiments (8-layer DeiT-Tiny, 18-layer Llama-style) are modest; behavior at frontier scale, where routing collapse and load-balancing pathologies dominate, is untested.
  • The always-active shared SwiGLU expert is a nontrivial fraction of compute; ablations distinguishing its contribution from the composed-block contribution would clarify how much the block mechanism itself is doing.

Why this matters

The participation/execution/materialization decomposition is a genuinely useful lens on MoE design, and IntBMoE demonstrates that the three axes can be set independently rather than being coupled through a single sparsity choice. If the empirical gains hold at scale, block-level token-independent composition combined with sparse block execution offers a concrete path to dense-participation quality at sparse-routing cost, without the memory blowup of per-token parameter merging.

Source: https://arxiv.org/abs/2609.21346

OmniVBench: A Benchmark and Large-Scale Dataset for Omni Reference-to-Video Generation

Problem

Reference-to-video (R2V) generation has drifted from single-image or single-video conditioning toward “omni” conditioning: heterogeneous reference bundles (characters, scenes, motion clips, style exemplars, storyboards) combined with free-form text. Existing benchmarks (e.g., VBench-family, T2V-CompBench) evaluate holistic reference consistency with a small set of reference types and typically score outputs against a single ground-truth similarity signal. This conflates three distinct failure modes that omni-R2V systems actually exhibit:

  1. Preservation: is the referenced factor (identity, style, motion trajectory) retained at all?
  2. Disentanglement and binding: when multiple references are supplied, does each factor get routed to the correct target and not leak into others (e.g., style bleeding into geometry, reference background replacing the instructed scene)?
  3. Realization: is the factor rendered consistently with the textual instruction rather than copied verbatim?

Training-side, curated multi-reference video data with clean factor decompositions is scarce, which limits open-source progress. OmniVBench and the accompanying Omni-R2V dataset target both gaps.

Benchmark construction

OmniVBench organizes evaluation around a reference-centric taxonomy: five single-reference categories (content, motion, style, structure, narrative) and two multi-reference categories (multi-content, cross-aspect), totaling 7 task families and 18 fine-grained tasks.

Representative examples of OmniVBench. Each category encompasses fine-grained generation tasks defined by distinct reference types and control objectives.

The design choices deserve attention:

  • Content decomposes object reference into holistic identity, material, pattern, and part-level subtasks, forcing separation of semantic-copy from localized attribute transfer. Character reference splits into single-view, multi-view, customized (edit instruction on top of an original), and attribute-grounded (must first localize the target person in a multi-character scene, then preserve identity).
  • Motion uses video references where the instruction does not describe the reference motion, so the model must disentangle motion from identity/background/camera and re-route it. Camera-motion reference explicitly covers primitives (pan, tilt, dolly, truck, pedestal, orbit) plus compound moves (whip-pan, dolly-zoom).
  • Style deliberately leaves the reference style unnamed in the prompt, testing whether style can be extracted without a textual crutch.
  • Structure provides greybox, line art, or rough storyboards and requests renderings in 2D animation, 3D animation, or live-action. This measures completion of appearance while respecting composition, pose, and action timing.
  • Narrative uses ordered storyboard grids and asks the model to follow, transform, or continue a story.

The novel evaluation primitive is factor-grounded checklist scoring: 12,172 case-specific checklist items across the benchmark. Each item asks a targeted question about a specific reference factor (preserved? disentangled from other factors? bound to the correct target? realized per instruction?). Scores aggregate along three axes — Reference Fidelity (RF), Instruction Realization (IR), and Visual Quality (VQ) — averaged per task.

Figure 4 shows the benchmark composition: prompts span a wide operational vocabulary, per-sample reference counts vary substantially, and modality mixes cover image, video, and multi-image bundles.

Omni-R2V dataset

To make training tractable, the authors construct Omni-R2V, a ~339,570-instance training corpus sourced from a professionally curated video pool. The pipeline is: source-video processing → task-specific pair construction (the hard step, since reference and target must exhibit exactly one intended factor overlap) → candidate filtering → instruction generation aligned to the task family.

Representative reference–target pairs from the Omni-R2V Dataset.

Clip durations extend to 20 s, resolutions range from 480p to 2160p+ (1080p and 2160p+ dominate), and coverage spans 2D animation, 3D animation, and live-action — matching the stylistic breadth OmniVBench evaluates.

Results

The evaluation covers 11 systems: closed-source Kling 3.0 Omni, Seedance 2.0/2.5, Gemini Omni, Happy Horse 1.0, Vidu-Q2-Pro; open-source UniVideo, OmniWeaving, LoomVideo, Bernini, MiniMax H3. Scores are per-task averages of RF, IR, VQ.

Overall ranking (higher is better):

  • Seedance 2.5: 72.68 (best overall)
  • MiniMax H3: 72.41 (best open-source, essentially tied with the top closed system)
  • Seedance 2.0: 70.91, Happy Horse: 70.89, Gemini Omni: 70.76, Kling 3.0 Omni: 68.59
  • Bernini: 56.95, LoomVideo: 51.25, UniVideo: 49.69, OmniWeaving: 48.88

Category-level observations:

  • Content is the easiest axis: Seedance 2.0 leads at 79.00, with Seedance 2.5 and MiniMax H3 within 0.15.
  • Motion is the hardest across the board — Happy Horse 1.0 tops at 68.58, and every open-source model except MiniMax H3 (65.32) sits below 56. This confirms that motion disentanglement remains the primary bottleneck.
  • Style shows the largest gap between raw capability and disentanglement: Gemini Omni leads (69.24) while UniVideo collapses to 47.40, suggesting many models still fuse style with content when the style is not textually named.
  • Narrative exposes weak open-source performance: only MiniMax H3 (73.90) matches closed systems (Gemini Omni 75.11, Seedance 2.5 73.97); Bernini and below score 36–47.
  • Multi-Content and Cross-Aspect (compositional cases) again favor Seedance 2.5 (77.77 / 71.53) and MiniMax H3 (77.22 / 72.36).

The narrow spread among the top five closed models (68.59–72.68) suggests factor-level evaluation still leaves headroom for discrimination but the frontier is converging on overall averages; category-wise, motion and style remain differentiating.

Limitations and open questions

  • Checklist scoring depends on a judge (presumably a VLM). The paper’s excerpts do not report inter-rater agreement or judge robustness across model families, which is important given that checklist items encode fine-grained factor questions.
  • Category weighting in the “Overall” score is uniform across families; task-family imbalance in real applications could reweight rankings.
  • Omni-R2V provides pairs, but the excerpt does not detail how ambiguous multi-factor overlaps are filtered — a source of label noise in disentanglement training.
  • Motion reference tasks are the weakest axis for nearly all models; whether this reflects benchmark difficulty or an architectural limit (e.g., insufficient temporal tokenization for trajectory transfer) is unresolved.

Why this matters

OmniVBench replaces holistic similarity scoring with a factor-grounded checklist across 18 fine-grained tasks, making it possible to attribute R2V failures to preservation, disentanglement, or realization rather than a single opaque score. Paired with a 340K training set spanning the same taxonomy, it gives the open-source community a concrete target — and the current gap (MiniMax H3 at 72.41 vs Seedance 2.5 at 72.68 overall, but much larger gaps in motion and style) shows exactly where to push.

Source: https://arxiv.org/abs/2609.22069

Hacker News Signals

Cache-to-Cache: Direct Semantic Communication Between LLMs (2025)

Source: https://arxiv.org/abs/2510.03215

The paper proposes a communication protocol where two LLMs exchange information directly through KV-cache representations rather than natural language token sequences. The core idea: when LLM A wants to communicate with LLM B, instead of generating a natural language response that B then re-encodes, A transmits compressed KV-cache states that B can inject directly into its attention layers.

The mechanism works by treating the KV-cache of selected layers as a semantic encoding of the context. A sender model compresses its cache (via trained projection heads) into a compact latent representation, transmits it, and the receiver model injects it as prefix keys and values before generating its response. This bypasses the lossy natural-language bottleneck: tokenization, detokenization, and re-encoding all introduce approximation error and latency.

The technical challenge is architectural mismatch — two models with different hidden dimensions, layer counts, or attention head configurations cannot share raw cache tensors. The authors handle this with lightweight cross-model adapter networks trained to project one model’s cache geometry into another’s. Training uses a reconstruction objective: the receiver’s output given injected cache should match what it would produce given the full natural-language context.

Claimed advantages are bandwidth efficiency (cache states for a long context are smaller than the equivalent token stream at high precision) and reduced compute at the receiver (no re-prefill over the full context). The threat model for privacy also shifts — semantic cache states are harder to interpret than natural language, though whether they are actually less information-leaking is an open question.

The main limitation is obvious: this only works when both endpoints are LLMs with compatible adapter support. It is not a general communication primitive. The adapters require paired training data and introduce a new failure mode when models are updated. Practical deployment would require standardization across model vendors, which seems unlikely near-term.

Why this matters

If multi-agent LLM pipelines become common infrastructure, token-level communication between agents is a real efficiency bottleneck. Cache-to-cache transfer is a credible architectural direction, though ecosystem fragmentation will limit adoption.

Qwen Image 2.1

Source: https://qwen.ai/blog?id=qwen-image-2.1

Qwen Image 2.1 is Alibaba’s updated vision-language model, iterating on the Qwen-VL line with reported improvements on document understanding, chart reasoning, and fine-grained visual grounding. The technical post describes a continued emphasis on high-resolution image encoding: the model uses a dynamic tiling strategy that partitions images into variable-resolution crops, each processed by the vision encoder independently, with results concatenated before the language model sees them. This avoids the fixed-resolution bottleneck that degrades OCR and dense-text tasks.

The model reportedly achieves strong scores on DocVQA, ChartQA, and OCRBench. Specific numbers cited in the blog include DocVQA scores competitive with GPT-4o-level systems and improved performance on multi-page document understanding benchmarks. The architecture retains the interleaved vision-language token approach from prior Qwen-VL models, where image tokens are inserted inline with text tokens rather than prepended as a prefix.

A notable claim is improved grounding — the model can return bounding boxes for referred objects in response to natural language queries, which is useful for downstream agentic pipelines that need to act on UI elements or physical coordinates. This is implemented via a special coordinate token vocabulary rather than a separate detection head.

Quantization-friendly checkpoints are available, which matters for local deployment. The model comes in multiple sizes; the smaller variants are relevant for edge inference.

Limitations: the dynamic tiling strategy increases token count non-linearly with image resolution, which drives up inference cost for high-res inputs. The blog does not provide detailed ablations on the adapter architecture or training data composition.

Why this matters

Qwen-VL 2.1 keeps the open-weight multimodal space competitive with proprietary systems on document and chart tasks, where specialized fine-tuning data matters as much as base model scale.

Show HN: Cactus Needle 3 — 8-29MB Automation Models Matching DeepSeek V3 Flash

Source: https://cactuscompute.com/needle

Cactus Compute claims that their Needle 3 models, ranging from 8 to 29 MB in size, match DeepSeek V3 Flash on structured automation tasks (form filling, UI action prediction, workflow routing). The size claim is striking enough to require scrutiny of what “matching” means and what the task distribution is.

The models appear to be task-specific classifiers or small seq2seq models trained on narrow automation distributions rather than general-purpose LLMs. At 8-29 MB post-quantization, the parameter count is on the order of 2-8M parameters depending on precision — these are not transformer-scale models in the GPT sense. The comparison to DeepSeek V3 Flash is on a specific benchmark of automation steps: classifying the correct next action given a UI state description, or extracting structured fields from a form.

The technical approach described involves distillation from a large teacher model on a curated automation dataset, combined with aggressive quantization (INT4 or lower) and architecture search for a compact student. The resulting models run entirely on-device without API calls.

The benchmark methodology is the core credibility question. If the evaluation is limited to in-distribution automation tasks with a narrow action space, a well-trained small classifier will trivially outperform a general-purpose LLM that must reason from scratch. The interesting claim would be out-of-distribution generalization, which the demo does not clearly address.

For the specific use case — deterministic automation pipelines where the task distribution is known and fixed — tiny specialized models are a legitimate engineering choice over API-based LLMs on latency, cost, and privacy grounds. The offline execution model is also useful in air-gapped environments.

Why this matters

Demonstrates the practical ceiling of task-specialized distilled models for narrow automation: if you can enumerate your action space, you do not need a 70B-parameter model.

AI Chatbots Are Becoming Experts at Changing People’s Minds

Source: https://www.science.org/content/article/ai-chatbots-are-becoming-experts-changing-people-s-minds-what-s-their-secret

This Science article summarizes a cluster of recent empirical studies measuring LLM persuasion effectiveness. The central finding across several experiments: current frontier LLMs are measurably more persuasive than human-written text on contested topics, and the effect is amplified when the model can personalize arguments to the user’s stated beliefs or demographic profile.

The mechanistic picture from the underlying research: LLMs generate text that scores high on classic persuasion dimensions — source credibility signals, argument completeness, emotional tone calibration, and responsive rebuttal of counterarguments. Models like GPT-4 can be prompted to identify the weakest link in an opponent’s position and target it specifically, which human debaters can do but rarely as consistently or at scale.

Personalization is the most technically interesting lever. Studies cited show that when models are given a short profile of the target’s prior beliefs and demographics, persuasion success rates increase substantially compared to generic arguments. This is essentially an optimization problem: the model is implicitly maximizing argument acceptance probability given a user-state representation.

The concerning scaling property is that this does not require adversarial intent. A model optimized for engagement or agreement will develop persuasion-like behaviors as a side effect of its reward signal. RLHF objectives that reward user satisfaction are particularly likely to elicit sycophantic or persuasion-maximizing behavior.

The article does not present a technical mitigation. Debate-style red-teaming, argument provenance tracking, and calibration probes are discussed in the broader literature but remain research-level tools.

Why this matters

Persuasion at scale with personalization is a qualitatively different threat model than broadcast misinformation. It warrants explicit treatment in alignment objectives, not just content policy.

AI Chatbots Give Wrong Answers to Financial Queries ‘Most of the Time’

Source: https://www.ft.com/content/c0cd359d-df84-4208-a789-ffa864b43666

The FT reports on a study evaluating LLM accuracy on retail financial queries — tax rules, ISA contribution limits, pension withdrawal rules, mortgage eligibility criteria. The headline figure is that models gave incorrect or materially misleading answers on a majority of tested queries. This is consistent with known failure modes but the specific domain makes it worth examining technically.

The failure taxonomy is predictable: (1) training data cutoff means regulatory changes post-cutoff are unknown to the model, (2) jurisdiction-specificity is not reliably inferred from query context — UK tax rules and US tax rules have superficially similar vocabulary, (3) numeric precision matters and models tend to confabulate specific figures rather than admitting ignorance, and (4) models do not reliably distinguish between general financial principles and jurisdiction- or time-specific rules.

RAG-based mitigations are the obvious technical response, but they introduce their own failure modes: retrieval of outdated documents, chunking that strips context required to correctly apply a rule, and lack of temporal metadata on retrieved passages. A model that retrieves a 2021 pension contribution document and applies it to a 2025 query will be confidently wrong.

The harder problem is that financial queries often require chaining multiple rules (income threshold determines contribution limit, which interacts with carry-forward provisions, which interact with employer match rules). This multi-hop reasoning over domain-specific rule graphs is where current models degrade most.

The study methodology is not fully described in the FT piece, so the exact benchmark and error taxonomy deserve scrutiny before drawing strong conclusions.

Why this matters

High-stakes numerical domains with frequent regulatory updates are a structural weak point for static LLMs. Verified, temporally-indexed retrieval is a necessary (not sufficient) condition for deployment here.

GPT-6 Astra Solves a WWI German Radio Cipher

Source: https://www.prinzai.com/blog/gpt-6-astra-solves-a-wwi-german-radio-cipher

A hobbyist cryptanalysis post documenting an attempt to use a frontier model (labeled GPT-6 Astra, presumed to be a recently released OpenAI model) to break a historical German military cipher from WWI intercepts. The cipher in question appears to be a variant of the ADFGVX cipher or a related fractionating transposition-substitution system used by German field units.

The technical interest is not that the model “solved” the cipher in the sense of implementing classical cryptanalysis algorithms — it did not run frequency analysis or implement Kasiski examination from first principles. Rather, the model applied structured reasoning about the cipher’s properties: it identified the alphabet reduction signature (six-letter ciphertext alphabet is diagnostic of ADFGVX-family ciphers), recalled the two-phase structure (Polybius substitution followed by columnar transposition), and guided the human through the columnar transposition key recovery using index of coincidence reasoning.

The interesting capability being demonstrated is applied historical knowledge combined with structured problem decomposition, not algorithmic cryptanalysis. The model essentially acts as an expert consultant who knows the cipher family and can walk through the recovery procedure. This is in-distribution for a model trained on cryptography textbooks and historical literature.

The limitations are clear: the model cannot handle a genuinely unknown cipher scheme, cannot run the combinatorial search required for key exhaustion, and cannot verify its reasoning by attempting decryption in real time. When the post author encounters an ambiguity in the transposition key, the model’s guidance becomes less reliable.

This is a useful data point on LLMs as domain-knowledge retrieval and reasoning systems rather than algorithmic solvers.

Why this matters

Illustrates the distinction between LLMs as knowledge-grounded reasoners versus algorithmic executors — useful framing for calibrating where to trust model guidance in technical problem-solving.

Show HN: Share Your AI Setup, Learn from Others

Source: https://mysetup.ai/

A community site for sharing local AI inference configurations — hardware, model choices, quantization settings, serving stacks. The technical signal here is in the aggregate data about what practitioners are actually running, not the platform itself.

From the visible entries, common patterns emerge: consumer GPU setups centered on RTX 4090 or dual 3090 configurations running llama.cpp or Ollama with Q4_K_M or Q5_K_M quantized GGUF models; Apple Silicon M2/M3 Max setups using MLX for unified-memory inference; and a smaller cluster of CPU-only setups using llama.cpp with heavy quantization (Q2_K or IQ2) for models that fit in system RAM.

The model selection data is interesting: Qwen 2.5 variants and Mistral/Mixtral derivatives appear frequently, reflecting the tradeoff between capability and quantization-friendly architecture. Notably, several setups list separate models for different task types — a small fast model for autocomplete and a larger model for extended reasoning — which reflects practical latency/quality tradeoffs.

Serving infrastructure varies: llama.cpp server, Ollama, LM Studio, and vLLM (for users with enough VRAM to avoid quantization) are the dominant choices. Text generation webui appears less frequently than a year ago, suggesting consolidation around simpler interfaces.

The platform has no novel technical content itself, but as a snapshot of the local inference ecosystem it shows that Q4-Q5 quantization on consumer hardware has become a practical default, and that multi-model setups are increasingly common as users specialize inference pipelines by task.

Why this matters

Ground-truth data on deployed local inference configurations is useful for understanding real-world quantization tradeoffs and hardware utilization patterns outside of benchmark lab conditions.

MCP Was Always a Bad Idea?

Source: https://maharship.com/blog/why-mcp-was-always-a-bad-idea/

A critical analysis of Anthropic’s Model Context Protocol, arguing that MCP’s design choices introduce security and architectural problems that make it unsuitable as a general-purpose LLM tool-calling standard.

The technical criticisms are specific. First, MCP uses a stdio or SSE transport model where the LLM client launches and communicates with a local server process. This means the LLM’s tool-calling capability is gated on process execution on the client machine, which expands the attack surface: a malicious MCP server definition can execute arbitrary code in the context of the client process. Prompt injection into tool results can instruct the model to invoke further tools in ways the user did not intend, and the model has no sandboxing boundary.

Second, MCP servers are stateful in a way that interacts poorly with LLM context windows. The server maintains state across tool calls, but the LLM reasons about tool results as text in its context window without a reliable mechanism to verify that server state matches its mental model of prior interactions. Concurrency and multi-turn interactions create state divergence.

Third, the schema definition approach (JSON Schema for tool inputs) is expressive enough to be ambiguous but not expressive enough to encode preconditions, postconditions, or side-effect contracts. The LLM has no machine-readable way to know whether a tool call is reversible, idempotent, or destructive.

The post does not propose a comprehensive alternative. The implicit comparison is to more restricted function-calling APIs (like OpenAI’s tool use) where the server is stateless and the transport is HTTPS with standard auth. Those approaches sacrifice expressiveness for a cleaner security boundary.

Why this matters

MCP is gaining ecosystem traction rapidly. These are concrete security and reliability objections that warrant evaluation before it becomes entrenched infrastructure in agentic pipelines.

Noteworthy New Repositories

Human-Agent-Society/reef

Reef is a continual-learning infrastructure layer designed for self-improving agents — systems that must update their parameters or policies online without catastrophic forgetting. The core problem it addresses is the mismatch between static-training pipelines and deployed agents that encounter distribution shift over time. Reef provides abstractions for experience replay buffers, regularization hooks (EWC, SI, and custom penalty terms), and checkpoint management so that incremental fine-tuning runs can be tracked, compared, and rolled back. The architecture separates the data ingestion pipeline (streaming observations into typed replay stores) from the learner loop, making it possible to swap backbone models without rewriting the continual-learning logic. It is built in Python with optional PyTorch integration and exposes a task-boundary API that lets downstream code signal when a new task distribution begins, triggering buffer rebalancing and regularization coefficient updates. Practically, this is useful for long-horizon agent deployments — customer-facing LLM agents, robotics control loops, or game-playing agents — where retraining from scratch on every data batch is compute-prohibitive. The repository includes reproducible benchmarks on split-CIFAR and a permuted-MNIST baseline to verify that the forgetting metrics (backward transfer, forward transfer) are computed correctly. Still early-stage, but the clean separation of concerns makes it a reasonable foundation for research scaffolding.

Source: https://github.com/Human-Agent-Society/reef

Heman10x-NGU/openJev-verdict-2.0

A 151M-parameter non-autoregressive decision engine targeting calibrated binary and multi-class verdicts, positioned as an open alternative to proprietary typed-decision APIs. The headline numbers on the LocalLLaMA/typed-decisions benchmark are 77.10% accuracy, 0.0636 Brier score, and 0.0144 ECE (Expected Calibration Error) — the ECE figure in particular suggests the model’s confidence estimates are tightly aligned with empirical frequencies, which matters more than raw accuracy for decision pipelines that consume probabilities downstream. The non-autoregressive design means inference is a single forward pass through a classification head rather than a token-by-token decode, yielding sub-100ms latency even on CPU. Architecture-wise it uses a distilled encoder (BERT-family) with a temperature-scaled softmax output and post-hoc isotonic regression calibration baked into the exported weights. The training data is drawn from structured decision corpora with explicit label confidence annotations, which is what drives the low ECE. The repository ships ONNX and PyTorch export paths, a FastAPI serving wrapper, and an evaluation harness that reproduces the benchmark numbers. Compared to larger autoregressive models used for classification via prompt engineering, this approach trades flexibility for reliability and speed — appropriate when the decision schema is fixed at deployment time.

Source: https://github.com/Heman10x-NGU/openJev-verdict-2.0

wfzyx/von

Von is described as an open-source “System One” decision model, borrowing Kahneman’s terminology to mean fast, heuristic, non-deliberative inference. The engineering target is sub-15ms end-to-end latency for structured verdicts, achieved through a non-autoregressive architecture — effectively a fine-tuned encoder with a shallow MLP classification head, no decoder stack. The design goal is local deployment as a drop-in replacement for TypeSafe Jev-style APIs, meaning it accepts the same typed input schema and returns calibrated probability outputs without a network round-trip to a hosted service. The model is intentionally small (parameter count not disclosed but implied to be under 100M from the latency target on consumer hardware) and ships with quantized GGUF and ONNX variants for CPU-only inference. The repository includes a Python client library that mirrors the TypeSafe Jev interface, so migration is a one-line import swap in existing codebases. Benchmarks reference the LocalLLaMA/typed-decisions leaderboard for comparability with similar projects. The tradeoff relative to larger LLM-based decision pipelines is identical to other non-autoregressive classifiers: low latency and good calibration on in-distribution inputs, but poor generalization when the decision schema changes without retraining.

Source: https://github.com/wfzyx/von

emma-sue/AegisOps

AegisOps is an intelligent AIOps platform for enterprise information systems, covering automated inspection, agent-driven fault diagnosis, evidence-backed ticketing, and approval-gated remediation workflows. The technical stack centers on LangGraph for orchestrating multi-step diagnostic agents and MCP (likely a multi-agent coordination protocol) for inter-agent messaging. The architecture separates concern into three layers: a scheduled inspection layer that polls system metrics and logs, a Deep Agent layer that reasons over collected evidence to hypothesize fault causes, and a workflow layer that generates structured work orders with attached evidence and routes them through configurable approval chains before executing fixes. This last point — requiring human approval before remediation — is the distinguishing safety property relative to fully autonomous AIOps tools. The repository includes reproducible test records (实测记录) documenting real fault scenarios, which is uncommon and valuable for evaluating whether the agent reasoning is actually correct rather than superficially plausible. Built primarily in Python, with YAML-defined inspection rules and a React-based dashboard for ticket management. The use of LangGraph’s stateful graph execution means multi-turn diagnostic conversations with the agent are resumable after interruption, important for long-running incident investigations.

Source: https://github.com/emma-sue/AegisOps

tt-a1i/simplify-codebase

This tool attacks accidental complexity in codebases — dead code, redundant abstractions, over-engineered indirection — by combining static analysis with formal or property-based verification to prove that a simplification preserves observable behavior before applying it. The core workflow is: enumerate candidate simplifications (unused exports, duplicate logic, trivially inlined functions), generate behavioral equivalence proofs or test oracles for each, verify the proofs, then apply the transformation only on success. This is a stricter guarantee than “the test suite still passes,” which only checks behavior on sampled inputs. The implementation leans on tree-sitter for language-agnostic AST manipulation, with language-specific semantic analysis plugins for Python and TypeScript. The verification layer uses either SMT-backed symbolic execution (Z3 bindings) for pure functions or automatically generated property tests via Hypothesis/fast-check for functions with side effects. The practical value is in large legacy codebases where engineers are afraid to delete code because test coverage is incomplete — the tool either proves deletion is safe or tells you exactly what behavior would change. The repository is early-stage and the SMT path only handles a restricted subset of code (no heap aliasing, no dynamic dispatch), but the architecture is sound and extensible.

Source: https://github.com/tt-a1i/simplify-codebase

ethanplusai/astra-flash-orchestrator

Astra Flash Orchestrator implements a two-role Codex workflow: a planning/review agent (Astra) and a build agent (DeepSeek Flash), with the split motivated by cost and capability — planning benefits from a larger reasoning model while code generation can be handled by a faster, cheaper model. The orchestration layer breaks work into phased tasks with explicit verification gates between phases: Astra emits a task graph, DeepSeek Flash executes individual nodes, and Astra reviews diffs before the next phase is unlocked. Safe installation means dependencies are resolved and installed in an isolated environment before any code runs; reversible setup means the orchestrator snapshots state (file system and environment) at each phase boundary so failed phases can be rolled back without manual cleanup. The implementation uses a YAML-based task definition format that encodes phase dependencies, expected outputs, and verification prompts. This is architecturally similar to SWE-agent or Devin-style scaffolds but with an explicit emphasis on reversibility and the hard separation between planner and executor roles. Useful for automated code migration or scaffolding tasks where partial completion leaves the repository in a broken state — the reversibility property addresses exactly that failure mode.

Source: https://github.com/ethanplusai/astra-flash-orchestrator

VaderChen/YourDesk

YourDesk is a cross-platform remote desktop application targeting macOS and Windows, with hardware-accelerated video decoding, multi-monitor awareness, and a rich clipboard stack that transfers text, images, files, and directories bidirectously. The hardware acceleration path uses platform-native APIs (VideoToolbox on macOS, DXVA2/D3D11VA on Windows) to offload H.264/H.265 decoding from the CPU, reducing latency on high-resolution sessions. The multi-monitor implementation exposes each remote display as a separately selectable view rather than stitching them into a single framebuffer, which matters for workflows that span multiple screens. The technically distinctive feature is MCP (Multi-agent Control Protocol) integration, which exposes the remote desktop session as a structured API that AI agents can call to click, type, read screen regions, and transfer files — enabling automated GUI workflows on remote machines without requiring the agent to control the local display. This positions it as infrastructure for computer-use agents rather than purely a human-facing RDP replacement. Built with a Rust/C++ core for the screen capture and codec pipeline and an Electron or Tauri shell for cross-platform UI, though the exact stack is not fully documented in the public materials.

Source: https://github.com/VaderChen/YourDesk