Daily AI Digest — 2026-09-29

Published

September 29, 2026

English · 日本語

arXiv Highlights

Post-Training Leaves Behavioral Shadows on Unrelated Decisions

Problem

Given a public base model M_0 with parameters \theta_0 and a private teacher M_T = M_0 + \delta obtained by post-training on some target task, can a student initialized from \theta_0 recover a target-task capability using only single-token, black-box, task-unrelated queries to M_T? Prior subliminal-learning work has shown that traits and preferences leak through unrelated generations, typically requiring extensive teacher outputs. This paper pushes that to the limit: one word per prompt, no target-task text, no logits, no teacher parameters, and a query set fixed before any teacher access. The question matters because it quantifies how much of \delta is behaviorally observable off-distribution — a practical concern for capability leakage, model theft via APIs, and understanding what post-training actually changes in the base’s decision surface.

Method: Active Taskless Distillation

The instrument, ATD, exploits the geometry of the base’s decision boundary. For each candidate prompt x_i constructed to elicit a choice between two ordinary single-token words a_i, b_i, compute under M_0 only:

q_i = \frac{\exp z_0(b_i \mid x_i)}{\exp z_0(a_i \mid x_i) + \exp z_0(b_i \mid x_i)}.

Retain x_i if |q_i - 0.5| \le \epsilon with \epsilon = 0.02, and additionally require that the argmax of M_0’s full vocabulary distribution is one of \{a_i, b_i\} — otherwise the “near-tie” is not really where the model would speak. On these knife-edge prompts, an infinitesimal update \delta tips the greedy output; the returned token is a one-bit measurement of \delta projected onto that pair’s direction.

Reading a post-training update through target-unrelated decisions with ATD.

The student, also initialized from \theta_0, is trained by cross-entropy on the retained prompt–word pairs. Nothing in the training data references the target task. The primary lineage uses Qwen2.5-1.5B-Instruct as both M_0 and student init; the teacher is a frozen rank-16 code-DPO LoRA, and the student is a fresh rank-16 LoRA.

Results

On HumanEval+, the flagship endpoint (chosen because execution-based scoring cannot be faked by imitation of surface tokens), 5,664 single-token teacher responses yield a +5.34 pp gain over an exact nuisance-matched shuffle control that disrupts prompt–response pairing while preserving all marginal statistics. The shuffle control is the critical baseline: it rules out that gains come from vocabulary exposure, prompt distribution, or generic fine-tuning nuisance.

Breadth experiments show non-trivial transfer to scientific knowledge (ScienceQA), commonsense reasoning (PIQA), and reading comprehension, and hold across additional model generations, sizes, and families with a separate task-specific teacher per target. The analyses in Section 5.4 confirm the active near-boundary design — not query volume — produces the signal, that the signal is source-specific (a code teacher yields code gains, not arbitrary gains), and that it is composable and graded in the number of retained pairs. Section 5.5 provides a useful negative: a large teacher gain does not automatically imply large transfer, indicating that only the component of \delta that reshapes the base’s near-tie decisions is legible through this channel.

Instrument extensions

A reference-relative pair-margin (RPM) objective replaces plain CE with a DPO-style loss that subtracts the frozen public margin:

\mathcal{L}_{\text{RPM}} = -\mathbb{E}_i \log \sigma\!\left(\beta\left[\log\frac{p_\theta(w_i^T \mid x_i)}{p_\theta(\bar w_i \mid x_i)} - \log\frac{p_0(w_i^T \mid x_i)}{p_0(\bar w_i \mid x_i)}\right]\right),

so the student learns the residual preference introduced by \delta rather than a preference already in M_0. On HumanEval+ the shuffle-relative gap rises from +5.03 to +6.30 pp; on PIQA both objectives stay positive (+1.25 vs +1.69 pp); on ScienceQA the students land at essentially the same accuracy (the wider RPM shuffle gap reflects a lower shuffle baseline); on OpenBookQA RPM underperforms CE (+2.07 vs +1.47 shuffle-relative). The authors present RPM as a boundary-aware refinement rather than a categorically better method.

Limitations and open questions

The setup assumes a known public ancestor — the student and analyst both hold \theta_0. In many real leakage threats the ancestor is unknown or the teacher has been continually trained, weakening the near-tie construction. The channel is small (5–6 pp on HumanEval+ is meaningful but far from full teacher recovery), and the paper does not characterize how the transferable fraction scales with \|\delta\|, LoRA rank, or the sharpness of M_0’s decision surface. It is also unclear which structural properties of a post-training update (\delta from DPO vs SFT vs RLHF, task-similarity to the carriers) determine legibility. Finally, the near-tie filter with \epsilon = 0.02 is a hyperparameter whose optimum likely depends on the base’s calibration.

Why this matters

ATD demonstrates that a single-token, task-unrelated query channel is sufficient to leak measurable capability across a black-box API, with a rigorous shuffle-matched control ruling out nuisance explanations. It also provides a clean scientific probe: post-training updates leave a globally distributed shadow on the base’s indifference set, which is a stronger claim about what fine-tuning changes than most interpretability work has established.

Source: https://arxiv.org/abs/2609.29233

How Far Are We from Removing the Visual Encoder? Scaling Laws for Encoder-Free Multimodal Pretraining

Problem

Contemporary MLLMs rely on a pretrained visual encoder (typically a CLIP/SigLIP-style ViT) that supplies a strong visual prior and compresses images into a small number of tokens before they enter the language decoder. Encoder-free MLLMs discard this module and feed patch embeddings (or pixels) directly to the decoder, yielding a single, unified transformer. This is architecturally attractive — no modality-specific pretraining, no bottleneck imposed by a frozen encoder’s inductive bias — but it is unclear whether the decoder can absorb the encoder’s role without paying an unacceptable compute penalty. The paper answers this quantitatively by fitting Chinchilla-style scaling laws separately for the two architectures and asking how the compute-optimal frontier and its allocation between parameters M and tokens D shift when the encoder is removed.

Method

The setup follows the standard IsoFLOP protocol. For a compute budget C \approx 6 M D, the authors sweep (M, D) pairs at fixed C, identify the loss-minimizing configuration, and fit

M_{\mathrm{opt}} \propto C^{a}, \qquad D_{\mathrm{opt}} \propto C^{b}, \qquad a + b \approx 1.

Two objectives are tracked separately: a text-only next-token loss and a multimodal loss on interleaved image–text data. The encoder-based baseline uses a fixed visual encoder producing a compressed token stream; the encoder-free variant patchifies images and feeds them into the decoder as ordinary tokens. Both bidirectional and causal attention masks over the visual tokens are studied to rule out mask-related confounds. Envelopes are constructed from raw per-run validation trajectories (Figure 11), and M_{\mathrm{opt}}, D_{\mathrm{opt}} are extracted by binning in \log C and refitting.

Envelope construction for the text objective, with fitted allocation laws on binned M_{\mathrm{opt}}, D_{\mathrm{opt}}.

Bootstrap resampling on the IsoFLOP fits yields 80% central intervals on a, which the authors use to argue that observed exponent differences are not sampling artifacts.

Results

The three headline findings are:

  1. Text allocation is invariant to encoder removal. The model-allocation exponent on the text objective is a = 0.427 for encoder-free versus a = 0.422 for encoder-based — indistinguishable, as expected since encoder removal changes only how visual tokens enter the decoder. Both are close to the Chinchilla a \approx 0.5 regime but tilted slightly toward data.

  2. Multimodal allocation shifts toward larger models. On the multimodal objective, a jumps from 0.464 (encoder-based) to 0.570 (encoder-free). The 80% bootstrap intervals are [0.458, 0.472] and [0.546, 0.595] respectively — non-overlapping, so the shift is statistically robust given the fitting protocol. Under causal masking on visual tokens the encoder-free exponent is 0.557 (Table 1), essentially identical to the bidirectional case (0.570), which excludes the mask as the driver. The interpretation: without a pretrained encoder, the decoder must itself learn visual features, and this representational burden is more efficiently absorbed by scaling parameters than by scaling data.

  3. Loss–compute frontiers converge on text and cross on multimodal. The text frontiers of the two architectures nearly overlap (unsurprising — text data is untouched). On the multimodal frontier the encoder-free model is worse at small compute but has a steeper slope; extrapolating the fitted laws, the crossover occurs near 10^{22} FLOPs, within the range of realistic pretraining budgets for frontier models.

The paper then probes how the decoder takes over the encoder’s function. Three signatures emerge during encoder-free training: (i) bidirectional attention over visual tokens becomes increasingly beneficial with compute (contrast this with text, where causal masking is preferred throughout); (ii) visual processing migrates to earlier layers as training progresses; and (iii) in MoE variants, expert routing for visual tokens becomes more specialized. Figure 7 makes the layerwise reallocation concrete: attention mass on visual tokens per decoder layer changes qualitatively across the “loss drop” that marks the emergence of visual competence in the 8B encoder-free run.

Multimodal validation loss and layerwise attention mass on visual tokens before/after the encoder-free transition.

Limitations and open questions

The 10^{22} FLOP crossover is an extrapolation from smaller-scale IsoFLOPs; the paper does not train at that budget, and the fitted exponents can drift when frontier runs are added, as prior scaling-law revisits have shown. The multimodal loss is a proxy for downstream capability — whether encoder-free models also close the gap on retrieval, grounding, and OCR-heavy benchmarks at the crossover is not established. The tokenization strategy for pixels (patch size, positional encoding, resolution) is fixed here; different visual tokenizers may shift both the intercept and the exponent. Finally, the analysis is on pretraining loss only, so instruction-tuning and alignment behavior of encoder-free models at scale remain open.

Why this matters

If the extrapolation holds, encoder-free MLLMs are not asymptotically disadvantaged — they are simply undertrained in current practice, and the “right” fix is to spend proportionally more on parameters than encoder-based recipes suggest. That reframes the visual encoder from a necessary component into a small-scale efficiency trick, with implications for how future unified multimodal transformers are designed and scaled.

Source: https://arxiv.org/abs/2609.35457

Knowing When Thinking Is Not Enough: Teaching Small Reasoning Models to Reason Beyond Their Parametric Knowledge

Problem

Test-time scaling via self-refinement is the dominant recipe for squeezing more from small reasoning models (sRMs), but the paper questions whether “think more” is the right primitive. The authors distinguish two failure modes at intermediate reasoning states: execution bottlenecks, where the correct continuation is reachable from the current state and reflection can surface it, and knowledge bottlenecks, where the missing element is factual and no amount of internal deliberation makes the solution reachable. Conflating these produces wasted compute on unreachable trajectories.

Overview of execution vs. knowledge bottlenecks and the FlyBy policy.

Diagnosing self-refinement’s ceiling

The authors run counterfactual interventions on Qwen3-{0.6B–14B} and Gemma4-{E2B, E4B, 12B} across AIME25/26, HMMT-Feb2026, and GPQA-Diamond. They restrict analysis to model-problem pairs with empirical solve rate in [0.25, 0.75] so that a viable path plausibly exists, then identify 13.4K endogenous evaluative-verification (EV) events and produce 215K paired counterfactual continuations (N=8 per condition) plus 57K information-conditioned continuations, all capped at 32K tokens.

Two findings drive the design:

  1. Frequency of self-verification does not predict gain. Injecting more EV events consolidates probability mass onto solutions already reachable from the state s preceding the emission e, rather than opening new solution manifolds.
  2. Refinement’s efficacy is localized to uncertain-but-reachable states. When correct continuations are unreachable from s, only information-conditioning — i.e., injecting external content — moves the trajectory into a solvable region.

This reframes the sRM gap as partly a reachability problem rather than a strategic-allocation problem.

Method: FlyBy

FlyBy treats external computation as a callable action inside the reasoning trace. The model can either continue local reasoning or emit a query action with a chosen depth d \in \{1, 2, 3\}, where each depth binds to a stronger backend and larger token budget:

  • d{=}1: DeepSeek-V4-Flash, 128 tokens, $0.094 / $0.188 per 1M in/out
  • d{=}2: DeepSeek-V3.2, 512 tokens, $0.269 / $0.400
  • d{=}3: DeepSeek-V4-Pro, 1536 tokens, $0.435 / $0.870

Two anti-delegation constraints matter: (i) external models never see the original problem, and (ii) queries with high n-gram overlap with the problem are rejected during both training and evaluation. This forces the sRM to reformulate a knowledge subgoal rather than outsource the task.

Training is two-stage:

  1. SFT bootstraps the multi-depth query action distribution, giving the model well-formed query syntax at each depth.
  2. Cost-aware RL (80 steps) calibrates whether to query, what to ask, and how much compute to purchase, balancing pass-rate against monetary cost.

FlyBy-4B is trained from Qwen3-4B; an 8B variant is also trained. The action space is composed inside the reasoning trajectory, so a single rollout may interleave local thinking with multiple queries of different depths.

Results

Evaluation covers 1,158 hard problems (defined by vanilla Qwen3-4B \mathrm{pass@}1 \leq 0.25) across six benchmarks, reporting \mathrm{pass@}8 and cost in milli-dollars per 8 rollouts.

Headline numbers (avg \mathrm{pass@}8 / avg cost m$):

  • Qwen3-4B: 21.17 / 10.63
  • Qwen3-8B: 34.23 / 15.38
  • Qwen3-14B: 41.64 / 20.35
  • Search-R1: 25.81 / 8.35 (retrieval cost excluded)
  • Query Opening baseline: 34.06 / 8.56
  • FlyBy-4B-SFT: 36.39 / 7.15
  • FlyBy-4B: 45.96 / 7.42
  • FlyBy-8B: 51.81 / 14.75
  • DeepSeek-V4-Pro (frontier ref): 56.89 / 50.08

FlyBy-4B outperforms Qwen3-14B by 4.3 pass@8 points at roughly one-third the cost, and FlyBy-8B closes to within 5.1 points of DeepSeek-V4-Pro at less than one-third of its cost. Per-benchmark, FlyBy-4B posts 60.01 on GPQA-Diamond (vs. 52.63 for Qwen3-14B) and 70.73 on ChemBench (vs. 59.30). On problems where Qwen3-4B fails all 16 rollouts, FlyBy-4B still reaches 28.7% \mathrm{pass@}8, evidence that the querying policy expands reachability rather than merely re-weighting within-model trajectories. Coverage gains over Query Opening grow with problem difficulty, consistent with knowledge bottlenecks dominating on harder items.

Limitations and open questions

  • The external tool is priced via a fixed API tier; the learned policy may not transfer if the tier structure or backend capability changes.
  • n-gram overlap is a shallow anti-delegation filter; semantic paraphrase leakage is not directly measured.
  • Reachability analysis relies on the [0.25, 0.75] solve-rate filter to identify viable states; behavior on truly unreachable-from-any-state problems is not characterized.
  • Only 80 RL steps are used; whether the cost-quality frontier continues to improve with more RL, or whether it collapses into over-querying, is unclear.
  • Depth is discretized to three levels; a continuous or learned budget could sharpen the cost-performance trade-off.

Why this matters

The paper offers a mechanistically grounded reason to stop treating “more thinking” as a monolithic scaling axis for small models: much of self-refinement is redistribution within a fixed reachable set, and knowledge bottlenecks require an information source, not more tokens. Framing external calls as a depth-typed action trained with cost-aware RL yields a 4B model that beats a 14B same-family model at lower cost, suggesting selective querying is a more productive lever than self-refinement for sRMs deployed at scale.

Source: https://arxiv.org/abs/2609.34327

DepthBench: Measuring How Residual Connections Enable More Computational Depth

Problem

Increasing transformer depth L is a canonical way to add capacity, but empirical evidence has accumulated that deep layers in Pre-LN transformers contribute progressively less: neighboring layers produce nearly identical residual-stream states, and effective computational depth saturates well below architectural depth. A wave of proposed fixes — LayerNorm Scaling (LNS), Sandwich-LN, DeepNorm, KEEL, MoDA, Hyper-Connections (HC, mHC), Attention Residuals (Block/Full AttnRes) — claim to alleviate this. The open question is whether any of these actually convert extra layers into extra computation, or whether reported gains conflate depth utilization with unrelated confounders (learning-rate schedules, initialization, parameter counts).

DepthBench addresses this by fixing total parameters and pretraining recipe, then sweeping the aspect ratio r = d_{\text{model}}/n_{\text{layer}} from shallow-wide to deep-narrow across 10 architectures at 200M-1.6B scale. If an architecture genuinely exploits depth, its validation loss should decrease as r decreases; if not, deeper-narrower shapes should stagnate or regress.

Setup and metrics

For each architecture the authors sweep L while shrinking d_{\text{model}} to hold total parameters fixed (400M principal suite, plus 300M fixed-backbone and 1.6B replicates). Beyond validation loss, they report teacher-forced NLL on coding/STEM/math following the pretraining-eval protocol of Team (2026) and Chen et al. (2026). Analysis complements loss with two structural probes:

  1. Layer-to-layer angular distance d(h_\ell, h_{\ell+n}) = \frac{1}{\pi}\arccos\!\left(\frac{h_\ell \cdot h_{\ell+n}}{\|h_\ell\|_2 \|h_{\ell+n}\|_2}\right) computed on residual-stream states (concatenated for HC/mHC, post-mixing for AttnRes).

  2. Explicit residual-path weights w_{i\to\ell} giving the effective contribution of sublayer i’s output to the pre-norm input at sublayer \ell. For Pre-LN this is trivially w_{i\to\ell}\equiv 1; for Full AttnRes it is the per-token softmax weight on f_i; for HC it is the telescoping product w_{i\to\ell}=\beta_i^\top A_{i+1}\cdots A_{\ell-1}\alpha_\ell over the hyper-connection mixing matrices.

Main quantitative findings

At 400M, Pre-LN validation loss rises monotonically from 2.759 at L=16 to 2.782 at L=32 — reallocating parameters from width to depth strictly hurts. Sandwich-LN, LNS, DeepNorm, KEEL, and MoDA are either flat or non-monotonic in L, with optima at intermediate or shallower shapes. Normalization tweaks, in other words, do not unlock depth.

HC and Full AttnRes are the outliers: they show monotonic improvement as models get deeper and narrower, holding across the 200M-500M range and reproducing at 1.6B. The NLL evaluations on coding/STEM/math track the validation-loss ordering, so the gain is not an artifact of the pretraining distribution.

Representation-level diagnosis

The angular-distance heatmaps (Figure 5) explain why.

Angular distance between layer \ell and layer \ell+n across nine architectures at L=32.

Pre-LN, Sandwich-LN, LNS, DeepNorm and KEEL all collapse toward the yellow (small-distance) regime in the deep-layer band: successive layers make near-trivial refinements to the residual stream. HC, mHC, Block AttnRes and Full AttnRes retain substantially larger distances and a heterogeneous, non-smooth structure — Block AttnRes shows sharp jumps at block boundaries with smoothness inside; HC/AttnRes show layer-specific transformations throughout. The critical claim is not that HC representations move farther overall, but that they move in a layer-specific rather than diffusion-like way.

Because learning rate materially affects these patterns, the authors normalize to 2\times 10^{-3} across architectures (excluding DeepNorm, which diverges). Figure 13 shows that at other LRs Pre-LN and LNS can look artificially better or worse on the angular-distance test, which is why prior work reporting depth utilization without LR control is not comparable.

The path-weight decomposition (Figure 7) makes the mechanism concrete: in Pre-LN every earlier sublayer feeds later layers with weight 1, so information from all depths is superimposed uniformly and later layers cannot selectively route around uninformative predecessors. Full AttnRes computes per-token softmax weights over predecessors, effectively learning a token-conditioned depth mixture; HC learns the mixing through the \beta_i^\top A_{i+1}\cdots A_{\ell-1}\alpha_\ell chain, which admits both amplification and suppression of specific earlier outputs.

Explicit residual-path weights for Pre-LN, Full AttnRes, and HC.

Limitations and open questions

  • All experiments are pretraining-loss/NLL evaluations; no downstream task-quality curves are reported, so it remains open whether depth gains from HC/AttnRes translate into commensurate benchmark improvements at compute-matched inference cost.
  • The Full AttnRes softmax-over-predecessors adds memory and bandwidth costs proportional to L per sublayer; the paper does not fully quantify wall-clock or KV-cache implications.
  • The 1.6B replicate is small relative to the depths where representation collapse is most acute in production LLMs (70B+ at L\ge 80); it is unclear whether the same aspect-ratio monotonicity persists.
  • DeepNorm’s exclusion at 2\times 10^{-3} leaves an asymmetry: its published claims presume a specific initialization scale, and DepthBench’s fixed-recipe protocol may under-represent it.
  • The angular-distance and path-weight diagnostics are correlational; the paper does not intervene (e.g., prune sublayers, ablate specific paths) to causally verify that heterogeneous representations cause the depth benefit.

Why this matters

DepthBench provides the first controlled comparison isolating aspect-ratio scaling from parameter count and training recipe, and it delivers a clean negative result: the popular normalization-based fixes (LNS, Sandwich-LN, DeepNorm, KEEL, MoDA) do not increase computational depth, while architectures that reshape the residual pathway itself (HC, Full AttnRes) do. This reframes the depth-utilization problem as fundamentally about routing rather than magnitude control.

Source: https://arxiv.org/abs/2609.32534

Beyond Teacher Assignment: Domain-Normalized Multi-Teacher On-Policy Distillation

Problem

Multi-teacher on-policy distillation (MOPD) is the standard recipe for folding several RL-trained specialists back into a single generalist student: each prompt carries a domain label d, the student samples a response y, and the corresponding teacher T_d scores every token. The per-token distillation advantage is

A_t = \log p_{T_d}(y_t \mid h_t) - \log \pi_u(y_t \mid h_t),

used directly as the policy-gradient weight under a clipped OPD loss. The routing decides which teacher speaks but not how loudly. Empirically, on Qwen3.5-9B the label-routed MOPD student scores 58.4% Total across six benchmarks — worse than the best single-teacher OPD student (Code teacher, 58.5%) and essentially failing to inherit the math specialist’s AIME gains (55.3/63.2 vs. the Math expert’s 59.7/69.4 on AIME25/26). The mechanism the authors identify is a scale mismatch: in the first training batch, IF log-ratios are 2.3–4.4x more dispersed than the pooled batch signal while math log-ratios are about half as dispersed. Because all three domains write to the same parameters, IF feedback dominates the update and math contributions get drowned out.

Method

DN-MOPD keeps the label-routing pipeline and inserts a single scalar rescaling per domain per batch. Let r_{i,t} = \ell^T_{i,t} - \ell^{\text{roll}}_{i,t} be the teacher-minus-cached-student log-probability on valid response tokens. Compute the population standard deviation \sigma_d over tokens with d_i = d and \sigma_{\text{all}} over the pooled batch. Each domain gets one multiplier

w_d = \operatorname{clip}\!\left(\frac{\sigma_{\text{all}}}{\sigma_d},\, 0.25,\, 4\right), \qquad \widetilde{A}_t = w_d A_t,

and the student is updated with the standard clipped OPD objective on \widetilde{A}_t. For an unclipped domain, \operatorname{Std}(w_d r_d) = \sigma_{\text{all}} by construction, so each domain’s rollout signal is placed on the same dispersion scale as the pooled batch. The rule targets dispersion rather than mean magnitude — it does not attempt to estimate teacher quality or capability gap, in contrast to Open-MOPD-style budget allocation by mean reward. Label routing corresponds to the special case w_d \equiv 1.

The training loop is: (1) student rolls out on labeled prompts; (2) matched teacher scores each response; (3) \{\sigma_d, \sigma_{\text{all}}\} are computed from cached log-ratios and w_d is applied to actor-side advantages; (4) clipped OPD update. The extra cost is a single reduction over the batch per step.

Results

At 9B, DN-MOPD reaches 59.6% Total vs. 58.4% for label-routed MOPD, 58.5% for the best single-teacher OPD (Code) and 58.9% for parameter averaging. The math recovery is the clearest signal: AIME25/26 jump from 55.3/63.2 (MOPD) to 58.9/67.7 (DN-MOPD), close to the Math expert’s 59.7/69.4 and the Math-teacher single-OPD’s 58.9/69.5. Code moves from 56.8/52.6 to 56.3/51.4 (essentially flat) and IF from 84.0/38.4 to 84.5/38.7. So the gain is not a uniform lift — it is specifically the previously-suppressed low-dispersion domain climbing back up, which is what the \sigma_{\text{all}}/\sigma_d rule predicts.

At smaller scales the pattern holds: on Qwen3.5-4B and 2B (Table 2), DN-MOPD beats label MOPD on the three-domain Total. SeqKD-SFT remains a strong offline baseline (62.6% at 9B, 55.9% at 4B), but it uses fixed teacher answers with a 16K generation cap and 84 updates — a different token-exposure and compute regime — and does not produce an on-policy specialist-merged student. Among the on-policy multi-teacher recipes (Uniform pool 58.0, Dynamic router 58.1, Label 58.4), DN-MOPD’s 59.6 is the only entry that both beats the best single-teacher OPD and recovers most of the math specialist’s advantage. The authors report robustness across three student seeds (42/43/44) and two evaluation caps (8K and 16K).

Limitations and open questions

The multiplier is a single scalar per domain per batch and the clip range [0.25, 4] is fixed; there is no per-token or per-response calibration, and no analysis of what happens when a domain is genuinely low-signal (small \sigma_d because the teacher and student already agree) versus low-scale for other reasons — both get amplified. Dispersion is a proxy for gradient contribution but ignores gradient direction and loss curvature, which the authors acknowledge. Code performance does not improve and slightly regresses on LCB v6, suggesting normalization is not sufficient when the teacher-student gap is already narrow. The comparison to SeqKD-SFT is not apples-to-apples on compute. Finally, all experiments are within the Qwen3.5 family with three domains; scaling to more domains, to teachers of mixed sizes, or to settings without clean domain labels is untested.

Why this matters

Multi-teacher distillation has been treated as a routing problem, but if teachers were independently trained with RL their advantage scales are not comparable and routing alone silently reweights the student’s objective. A one-line dispersion normalization fixes the dominant failure mode — math being erased by IF — at negligible cost, and gives a concrete diagnostic (\sigma_d vs. \sigma_{\text{all}}) to check before blaming the router.

Source: https://arxiv.org/abs/2609.35347

Self-Evolving Coding Agents: From Digital Programs to Physical-World Intelligence

Problem

Vision-language-action (VLA) and world-action (WAM) models compress observations and instructions into action chunks:

(\text{observations},\text{instruction}) \longrightarrow \text{action chunk} \longrightarrow \text{stop}.

Task state, subgoal progress, preconditions, and recovery logic are all implicit in network activations, so they cannot be inspected or edited. The paper reports a diagnostic finding that motivates the whole framework: a policy trained without the language instruction nearly matches one trained with it, indicating that much of what these models learn is a scene-to-trajectory fit rather than a task representation. For an n-step task with per-step success p_i, open-loop success is approximately \prod_i p_i; without state-level verification, failures compound and cannot be localized. This is why RoboCasa365 long-horizon composite tasks collapse to 34.3% success for a strong VLA (XR-1) even when its atomic-skill success is around 78%.

Method: Physical Coding

The proposal is to make the state and the policy explicit executable artifacts.

  • Code as World W_t: a program built from images, depth, sensor readings, and tool outputs. It encodes objects, relations, measurements, constraints, and progress predicates that can be evaluated against evidence.
  • Code as Policy P_t: a program that plans, dispatches tools (including VLA/WAM policies), handles branches/interrupts, verifies, and recovers.

Code as World and Code as Policy as the executable interface between the agent and the physical environment.

The runtime, called HexaAnything, wraps this with a typed Harness H_t=(O_t,\mathcal{T}_t,V_t,R_t,M_t,\Gamma_t) — observation interface, tool registry, verifier, recovery operator, provenance-aware memory, and execution contracts (permissions, budgets, safety gates). At each step,

n_t = \pi_{\theta_t}(z, W_t, P_t, O_{\le t}, M_t), \qquad u_t = \mathrm{Dispatch}(n_t, \mathcal{T}_t),

x_{t+1} \sim E(x_t, u_t), \quad W_{t+1} = \mathrm{UpdateWorld}(W_t, O_{t+1}, y_t), \quad q_t = V_t(W_{t+1}, y_t, z).

Crucially, q_t is a typed verifier verdict computed from evidence, not a model-generated completion token. Depending on q_t, the Harness continues P_t, interrupts, invokes R_t, requests a new observation, or terminates. The complete trace \xi=\{z,W_{0:T},P_{0:T},O_{0:T},u_{0:T},y_{0:T},q_{0:T},M_{0:T}\} is the unit for attribution and later model updates.

The Observe–Plan–Act–Verify–Recover loop, with verified traces feeding back to model, Harness, and environment updates.

Self-evolution is organized over four coupled targets — Harness, data/environments, model/algorithms, and embodiment/compute — with all edits gated by independent verification against fresh evidence, and rollback on failure.

Four evolvable targets share the executable world–policy interface; execution evidence proposes edits, evaluation admits or rolls them back.

Results

Harness on top of a fixed VLA (RoboCasa365). With XR-1 held fixed as an action tool and GPT-5.6-Sol as planner, HexaAnything raises success from 34.3% → 38.3% on Composite-Unseen (+4.0), 54.8% → 61.5% on Composite-Seen (+6.7), and 56.6% → 61.1% overall. A generic coding agent (Codex) with the same LLM reaches 59.5% overall but does not exceed native XR-1 on Composite-Unseen (34.1%), showing the gain is due to the structured world/policy interface, not just LLM planning. Swapping the planner to an open Qwen3.8-27B still yields 37.3% on Composite-Unseen (+3.0 over XR-1). On three Composite-Unseen tasks with 100 seeds each, gains reach +31.0, +15.0, and +17.0 points; e.g., LoadKebabSandwich moves from 17% to 48% because “both ingredients inside” becomes a verifier-backed precondition for closing the oven, rather than being encoded implicitly in XR-1’s trajectory distribution.

Beyond manipulation (PhyBench). Three simulated experiments — Hooke’s law, pendulum-based g, and coupled-oscillator normal modes — require the agent to run the apparatus, log measurements bound to conditions, fit models, and decide whether to remeasure from residuals. Reference values are hidden and used only by the evaluator after submission. This closes a gap that action-only models cannot address: the deliverable is a quantitative claim, not a terminal grasp.

Real robot deployment (AgileX PiPER-X dual-arm). Under a “Vibe as Policy” arrangement — an agent writes and freezes tools between episodes, a frozen execution agent composes them at run time via URAI — five of seven tabletop tasks succeed 3/3. Against published GPT-Policy/Robocurve baselines using the same underlying LLM on other hardware, HexaAnything completes tic-tac-toe 2.7× faster (5.0 vs 13.6 min), bottle-cap unscrewing 3.5× faster (5.1 vs 17.9 min), and block-into-bowl 2.4× faster with 4.7× fewer output tokens (443 vs 2.1k). The two partial tasks (toss blocks 1/3, fold clothes 1/3) fail on perception and reach, not on the tool interface.

Limitations

The report explicitly covers only Harness and tool evolution plus a first data-to-model update; joint evolution across all four targets (including embodiment/compute) is left open. Real-robot trials are three seeds per task and the cross-hardware speed comparisons are not same-robot. Verifiers themselves are hand-authored predicates whose failure modes (missing state variables, mis-specified constraints) are diagnosed post hoc; there is no principled account of verifier completeness. The approach also assumes that failures can be attributed to a specific editable artifact — a strong assumption when the underlying VLA fails silently within its action chunk.

Why this matters

Physical Coding is a serious attempt to give robotic agents the property that made digital coding agents self-improving: a single executable medium in which observation, action, and feedback are inspectable, versioned, and edit-testable. The empirical wins are modest per-benchmark but consistent across a fixed-VLA setting, an open-weights planner, a scientific-experiment domain, and a real dual-arm platform, which is unusually broad evidence that the interface — not the underlying policy — is doing the work.

Source: https://arxiv.org/abs/2609.35432

MassAlloc Attention: Let Attention Allocate Its Own Compute

Problem

Standard causal attention pays the full post-score cost — softmax update, V loads, PV accumulation on the forward, and probability reconstruction plus \mathbf{dP}, \mathbf{dS}, \mathbf{dQ}, \mathbf{dK}, \mathbf{dV} on the backward — for every legal (q,k) tile, even though the realized softmax distribution assigns near-zero mass to most of the score space. Prior sparsity approaches (top-k, block/window patterns, hash bucketing) commit to a fixed structural budget before observing \mathbf{S} = \mathbf{Q}\mathbf{K}^\top, so they either drop information or over-provision. MALA proposes a middle path: always pay for score discovery, then let the score distribution itself decide whether to execute post-score work.

Method

MALA separates each tile into (i) score discovery, forming \mathbf{S}_{ij} = \mathbf{Q}_i \mathbf{K}_j^\top, and (ii) post-score execution. All score tiles that are causally legal are computed; the question is only which tiles progress to the softmax/PV path.

The retention criterion is a length-aware, dimensionless tolerance \tau > 0. For query q with causal support \mathcal{C}_q and L_q = |\mathcal{C}_q|, the reference is the uniform contribution 1/L_q; the per-tile test compares the largest per-key contribution ratio in that tile against \tau/L_q. In equal-length causal training with zero-based positions this reduces to L_q = q+1; in single-token decoding L_0 = N_k. Rescaling by L_q makes \tau scale-free across positions and context lengths, so a single \tau (the paper uses \tau=1 everywhere: forward, backward, prefill, decode) governs both training and inference.

The forward pass makes the retention decision online, using the evolving online-softmax normalizer (the running m_i and \ell_i state familiar from FlashAttention) as its estimate of contribution scale. Because the online normalizer is monotone-improving, the decision uses whatever mass has been observed up to that tile. The backward pass has the luxury of the finalized normalizer \ell_i^{\mathrm{final}} and reconstructs a nested retained support: any tile retained under the finalized normalizer would have been retained under some prefix of the online normalizer, so backward retention is a superset-consistent replay using only standard attention state. No extra buffers beyond the row-wise m_i, \ell_i, \mathbf{L}_i are required, so the primitive fuses cleanly into a FlashAttention-style kernel.

Crucially, \tau specifies admissible contribution, not a compute budget. Sharp heads/rows reject many tiles; diffuse rows retain many. Allocation is therefore adaptive across layers, heads, inputs, and positions.

Results

The central experiment is a matched-work study at 8K context. MALA under \tau=1 realizes an average of ~1,024 retained post-score key slots per query. Three diagnostic oracles are constructed that execute exactly the same total post-score work — a shared budget of ~1,024 slots per query — but rank candidate regions using the finalized per-instance reference mass. They differ only in the granularity at which the work is distributed across decisions:

  • Position-only: mean omitted mass 0.6012%, mean relative output L_2 error 0.6231%.
  • Layer/Head/Position (static specialization): 0.1721% and 0.2817%.
  • Input/Layer/Head/Position (per-instance oracle): 0.0182% and 0.0164%.

The per-instance oracle beats static layer-head specialization by 9.5\times on omitted mass and 17.2\times on output error, quantifying the value of distribution-adaptive rather than structure-adaptive allocation. MALA’s online decisions, without access to the finalized normalizer, land at 0.0188% mean omitted mass and 0.0174% mean output L_2 error — essentially indistinguishable from the offline oracle (0.0182% / 0.0164%). This is the paper’s cleanest result: given the same compute, MALA’s online allocation recovers nearly all of the achievable benefit of oracle per-instance allocation.

Across context lengths from 1K to 32K, the same \tau=1 maintains low forward output error and low gradient error relative to a FullAttn reference, suggesting the length normalization in \tau/L_q is well-calibrated across scales rather than an 8K-specific tuning artifact.

In a controlled associative-recall setting, MALA tracks FullAttn as context grows, reaching 89.67% accuracy at 8K versus 89.97% for FullAttn — a 0.30-point gap that indicates the long-range retrieval channel is preserved despite skipping most post-score work.

Scaling-law and continued-training studies run at 14B and 32B on 128 H100s use the same \tau=1; downstream knowledge, reasoning, and long-context evaluations are reported at both scales.

Limitations and open questions

  • The primitive still pays O(N_q N_k) for score discovery; savings are strictly in the post-score path. For very long contexts the QK matmul cost is not addressed.
  • Retention decisions on the forward use only the online normalizer prefix, which is systematically an under-estimate early in a row; the paper argues backward nesting is safe but does not characterize how often the online policy is strictly conservative vs. the offline oracle across regimes beyond 8K.
  • \tau=1 is asserted as canonical across forward/backward/prefill/decode, but no sensitivity sweep on \tau is summarized in the shown sections, and the 0.30-point associative-recall gap suggests small but nonzero information loss that may compound in tasks with different attention entropy profiles.
  • No wall-clock or FLOP-accounting numbers appear in the excerpts provided; the “compute allocator” framing is validated on omitted-mass and output-error metrics rather than on realized throughput.

Why this matters

MALA reframes attention sparsity from a structural prior (windows, blocks, top-k) to a runtime allocator driven by the softmax distribution itself, with a single scale-free tolerance \tau/L_q that transfers across training, prefill, and decoding. The matched-work result — online decisions within 0.0006 percentage points of an offline per-instance oracle — is a strong signal that the online-softmax state already contains most of the information needed to skip low-contribution work, which is exactly the state FlashAttention-style kernels already maintain.

Source: https://arxiv.org/abs/2609.32712

Hacker News Signals

ESP32S3 cluster running 1.58-bit (BitNet) Language model

BitNet b1.58 quantizes every weight to \{-1, 0, +1\}, which collapses matrix-vector products into additions and subtractions — no multiplies. This repository deploys that idea on clusters of ESP32-S3 microcontrollers (dual-core Xtensa LX7 at 240 MHz, 512 KB SRAM, up to 8 MB PSRAM via QSPI). Each node holds a shard of the model in its PSRAM and communicates with neighbors over UART or SPI. The 1.58-bit encoding means weights pack tightly: roughly 1.58 bits per parameter versus 16 bits for fp16, a ~10x reduction in weight storage. The implementation uses lookup tables for the ternary multiply-accumulate, which fits comfortably in the L1 data cache of the LX7 cores. The cluster topology is linear: token embeddings and logits pass through a chain of nodes, each responsible for a contiguous block of transformer layers. No dynamic routing or all-reduce is needed for inference, which keeps the inter-node protocol trivial. The model sizes demonstrated are in the sub-100M parameter range — small enough that a handful of boards can hold the full parameter set. Inference speed is slow by any server standard (seconds per token), but the point is feasibility: a complete autoregressive language model running on hardware costing a few dollars per node with no cloud dependency. Open questions include how layer-parallel execution (pipelining prefill across nodes) could improve throughput, and whether attention KV caching fits in the constrained PSRAM budget at longer context lengths.

Source: https://github.com/Low-Zi-Hong/ESP32s3-LLM-Cluster

Faster prompt lookup drafting in llama.cpp

Prompt lookup decoding is a speculative decoding variant that sources draft tokens directly from the prompt text rather than a separate smaller model. The idea: for tasks like summarization, editing, or Q&A over a document, the model’s output frequently copies n-grams verbatim from the input. During generation, a sliding window scans the prompt for matches to the last k tokens of the current context; if a match is found, the next m tokens following that match in the prompt are used as speculative drafts. The target model then verifies all m tokens in a single forward pass using the standard speculative-decoding acceptance criterion. No auxiliary model is needed, which eliminates the memory and coordination overhead that makes speculative decoding awkward in practice. This post examines the llama.cpp implementation and profiles where cycles are spent. The main bottleneck identified is the n-gram lookup itself: the naive implementation is O(|\text{prompt}| \cdot k) per generation step. The author proposes and benchmarks a hash-based index built once over the prompt tokens, reducing lookup to O(k) amortized. The speedup is measurable for long prompts (tens of thousands of tokens) where lookup overhead becomes non-trivial. The post also notes that the acceptance rate — the fraction of speculative tokens accepted — is highly task-dependent: near 1.0 for near-verbatim extraction tasks, near 0 for open-ended generation, which bounds the technique’s applicability. The optimization lands in llama.cpp as a practical latency improvement for document-heavy use cases on CPU inference.

Source: https://jadidbourbaki.github.io/blog/prompt-lookup-llama-cpp/

A single function Jev-like wrapper for LLMs, including vision models

“Jev” refers to a pattern where a single structured function call routes a task description plus a typed output schema to an LLM and returns a parsed, validated result — essentially treating the LLM as a function from natural language plus type signature to a concrete value. This post implements that pattern in ~100 lines of Python, extending it to vision models by accepting image inputs alongside text. The wrapper constructs a structured output prompt, calls the API (OpenAI-compatible endpoints), parses the JSON response, and validates it against a Pydantic schema, retrying on parse failure up to a configurable limit. For vision, the image is base64-encoded and injected into the messages array per the multimodal content block specification. The author’s design decisions are minimal: no framework dependency, no agent loop, no streaming — just the core request-response pattern. The value is that this covers a large fraction of practical LLM integration use cases (extraction, classification, structured generation) with a surface area small enough to audit and modify. The schema-driven retry loop is the most mechanically interesting piece: on a failed parse, the validation error is appended to the conversation as a user message, prompting the model to correct its output format rather than starting fresh. This is cheaper than a full retry and often converges in one correction step. The post also handles function calling as an alternative to JSON mode for models that support it but not native structured outputs, using a dummy tool definition whose single argument carries the output schema.

Source: http://allanrbo.blogspot.com/2026/09/a-jev-like-wrapper-for-llms-including.html

Calling the AI bluff: Adding “Do not guess” cut made-up claims from 71% to 20%

This is an empirical benchmark measuring LLM hallucination rates on factual lookup tasks, with a specific intervention: appending the instruction “Do not guess. If you do not know, say so.” to the system prompt. Across the benchmark’s test set, models without the instruction fabricated answers at a 71% rate; with it, the rate dropped to 20%. The benchmark is a set of questions with objectively verifiable answers where the ground truth is known. The 51 percentage-point reduction is large enough to be practically significant, though the residual 20% rate means the intervention is not a solution. Mechanistically, this aligns with what is known about instruction-following in RLHF-tuned models: the models have latent uncertainty signals but suppress them by default in favor of a confident-sounding answer, because training data and RLHF reward modeling both reward fluent, complete responses. An explicit permission to abstain unlocks a different output distribution. The post is careful to note this is not a controlled experiment across all models or all task types, and the 71% baseline is specific to their question set, which may be adversarially selected for obscure facts. Open questions include whether the effect degrades with longer context (where the instruction may be down-weighted), how it interacts with chain-of-thought prompting, and whether calibration (expressing uncertainty with probabilities rather than binary abstain/answer) would be more useful than abstention.

Source: https://earnanhonestdollar.com/bench

Jeeves: Reasoning improves Jev-like decision models

PostHog’s Jeeves applies the Jev pattern (structured LLM function call returning typed decisions) to product analytics rules — specifically, to decisions like “should this feature flag be enabled for this user” or “which experiment variant should this session receive.” The key addition over plain structured output calls is an explicit reasoning step: the model first produces a chain-of-thought trace, then commits to a typed decision. The repository shows that interposing this reasoning step measurably improves decision accuracy on their internal benchmark versus direct structured output. The architecture is a thin Python layer over a reasoning-capable model (o-series or equivalent). The decision schema is defined as a Pydantic model; the system prompt encodes the relevant business rules in natural language; the model reasons through them before outputting the structured result. The reasoning trace is also logged, which gives a human-readable audit trail for why a particular flag or variant was assigned — useful for debugging and compliance. The engineering tradeoff is latency and cost: reasoning models are slower and more expensive per call, which matters if decisions are made at request time in a hot path. The repo addresses this by noting that many such decisions can be made offline or cached per user-segment rather than per-request. The interesting design question this raises is when a rule engine (deterministic, fast, auditable) should be replaced by or augmented with an LLM-based decision model, and whether the reasoning trace is a net improvement over explicit rule authoring.

Source: https://github.com/PostHog/jeeves

Sonnet 5.5

Anthropic’s Claude Sonnet 5.5 is a mid-tier production model release, positioned between Haiku (fast/cheap) and Opus (high-capability). The primary technical claims are improved coding and agentic task performance relative to Sonnet 5, with particular emphasis on multi-step tool use and longer-horizon task completion. The model uses Anthropic’s extended thinking feature, which exposes an intermediate reasoning chain before the final response, similar to OpenAI’s o-series approach. Context window is 200K tokens. The HN thread’s high score (833) and comment volume (563) reflect significant practitioner interest — this is a widely deployed model class. Discussion in the thread centers on real benchmark comparisons against GPT-4.1 and Gemini 2.5 Pro on SWE-bench and similar coding evals, with mixed reports depending on task type. The agentic improvements are the most commercially relevant: Sonnet 5.5 reportedly maintains coherence over longer tool-use chains with fewer spurious tool calls and better error recovery. Anthropic has not published full technical details of the architecture or training procedure, which is standard for their releases. The cost is positioned between the Haiku and Opus tiers. From a research standpoint, the interesting signal here is the continued convergence of “hybrid” reasoning models — combining fast direct-response mode with slower explicit reasoning — as the dominant deployment pattern across all major labs, suggesting this is stabilizing as an architectural standard rather than a temporary differentiator.

Source: https://www.anthropic.com/claude-sonnet-5-5

MicroLLM Lab: Try 7 tiny LLMs in the browser

This demo runs seven sub-1B parameter language models entirely client-side using WebAssembly and WebGPU. The technical stack is llama.cpp compiled to WASM for CPU inference, with a WebGPU path for supported browsers enabling GPU shader execution without a server round-trip. Models range from ~135M to ~500M parameters and are quantized to 4-bit (GGUF format), keeping download sizes in the 80–300 MB range — feasible for browser delivery over a fast connection. Each model is fetched from a CDN and cached in IndexedDB on first load, so subsequent sessions avoid re-downloading. The WASM build of llama.cpp exposes a simple JS API: load model, set prompt, stream tokens. The WebGPU path uses compute shaders for matrix-vector multiply, which is the dominant operation in autoregressive decoding; on a modern discrete GPU this yields roughly 10-30 tokens/sec for these model sizes, compared to 2-8 tokens/sec on CPU via WASM. The demo is notable as a benchmark of what the browser runtime can now sustain: 4-bit quantization plus WebGPU brings small-model inference into interactive territory without any server infrastructure. Limitations are real: cold-start latency (model download + WASM compilation) is 10–60 seconds; WebGPU is not available in Firefox stable or Safari without flags; and these model sizes are too small for most practical tasks. The engineering interest is in the toolchain — demonstrating that the llama.cpp → WASM → WebGPU pipeline is production-plausible for edge/offline scenarios.

Source: https://stateofutopia.com/experiments/microllmlab/

What would a serious AI product look like?

Glyph’s post is a systems-design argument, not a benchmark or release. The core claim is that current AI product integrations fail the standard engineering criteria applied to any other infrastructure dependency: they lack deterministic behavior, defined failure modes, testable contracts, and composable interfaces. The post draws on the author’s background in Python infrastructure (Twisted, various async libraries) to argue that “serious” software means software whose behavior can be reasoned about, tested, and debugged systematically — and that wrapping an LLM API call in a function and shipping it does not meet that bar. Specific technical points: LLM outputs are not functions in the mathematical sense (same input, different output across calls); retry-on-failure semantics are undefined when the failure mode is “wrong answer” rather than a network error; evaluation is conflated with testing but requires different tooling; and the operational cost of debugging a production failure is much higher when the component is a stochastic model than when it is a deterministic service. The proposed direction is not to avoid LLMs but to treat them as probabilistic components with explicit contracts — define the output distribution you require, measure conformance to it, and build the system to degrade gracefully when conformance fails. This maps onto ideas from robust statistics and fault-tolerant systems design more than from typical ML deployment literature. The post will resonate with engineers who have maintained production systems and are skeptical of the “just prompt it” approach, and less so with researchers focused on capability improvement rather than reliability engineering.

Source: https://blog.glyph.im/2026/09/serious-ai-product.html

Noteworthy New Repositories

Human-Agent-Society/reef

Infrastructure for continually self-improving agents. Reef provides a runtime scaffold in which agents can propose, execute, and evaluate modifications to their own tooling, memory, and policy prompts — closing the loop between task performance and capability update. The architecture separates concerns into three layers: an environment registry (tasks, tools, sandboxed execution), an agent core (LLM backbone + structured action space), and a self-improvement controller that tracks performance deltas and gates updates behind regression checks. Agents log structured traces; the controller runs automatic evaluation suites before committing any change, preventing silent capability degradation. The codebase is Python-first, uses a plugin interface for swapping LLM backends, and ships a Docker-based sandbox to isolate side effects during self-modification cycles. Compared to simple ReAct or tool-use frameworks, Reef’s distinguishing feature is the versioned capability store: every self-modification is a diff against a known-good checkpoint, making rollback deterministic. Practical use cases include automated benchmark climbing, iterative prompt engineering pipelines, and research on open-ended agent learning. The 7k-star traction suggests strong interest from the agent-systems community, though the maturity of the self-improvement loop relative to production workloads is worth scrutinizing before deployment.

Source: https://github.com/Human-Agent-Society/reef

Heman10x-NGU/openJev-verdict-2.0

A 151M-parameter non-autoregressive decision engine targeting calibrated binary and multi-class verdicts — the kind of structured outputs common in typed-decision pipelines on platforms like LocalLLaMA. The model architecture decouples encoding from output generation, emitting full decision vectors in a single forward pass rather than token-by-token, which gives deterministic latency independent of output length. Reported numbers: 77.10% accuracy, Brier score 0.0636, ECE 0.0144 — the ECE figure in particular indicates strong probability calibration relative to typical autoregressive models fine-tuned for classification. The repo claims to beat TypeSafe Jev and Laya on the typed-decisions benchmark. At 151M parameters, the model sits in a sweet spot: small enough for CPU inference or a single consumer GPU, large enough to capture non-trivial semantic structure. The non-autoregressive design means calibration is baked into a single softmax head rather than accumulated across a decoding chain, which simplifies temperature scaling post-hoc. Weights, training config, and evaluation scripts are included, making this a concrete drop-in for applications that need fast, calibrated verdicts — content moderation, routing, intent classification — without paying the latency cost of a full generative model. The main open question is out-of-distribution calibration behavior.

Source: https://github.com/Heman10x-NGU/openJev-verdict-2.0

emma-sue/AegisOps

An intelligent AIOps platform for enterprise IT operations, built around LangGraph and a deep-agent architecture with MCP (Model Context Protocol) for tool invocation. AegisOps covers three operational loops: automated inspection (scheduled health checks against configurable asset registries), agent-driven fault diagnosis (a graph-structured reasoning chain that gathers evidence, forms hypotheses, and ranks root causes), and a ticket/approval workflow that produces auditable evidence bundles for human review before remediation is authorized. The LangGraph backbone means control flow between agent nodes is explicit and inspectable — a meaningful engineering choice for ops tooling where black-box decision chains are unacceptable in production. The MCP layer abstracts tool calls (SSH, API queries, log pulls) behind a uniform interface, making it straightforward to add new data sources. The repo includes reproducible test records against real infrastructure scenarios, which is comparatively rare for AIOps demos and raises the credibility of the fault-diagnosis claims. The Chinese-language documentation targets enterprise IT teams in that market, but the architecture is general. Key open question: how the agent handles ambiguous or conflicting evidence in the diagnosis chain without human guidance, and how approval workflows scale when alert volumes spike.

Source: https://github.com/emma-sue/AegisOps

surya-koritala/Glyd

Glyd claims lossless compression of AI model weights achieving 33% GPU memory reduction with bit-for-bit identical numerical output — meaning inference results are exactly reproducible, not approximately equivalent. The mechanism is entropy coding of weight tensors: weights are stored in a compressed representation on disk and in host memory, then decompressed on-the-fly into GPU registers or L2 cache rather than being held fully materialized in VRAM. This is distinct from quantization (which changes values) and from mixed-precision (which changes compute semantics). The practical target is models that are slightly too large to fit a consumer GPU’s VRAM budget without sacrificing fidelity. Implementation appears to wrap standard PyTorch loading and module registration, requiring minimal changes to existing inference code. The 33% figure will depend heavily on weight distribution and model architecture — models with more structured or low-entropy weight tensors compress better. Throughput overhead from decompression is the main engineering concern; whether Glyd amortizes this cost through prefetch scheduling or batched decompression is worth examining in the codebase. For researchers running ablations that require exact numerical reproducibility while working under VRAM constraints, this is a more principled option than naive offloading.

Source: https://github.com/surya-koritala/Glyd

wfzyx/von

Von is an open-source non-autoregressive decision model positioned as a local, sub-15ms alternative to TypeSafe Jev for System One decision tasks — fast, heuristic-style judgments where latency matters more than extended chain-of-thought reasoning. The sub-15ms figure is for CPU inference, which is the key differentiator: the model is designed to run in-process without a GPU, making it embeddable in web servers, CLI tools, and edge applications. Non-autoregressive generation means the entire output is produced in a single forward pass, eliminating the O(n) sequential dependency of standard decoder-only models and giving constant-time output regardless of decision complexity. The “System One” framing is apt — this is not a model for multi-step reasoning but for high-throughput, low-latency classification and routing. Drop-in compatibility with the TypeSafe Jev interface means existing integrations require minimal refactoring. At 766 stars, it has gained traction among developers building latency-sensitive pipelines. The open questions are weight size and training data provenance — for a model intended as infrastructure-level tooling, reproducible training and documented data sourcing matter significantly for trust and compliance.

Source: https://github.com/wfzyx/von

XiantingWu/PQCensus

PQCensus is a static analysis tool for post-quantum cryptography migration: it scans software repositories, inventories all cryptographic primitives in use, and produces structured reports identifying algorithms vulnerable to quantum attacks (RSA, ECDH, ECDSA, etc.) alongside remediation guidance pointing toward NIST PQC standards (ML-KEM, ML-DSA, SLH-DSA). The analysis is entirely local — no data leaves the machine — and has zero mandatory runtime dependencies beyond the core scanner, which matters for security-sensitive codebases and air-gapped environments. Output formats are SARIF (integrates with GitHub Advanced Security, VS Code, and most CI pipelines) and CycloneDX (standard SBOM format), making it composable with existing vulnerability management workflows. The evidence-grounded approach means each finding cites the specific code location and the cryptographic call graph path, not just a file-level flag. This is meaningfully more useful than grep-based inventory tools. As NIST finalizes PQC standards and regulatory pressure for quantum-readiness grows (e.g., CISA guidance, NSM-10), automated inventory becomes a prerequisite for migration planning. The tool targets both greenfield audits and ongoing CI enforcement. Main limitation: static analysis will miss dynamically constructed algorithm strings and runtime-loaded crypto libraries.

Source: https://github.com/XiantingWu/PQCensus

ethanplusai/astra-flash-orchestrator

A code-generation orchestration layer that splits planning and building across two models: Astra handles task decomposition, phased planning, and post-hoc review; DeepSeek Flash (a fast, cost-efficient model) handles actual code synthesis within each phase. This mirrors how human engineering teams separate architecture/review roles from implementation, and avoids wasting a capable planning model on boilerplate generation. The workflow is structured around the OpenAI Codex API but the architecture is model-agnostic at the planner level. Key engineering properties: tasks are broken into discrete phases with verification checkpoints between them, installations of generated dependencies are sandboxed and validated before being committed, and all setup operations are reversible — meaning failed or unwanted generation runs can be cleanly rolled back rather than leaving the working directory in a dirty state. The reversibility mechanism is the most operationally interesting feature; it suggests either snapshotting or a transaction log over filesystem operations. For developers running autonomous coding agents against real codebases, the combination of phased execution and rollback reduces the blast radius of bad generations. The 709-star count reflects genuine interest in production-safe agentic coding workflows beyond one-shot generation demos.

Source: https://github.com/ethanplusai/astra-flash-orchestrator