Daily AI Digest — 2026-09-28
arXiv Highlights
Block Sparse Attention with Log-Linear Complexity
Problem
Self-attention scales as O(N^2) in sequence length N, and while block-sparse attention (e.g., Native Sparse Attention, MoBA, Quest) reduces the compute of the attention product itself, the selection of which key blocks to route each query to remains the asymptotic bottleneck. Standard practice computes a score s_{ij} = q_i \cdot \bar{k}_j for every query i and every candidate block j, then applies Top-K. Even if the retained set is sparse, scoring is dense: O(N \cdot N/B) = O(N^2/B). For N in the hundreds of thousands this scoring pass dominates wall-clock, especially during prefill. PISA (Pyramid Sparse Attention) targets this term directly, aiming for O(N \log N) end-to-end while preserving retrieval quality.
Method
PISA replaces the flat Top-K over N/B blocks with a hierarchical Top-K over a pyramid of pooled keys. Given keys K \in \mathbb{R}^{N \times d}, construct L = \lceil \log_r (N/B) \rceil levels by repeated pooling with factor r (so level \ell has N/(B r^\ell) super-blocks). Selection proceeds coarse-to-fine:
- At the coarsest level, score the query against all super-blocks (cheap because there are few).
- Retain the top k super-blocks.
- Expand each retained super-block into its r children at the next finer level, giving k \cdot r candidates.
- Score, retain top k, and recurse until the finest (block) level.
Because k is bounded and independent of N, each query performs O(k r \log N) scoring operations, yielding O(N k r \log N) = O(N \log N) total.
Scoring at each level is done with LogSumExp rather than max or mean pooling of dot products:
s^{(\ell)}_{ij} = \log \sum_{k \in \text{block}_j^{(\ell)}} \exp(q_i \cdot k / \sqrt{d}).
LogSumExp is the exact contribution a block would make to the softmax normalizer, so it is a principled upper bound on aggregate relevance and is monotone under refinement: a super-block’s LSE dominates any child’s LSE. This makes hierarchical pruning admissible in expectation — a discarded super-block cannot contain a child whose softmax mass exceeds the retained set by more than a bounded slack. In practice PISA uses the LSE of pooled key representations (not exact LSE over all children), trading a small approximation for the ability to precompute pooled keys once per layer.
Implementation
The authors provide fused Triton kernels for both prefill and decode. The critical engineering point: the query–key score matrix is never materialized. At each level the kernel loads the query tile, streams the (small) candidate key tile, computes LSE scores in registers, performs an on-chip Top-K, and emits indices to the next level. Only at the finest level are the selected K, V tiles gathered for the standard FlashAttention-style output computation. Backward passes similarly fuse routing gradients; because Top-K is non-differentiable, gradients flow only through the retained blocks (straight-through on selection), which is standard for this family.
Pooled key pyramids are constructed once per prefill and cached alongside the KV cache. During decode, only the newly appended block requires pyramid updates, which is O(\log N) per token.
Results
The abstract reports evaluation on long-context language modeling and retrieval benchmarks. PISA matches full attention on perplexity and long-context recall while achieving substantially higher throughput than both dense FlashAttention and prior block-sparse baselines (NSA, MoBA) at N \geq 64K. The advantage grows with N: at short contexts the constant factor of hierarchical routing is comparable to flat Top-K, but at N = 128K and beyond, the O(N^2/B) selection cost of flat methods overtakes the O(N \log N) pyramid. Retrieval-style tasks (needle-in-haystack, multi-document QA) show that LSE scoring is important: replacing it with mean or max pooling degrades recall noticeably, confirming that softmax-consistent aggregation is the right criterion for hierarchical pruning.
Limitations and open questions
- Hierarchical Top-K is greedy: a block whose pooled LSE is low but whose single best key would be selected under exact scoring can be pruned at a coarse level. The pooled-LSE monotonicity is exact only for true LSE over children, not for LSE of pooled representations; the practical approximation gap is bounded empirically but not analyzed.
- Selection remains non-differentiable; the training signal for routing comes entirely through retained blocks, which may cause pathological routing collapse on tasks with sharp, sparse attention patterns.
- The method assumes queries and keys have compatible geometry across pooling levels; strong positional encodings (RoPE at extreme frequencies) may distort pooled representations. The paper does not address whether pooling should be applied pre- or post-RoPE.
- Head-heterogeneity is not exploited: different heads likely need different k and r, but the reported configuration uses uniform hyperparameters.
- The abstract text is truncated; concrete speedup ratios and downstream accuracies must be read from the full paper.
Why this matters
If block-sparse attention is to replace dense attention at long context, the selection step itself must be sub-quadratic — otherwise sparsity only reduces a constant. PISA is the first practical demonstration that hierarchical, softmax-consistent routing achieves O(N \log N) end-to-end with fused kernels, closing the last quadratic term in the sparse-attention stack.
Source: https://arxiv.org/abs/2609.31093
TrackEverything: Long Horizon Dense Tracking via De-Duplicating 3D Scene Representations
Problem
Point tracking has been split into two regimes: sparse trackers (CoTracker3, SpatialTracker-v2, TAPIP-3D) that follow a user-specified set of queries across long horizons, and dense trackers (VDPM, D4RT, Any4D, DeltaV2) that follow every pixel but only over short clips (48–64 frames). The bottleneck is that per-frame representations scale linearly in T: every new frame re-instantiates surfaces that were already observed. For a 1000-frame video of a mostly static scene, this is nearly all wasted state. TrackEverything reframes tracking so that model state scales with the amount of unique physical scene geometry rather than with T.
Method
Input: RGB video \mathbf{I}\in\mathbb{R}^{T\times H\times W\times 3} and per-frame world-coordinate pointmaps \mathbf{P}\in\mathbb{R}^{T\times H\times W\times 3} (from sensors or VGGT-\Omega). Output: dense 3D trajectories \mathbf{X}\in\mathbb{R}^{N\times T\times 3}, visibility \mathbf{V}\in\mathbb{R}^{N\times T}, and per-track static/dynamic logits \mathbf{S}\in\mathbb{R}^{N}.
Encoding into a 3D feature cloud. The video is chunked into non-overlapping windows of L=16 frames. Each frame passes through a frozen DINOv3-small backbone plus a trainable ViT-Adapter head, producing (L, H/4, W/4, D) features with D=384. Tokens are paired with their world-coordinate 3D positions from the downsampled pointmap (canonicalized to \mathrm{cam}_0). Within each frame, tokens are quantized by \lfloor \mathbf{x}/v \rfloor and mean-pooled per voxel to reduce intra-frame redundancy. Crucially, this voxelization is per-frame, not across frames — otherwise dynamic surfaces passing through a voxel at different times would be merged incorrectly. Concatenating across the L frames yields a space-time feature cloud of shape (N, D).
Endpoint refiner + trajectory refiner. The core insight is that most points in most windows are static, so predicting a dense within-window trajectory for every point is wasteful. A recurrent transformer (the endpoint refiner) iteratively predicts each track’s end-of-window 3D location and a static/dynamic classification. A lightweight trajectory refiner then decodes dense within-window paths only for tracks flagged dynamic; static tracks get their trajectory filled by copying their endpoint.
3D WAFT. The paper replaces the 4D correlation volumes standard in 2D tracking (e.g., WAFT-style all-pairs correlations) with feature sampling directly in the 3D scene cloud around each track’s current 3D estimate. This avoids the O(HW \cdot HW) memory cost per frame and instead scales with local neighborhood queries in the point cloud.
Cross-window voxel de-duplication. After each window, tracks arriving at the same 3D voxel are merged, since two physical entities cannot share a voxel at one instant. This is the mechanism that decouples state size from T: static scene content contributes one representative regardless of how many windows observe it. Only dynamic tracks (a small fraction in most scenes) and genuinely new surfaces cause the representation to grow.
Results
Evaluation is on the TAPVid-3D minival split (50 clips each from ADT, DriveTrack, PStudio) using APD-P (depth-relative pixel thresholds \{1,2,4,8,16\} px unprojected via intrinsics) and APD-M (metric thresholds \{0.1, 0.3, 0.5, 1.0\} m). Baselines cover sparse (SpatialTracker-v2, TAPIP-3D, CoTracker3), first-frame dense (DeltaV2, Any4D), and all-frame dense (VDPM, D4RT). VGGT-\Omega pointmaps are supplied uniformly where accepted. All methods are run on 48-frame clips; sparse trackers, DeltaV2, and TrackEverything are additionally run on full-length sequences (up to ~300 frames on ADT).
The qualitative comparison illustrates a structural advantage: first-frame dense trackers (Any4D, DeltaV2’s dense branch) cannot track surfaces that first appear after frame 0, while sparse trackers require query specification and become prohibitively expensive when queries are densified over later frames. TrackEverything handles both because new surfaces enter the persistent scene cloud whenever they are observed, and merging is voxel-based rather than trajectory-based. Dense-tracking numbers on PointOdyssey and Dynamic Replica appear in the supplementary Table 3; D4RT is included for reference only, as it uses roughly 20\times the parameter count with private training data and evaluates only 48-frame clips.
Limitations and open questions
The system inherits pointmap quality from an external predictor (VGGT-\Omega) or sensor; degradation in that geometry propagates directly to track quality, since the entire representation lives in world coordinates. Voxel de-duplication assumes reliable static/dynamic separation — misclassifying a slow-moving surface as static will collapse it into a single voxel and lose its motion. The choice of voxel size v trades memory against spatial resolution, and no adaptive strategy is described. Windowing at L=16 means motion coherence across window boundaries relies on endpoint hand-off through the merged voxel state; long-range temporal reasoning within a single trajectory is not directly modeled. Finally, dense 3D ground truth is unavailable for real dynamic scenes, so quantitative evaluation is on sparse annotated points only — full-scene fidelity is validated only qualitatively and in simulation.
Why this matters
TrackEverything demonstrates that a 3D-native, de-duplicated scene representation lets dense point tracking scale to 1000+ frame videos without proportional memory growth — the model’s state scales with unique geometry, not clip length. This is the right factorization for downstream world-model, 4D reconstruction, and manipulation pipelines that need persistent per-surface tracks over long horizons.
Source: https://arxiv.org/abs/2609.30222
FuseReg: Regularizing Layer Fusion Mitigates the Reconstruction-Generation Gap in Representation Autoencoders
Problem
Representation autoencoders (RAEs) tie image generation to a frozen visual encoder by using its intermediate features as both the reconstruction latent for a pixel decoder and the target latent for a diffusion transformer. The design choice that has been treated as a hyperparameter is which encoder layers to fuse into that shared latent. Shallower layers retain low-level pixel detail useful for reconstruction; deeper layers carry semantics that make diffusion easier. A single fixed fusion (single-layer, prefix sum, or a learned global gate) forces the decoder and generator to agree on one point of that trade-off, and the authors show that learned global gates degenerate to a single-layer shortcut. This coupling is the source of the reconstruction–generation gap that FuseReg targets.
Method
Let E be a frozen encoder with K candidate layers (K=23 for DINOv3-L). Denote layer-normalized token maps h_k = \mathrm{LN}(E_{l_k}(x)) \in \mathbb{R}^{N \times d}. Any fixed fusion can be written as
z_a = \sum_{k=1}^K a_k h_k, \qquad \mathbf{1}^\top a = 1,
where fixed subsets, prefix sums, and learned global gates differ only in how a is set. FuseReg replaces the fixed weight vector with a distribution over nonempty subsets S \subseteq \{1,\dots,K\}, each layer retained independently with probability 1-p, and forms
z_S = \frac{1}{|S|}\sum_{k \in S} h_k.
This normalized subset fusion is applied at both the pixel-decoder input and the DiT target latent during training.
The core statistical property is Lemma 1: for every input,
\mathbb{E}_S[z_S \mid x] = \bar{h}, \quad \mathbb{E}_S[(z_S - \bar{h})(z_S - \bar{h})^\top \mid x] = c_K(p)\,V(x),
with \bar{h} = K^{-1}\sum_k h_k, V(x) = K^{-1}\sum_k (h_k - \bar h)(h_k - \bar h)^\top, and c_K(p) = \mathbb{E}[(K-R)/(R(K-1))] under R \sim \mathrm{Binomial}(K, 1-p) conditioned on R>0. The subset-averaged latent is mean-preserving with respect to the all-layer fusion, and the induced covariance lives exactly in the subspace of cross-layer disagreement. In a linear squared-loss surrogate, this is equivalent to a Tikhonov penalty proportional to c_K(p)\,\mathrm{tr}(W^\top W \Gamma) on any downstream linear head W, i.e., the head is regularized to be insensitive to directions where encoder layers disagree, without shifting its behavior on the shared subspace.
Concretely, to reimplement: (i) freeze E; (ii) at each training step sample a Bernoulli mask over the K candidate layers with retention 1-p, resample if empty; (iii) build z_S by mean pooling the selected LN’d layer tokens; (iv) feed z_S to both the ViT pixel decoder (trained from scratch) and, in the generation stage, use z_S as the DiT target for flow/diffusion loss. Architectures, param counts, and optimizer settings match RAEv2 baselines.
Results
The setup uses DINOv3-L with K=23, ImageNet-256, a ViT decoder trained from scratch, DiT-Base/XL generators with 50 Euler steps, and the standard 50k-sample FID/IS protocol.
Decoder flexibility: a single FuseReg decoder handles the full (K=23), sparse (K=7), and single-layer (\ell_{11}) fusions without retraining and achieves higher PSNR than decoders specialized to each fixed fusion. Baselines trained on one fusion degrade sharply when fed another; the FuseReg decoder does not.
The cosine-similarity maps at an intermediate decoder block show that FuseReg maintains coherent, object-aligned neighborhoods for the same query patch across K=23, K=7, and \ell_{11} inputs, whereas the fixed-fusion RAEv2 decoder produces qualitatively different similarity structures when the fusion changes.
Generation: swapping only the pixel decoder for the FuseReg decoder — leaving the RAEv2 DiT-XL generator and its sampled latents unchanged — reduces unguided gFID by 27%. The regularization principle applied jointly to the DiT stage yields further gains (extended in later sections/tables of the paper).
Limitations and open questions
The theory that identifies c_K(p)\Gamma as the induced regularizer is derived under a linear squared-loss surrogate; the transfer of these guarantees to nonlinear decoders and flow-matching generators is empirical. The independent-Bernoulli mask does not model correlated layer redundancy, and the optimal p (and whether it should differ between decoder and generator stages) is not characterized analytically. Reported numbers are on ImageNet-256 with DINOv3-L; interactions with alternative encoders (SigLIP, CLIP, MAE), higher resolutions, and text-conditioned generation are not evaluated here. Finally, “27% unguided gFID reduction” from decoder replacement leaves open how much of the gap between reconstruction fidelity and generation quality remains attributable to the DiT rather than the latent interface.
Why this matters
FuseReg reframes an ad hoc architectural choice — which encoder layers form the RAE latent — as a distributional training objective whose regularization effect is characterizable in closed form. That a decoder swap alone drops unguided gFID by 27% on DiT-XL suggests the reconstruction–generation gap in RAEs is largely an artifact of overfitting the pixel decoder to one layer composition, not a fundamental capacity limit of the generator.
Source: https://arxiv.org/abs/2609.31620
InternW0-Δ: A World Action Model Bridging Predictive Dynamics and Actions with 20K+ Hours of Open Data
Problem
World Action Models (WAMs) aim to jointly model visual dynamics and action generation for generalist manipulation. The recurring difficulty is fusing heterogeneous priors — pretrained video generators, VLM semantics, 4D geometry/motion, and low-level control — into a single policy that generates actions at inference without requiring an expensive video rollout, and that trains cleanly across robots with different morphologies, DoF, and control conventions. InternW0-Δ addresses both the architectural fusion problem and the data unification problem, pretraining on a 20K+ hour heterogeneous corpus of robot demonstrations, egocentric video, ego-to-robot transfer data, and UMI hand-collected trajectories.
Architecture
InternW0-Δ couples a pretrained Wan2.2-TI2V-5B video expert with a randomly initialized action expert (ActionDiT) through 30 directed Mixture-of-Transformers (MoT) blocks. A frozen Wan VAE encodes a sparse visual memory of three observation types — an anchor frame A from the episode start, a recent frame R (lagging the current frame by 32 action steps), and the current frame C. T5 instruction embeddings condition the video expert, while a frozen VLM supplies task-conditioned scene semantics to the action expert. Proprioception is projected independently into both streams.
Each MoT block maintains separate query/key/value projections for the video and action streams, performs joint masked self-attention, then routes through expert-specific cross-attention and FFNs.
The critical property is directed information flow: forward activations from realized future observations F never reach the action prediction path. This lets the model use future-video supervision during training but skip any future rollout at inference — a substantial latency win over WAMs that generate video first and read off actions.
Causal Imprint and 4D distillation
Two auxiliary mechanisms inject predictive and geometric priors without inference-time cost.
Causal Imprint introduces \Delta tokens that summarize future-relevant scene changes from R and C. Because future frames only supervise these tokens during training (through the directed MoT mask), the action expert receives change-oriented predictive features at inference without generating future video.
4D-aware distillation uses a query-based student branch that aggregates clean A/R/C features from the video expert and regresses them onto cached descriptors from a frozen Track4World teacher via MSE:
\mathcal{L}_{4\mathrm{D}} = \| f_\theta(z_A, z_R, z_C) - \phi_{\text{Track4World}}(x_A, x_R, x_C) \|_2^2.
Distillation is applied to only one sample per per-GPU mini-batch of 16, keeping compute overhead marginal.
Unified 80-D action space
To mix demonstrations across arms, dexterous hands, mobile bases, and heads, all trajectories are mapped into a canonical 80-dimensional action vector with fixed semantic slots (Table 1 in the paper): two 7-DoF arm joint blocks (indices [0,7) and [40,47)), two 9-D EEF pose blocks, per-side gripper (1) and hand (12) slots, torso (5), lift (1), head (3), and mobile base (3), with reserved slots for future embodiments. EEF state uses a 3D position plus continuous 6D rotation; EEF actions use a 3D translation increment and 3D local-frame rotation vector. Joint actions are absolute target positions. This eliminates the ambiguity that arises from mixing native action layouts across sources.
Training
Pretraining data mix: 80% real robot demonstrations, 10% Ego2Robot, 8% UMI, 2% egocentric. Streams are weighted by valid start frames with trajectory-level balancing so that long segments do not dominate. Samples contain 33 video frames and a 32-step action block at a 4:1 video-to-action frequency ratio, packed into a 384 \times 256 canvas. Pretraining runs in two phases: standard joint training, then continued pretraining with the 4D distillation objective enabled. Post-training specializes checkpoints to target embodiments across four simulators (LIBERO-Plus, RoboTwin 2.0, EBench, RoboDojo) using each benchmark’s official protocol.
Infrastructure
The infrastructure section is unusually candid about bottlenecks. The frozen VAE and VLM encoders dominate cost during exploratory training because samples are revisited across epochs. A Cache Manager (built on LiteGen) serves VAE latents z_i = E_{\text{VAE}}(x_i) and VLM contexts h_i = E_{\text{VLM}}(x_i, p_i, s_i) via a keyed lookup
\text{key} = \text{hash}(D, e, k, v, P, M)
over dataset version, episode, temporal chunk, view, and preprocessing/model fingerprints. Both online (lazy, with async persistence) and offline (batched pre-encoding with manifest validation) materialization are supported. Metadata validation prevents silent reuse across incompatible preprocessing or model versions. The MoT backbone is separately handled by layerwise compilation and activation checkpointing.
Results and limitations
The paper claims InternW0-Δ outperforms prior methods across the four simulation benchmarks and on real-robot platforms, with results taken from each benchmark’s published leaderboard under matching protocols. The provided sections do not include per-benchmark success rates; the abstract and experiments section refer to “outperforms” without listing specific deltas in the excerpt available here. Readers should treat the quantitative claims as pending full-table inspection.
Open questions: (i) the marginal contribution of Causal Imprint vs. 4D distillation is not isolated in the excerpt; (ii) applying 4D distillation to only 1/16 samples per batch is a strong design choice whose sensitivity is not characterized; (iii) the 80-D canonical action layout reserves only 10 slots for future embodiments, which may constrain scaling to whole-body humanoid control; (iv) the sparse anchor/recent/current memory sacrifices dense temporal context that dense-video WAMs retain — the tradeoff against action fidelity for long-horizon tasks is not quantified here.
Why this matters
InternW0-Δ is a concrete blueprint for training-only future supervision: it demonstrates that predictive video dynamics and 4D geometric priors can be distilled into an action expert rather than run alongside it, eliminating video-rollout latency at inference. Combined with a unified 80-D cross-embodiment action space and a 20K+ hour open-data recipe, it establishes a reproducible baseline for the next generation of world-action models.
Source: https://arxiv.org/abs/2609.31394
Tactile-JEPA: Topology-Aware Self-Supervised Representation Learning for Distributed Tactile Sensors
Problem
Vision-based tactile sensors (GelSight, Digit) produce dense image streams and inherit the full stack of vision SSL methods — MAE, DINO, I-JEPA. Distributed electronic skins do not: their output is a multivariate time series over a sparse, irregular set of taxels covering the body of the robot. Prior work has largely ignored this geometry, either training tactile encoders from scratch on raw signals or treating the taxel array as a pseudo-image. Neither respects the fact that taxel adjacency on the sensing surface — not row/column adjacency in a tensor — is what encodes locality of contact. The paper asks how to pre-train a general-purpose encoder for e-skins that exploits taxel topology without requiring taxel positions at deployment time.
Method
Setup. A sensor with N taxels sampled at f_s produces X \in \mathbb{R}^{N \times T \times m}, with m=1 for pressure sensors and m=3 for magnetic skins. Pre-training operates on short windows X_\tau^{(k)} \in \mathbb{R}^{N \times \tau \times m} where \tau = \lfloor T_w f_s \rfloor, capturing a short contact history within a single event. The layout (a taxel connectivity graph G) is required at pre-training but not downstream.
The architecture follows I-JEPA: a context encoder \mathbf{E}_\theta, a predictor \mathbf{P}_\phi, and an EMA target encoder \bar{\mathbf{E}}_{\bar\theta}. Per-taxel embeddings for masked (target) taxels are predicted from the unmasked (context) taxels, and matched to target-encoder outputs by MSE:
\mathcal{L} = \frac{1}{|M|}\sum_{i \in M} \| \mathbf{P}_\phi(\mathbf{E}_\theta(X_{\text{ctx}}))_i - \bar{\mathbf{E}}_{\bar\theta}(X)_i \|_2^2.
The novelty is the mask sampler.
- Masks are drawn over the taxel graph G, not a pixel grid. A “local” mask is a connected subgraph around a seed taxel, obtained by BFS expansion until a target size is reached. A “global” mask samples a set of taxels spread across the surface (e.g., by farthest-point sampling on G).
- Two scales are mixed. The default configuration samples 2 local + 2 global target masks (2L+2G) per window. Local masks force fine-grained modeling of the contact patch; global masks force representation of the whole-hand contact state.
Context is by default a global random subset of the remaining taxels (variant “†” replaces this with a local context mask). The target encoder is updated by EMA of the context encoder.
Downstream, \mathbf{E}_\theta is frozen and a lightweight head is trained per task. Because the objective is defined within one window, temporal integration across windows is deferred to the downstream decoder, and inference can run at the control loop rate.
Results
Evaluation spans three datasets covering different transduction principles and embodiments: Sparsh-skin (magnetic, sensorized hand), Tactile socks (piezoresistive, wearable), and DECO-50 (assembly subset, bimanual manipulation). Downstream tasks include force estimation, in-hand pose estimation, and behavior-cloning policy learning.
Headline numbers from the conclusion: relative to the strongest SSL baselines, Tactile-JEPA reduces in-hand orientation error by 20.8% and force estimation error by 6.3%. Beyond accuracy, the authors emphasize training stability: Tactile-JEPA pre-trains without collapse on all three sensor types, whereas competing SSL methods collapse on some datasets — an important operational property given how noisy raw e-skin signals are.
The masking ablation (Table IV) isolates why the design works. On Sparsh-skin force RMSE, 2L+2G reaches 14.05 \pm 0.73, beating 4I-JEPA at 14.99 \pm 1.03, 4L at 15.52, and 4G at 15.17. On in-hand x-accuracy, 2L+2G attains 95.74\% \pm 1.05 versus 93.50\% for 4I-JEPA. On DECO-50 policy RMSE, 4G is best (0.4600 \pm 0.0019) — consistent with action prediction depending on whole-hand contact state — while 2L+2G is a close second at 0.4635. Two takeaways:
- Graph-based masking helps independently of scale. 4L (compact, graph-connected local masks) beats 4I-JEPA (compact, grid-based block masks) at the same mask budget on force estimation and policy learning, isolating topology-awareness as the source of gain.
- Multi-scale targets provide the best general-purpose trade-off: 2L+2G is best or near-best across tasks, while single-scale variants overfit to a task type (4G for policy, 4L for local force detail).
The “†” variants (local context instead of global) show that although local context can help force estimation (13.28 \pm 0.83 for 2L+2G†), it substantially degrades pose accuracy (drops to 92.83\%) and policy learning, so global context remains the right default.
Limitations and open questions
- The layout graph is required for pre-training. The paper does not address unknown or noisy adjacency; sensor manufacturing or wear could perturb G.
- Temporal modeling across windows is offloaded to the downstream head; whether a temporally aware JEPA objective would further help contact-dynamics tasks is untested.
- Evaluation covers three datasets but only one task per dataset in some cases; cross-embodiment transfer (train on one skin, deploy on another with different N and G) is not systematically probed.
- Mask hyperparameters (sizes of local vs. global masks, number of masks) are ablated only coarsely; scaling behavior with N (hundreds vs. thousands of taxels) is not characterized.
- Comparison is to SSL baselines adapted from vision; comparison to GNN-based tactile encoders that use G architecturally rather than only for masking would sharpen the story.
Why this matters
Distributed e-skins are the practical route to whole-body tactile sensing, but the community has lacked a pre-training recipe that respects their irregular geometry. Tactile-JEPA shows that injecting topology only into the mask sampler — keeping the encoder layout-agnostic — is sufficient to obtain stable pre-training and consistent downstream gains across sensor types, which is exactly what is needed for a reusable tactile backbone in VLA-style pipelines.
Source: https://arxiv.org/abs/2609.24385
SLCA-GRPO: Resolving Cross-Segment Credit Misattribution in Tool-Calling RL
Problem
Tool-calling agents emit two structurally distinct token classes in the same trajectory: structured tool invocations (function names, argument JSON) that must satisfy executable schemas, and natural-language summaries that convey results to the user. Standard on-policy RL with group-relative advantage estimation (GRPO) computes a single scalar advantage per trajectory and broadcasts it uniformly across all tokens. This is a structural mismatch, not a hyperparameter issue: a good summary written on top of a wrong tool call still receives whatever terminal reward the trajectory earns, and the gradient contribution on tool-decision tokens is polluted by variance in summary quality (and vice versa). The authors formalize this as cross-segment credit misattribution: the summary-to-tool “support path” in the policy gradient is nonzero under unified advantages, so noise in one segment leaks into the other’s update.
The mechanism is illustrated cleanly by the paper’s contrast between unified broadcast and segment-locked routing: under GRPO, \hat{A}_{\text{traj}} is applied to every token, so tool-token gradients absorb summary-driven variance; under SLCA, tool tokens receive \hat{A}^{\text{tool}} and summary tokens receive \hat{A}^{\text{sum}}, with no cross-flow.
Method
SLCA-GRPO decomposes the problem into three requirements — expose segments, produce segment-specific feedback, block cross-segment advantage flow — and instantiates them without additional rollouts from intermediate states (i.e., without step-level MC estimation that would blow up compute).
Segmentation. A binary mask over each rollout partitions tokens into a tool segment \mathcal{T}_{\text{tool}} (structured invocation tokens) and a summary segment \mathcal{T}_{\text{sum}} (user-facing natural language). Segmentation is deterministic from the output schema.
Hierarchical Rewards (HierR). Feedback is split into two channels:
- R^{\text{tool}}: dense execution-level rewards, checking schema conformance, argument correctness, and successful tool return.
- R^{\text{sum}}: terminal outcome/preference reward over the final summary.
Segment-Locked Credit Assignment (SLCA). Within one group of rollouts, advantages are normalized independently per segment:
\hat{A}^{\text{tool}}_i = \frac{R^{\text{tool}}_i - \mu_{\text{tool}}}{\sigma_{\text{tool}}}, \quad \hat{A}^{\text{sum}}_i = \frac{R^{\text{sum}}_i - \mu_{\text{sum}}}{\sigma_{\text{sum}}}
Routing is then enforced by the mask: for token t in rollout i,
\hat{A}_{i,t} = \mathbb{1}[t \in \mathcal{T}_{\text{tool}}]\,\hat{A}^{\text{tool}}_i + \mathbb{1}[t \in \mathcal{T}_{\text{sum}}]\,\hat{A}^{\text{sum}}_i
The final objective is a standard PPO clipped surrogate with this segment-aware \hat{A}_{i,t}. Because normalization is per-segment, a rollout with a strong tool trace but weak summary contributes positive gradient only to its tool tokens; the summary-to-tool support path in the gradient vanishes.
Schema-Guided LLM Simulator (SGLS). Real tool APIs are expensive and rate-limited; RL requires many rollouts. SGLS is an LLM-driven simulator that consumes the tool schema and simulates responses consistent with argument types and semantics. This yields interleaved user/tool/assistant trajectories at scale for the RL loop.
Data. From Toucan-1.5M the pipeline yields 42,423 SFT and 31,818 RL samples (mix of single-turn and decomposed multi-turn), with a held-out 4,000-sample Toucan-Test.
Evaluation setup
Three axes: in-domain (Toucan-Test, 4k samples), generalization (BFCL V3, single-turn with relevance detection excluded for the main comparison; multi-turn accuracy reported in the appendix), and collaborative robustness (\tau^2-Bench Airline/Retail/Telecom, Pass@1). Ablations remove SLCA, SGLS, and HierR individually.
The excerpt provided does not contain the numerical results tables; the evaluation section defines the three-part protocol (failure diagnosis under unified/additive credit, cross-schema and long-horizon transfer, and component ablations) but numerical values are held in tables not included in the supplied sections. What is explicitly stated is the dataset scale (31,818 RL trajectories, 4,000 test) and the exclusion of relevance detection from the main BFCL V3 single-turn comparison, restricting that measurement to atomic schema generalization (e.g., AST accuracy).
Limitations and open questions
- Segment definition depends on parseable schemas. Free-form reasoning traces or agents that inline tool arguments into prose would need a learned segmenter; the current mask is deterministic.
- SGLS fidelity is a training-time assumption. Simulator LLM biases can be absorbed into the tool policy; the paper does not, in the provided excerpt, quantify sim-to-real gap on live APIs.
- Two-segment decomposition. Real agents have more strata (planning, reflection, tool, summary). Extending SLCA to k>2 segments is straightforward algebraically but raises variance concerns since per-segment normalization uses group-level statistics.
- Independent normalization changes the relative scale of gradients across segments. There is an implicit weighting that the paper does not appear to sweep in the excerpt; a temperature between \hat{A}^{\text{tool}} and \hat{A}^{\text{sum}} magnitudes may matter.
- Comparison to gradient-projection methods (e.g., PCGrad-style approaches) is deferred to an appendix schematic rather than empirical head-to-head.
Why this matters
Trajectory-level advantage broadcasting is the default in GRPO-family algorithms now widely used for agentic post-training, and it silently miscredits heterogeneous outputs. SLCA-GRPO shows that respecting the structural segmentation of the output — with per-segment reward channels and per-segment normalized advantages — is a cheap, rollout-free fix that closes an obvious gradient leakage path. The same pattern likely applies to any interleaved-output RL setting: code+explanation, plan+action, chain-of-thought+answer.
Source: https://arxiv.org/abs/2609.29050
TRACE: Temporal Audit and Condition-aware Evaluation of Streaming Video Understanding
Streaming video understanding benchmarks typically report aggregate QA accuracy or event-detection F1, treating the underlying evaluation as if it were static: a question, a video, an answer. TRACE argues this is fundamentally under-specified. In a streaming setting, two systems with identical QA scores can differ in when evidence was legally available, how much history they maintained, whether they self-triggered a response, and what workload they incurred to do so. The paper introduces a condition-aware benchmark and protocol that binds temporal validity, execution conditions, and operational outcomes into a single evaluation.
Design: temporal validity, execution conditions, operational outcomes
TRACE is organized around three tightly coupled premises. First, every answer must be tied to a video-grounded evidence time — the earliest moment at which the visible content supports the correct answer under the given question. For QA, annotators mark this alongside question-arrival time; the interval between them defines an evidence-to-question distance. The authors are careful to note this distance is not itself a difficulty metric, since repeated evidence and content complexity confound it. Second, proactive tasks are given instruction-dependent trigger semantics rather than a single generic event timestamp. TRACE distinguishes four trigger rules:
- Event-onset: first time the target condition holds.
- Event-completion: end of the requested action.
- Sufficient-evidence: earliest time visible clues uniquely determine the answer and rule out alternatives.
- State: an interval during which the answer remains valid.
These are properties of the instruction, not of the video: the same action may use an onset or completion trigger depending on how it is queried. Third, evaluation is executed under a unified causal Core–Adapter protocol. Core delivers timestamped RGB frames at 1 FPS with real-time pacing and controls question and instruction arrival; Adapter records what the model actually did — which frames were processed, when responses fired, workload consumed.
Benchmark construction
The standard set contains 1,240 records over 517 videos, drawn from the visual-only subsets of StreamingBench and OVO-Bench (audio, subtitles, and external knowledge tasks are excluded). Crucially, the annotations are re-audited rather than trusted. Of the reviewed records, 328 were edited and 52 excluded. Among the edits, 325 of 328 touched temporal or trigger fields: 100 of 103 QA edits revised timing, and all 225 proactive edits revised or added trigger annotations. On directly comparable timestamps, the median absolute revision was 3.0 s for QA and 2.24 s for proactive tasks, with a heavy upper tail. These revisions are not cosmetic — they determine whether visual information is legally available before a question, how response delays are measured, and whether an unassigned response is a false alarm or a target window is a miss.
Evaluation protocol and configurations
Eight systems are evaluated in eight configurations: AURA, MOSS-VL, MOSS-Preview, LiveCC, ThinkStream, VideoLLM-Online, MiniCPM-O 4.5, and JoyAI (as a full end-to-end system). Configurations differ along three explicit axes made visible in the paper’s Table 3: (i) visual state maintenance — native streaming session vs. prefix reconstruction at query time (as in AURA’s QA path) vs. per-frame KV persistence (VideoLLM-Online) vs. external summary memory (JoyAI); (ii) proactive trigger mode — autonomous self-initiation for all main-comparison runs, with JoyAI’s trigger mechanism listed as unverified autonomous; (iii) deployment boundary — model-plus-Adapter vs. system-level. All QA uses the same questions and answer parser; proactive semantic scoring uses a fixed Qwen3.5-35B-A3B judge. Runs use NVIDIA A100 80GB, and the authors are explicit that timing and workload numbers are configuration-specific, not hardware-normalized.
Reporting is multidimensional: answer quality, timeliness, response-selection behavior (which decouples proactive quality from firing decisions), workload (generation cost), completion at three separately-denominated granularities (QA-record, proactive-record, stream), and reliability. Completion at different granularities is deliberately not comparable across levels.
Findings
The analysis produces three claims that the numbers back. First, the temporal audit changes what “correct” and “on-time” mean: because trigger annotations were revised for all 225 edited proactive records, the same model output can flip between in-window and out-of-window under audited vs. original boundaries. Second, proactive performance does not lie on a single axis — quality, timing, and response-selection separate. Systems that produce high-quality answers when they respond may fire poorly, and vice versa; a single F1 collapses these. Third, near-identical QA accuracy can mask substantial differences in completion rate, answer validity, and generation workload. The abstract makes this concrete: “nearly identical QA accuracy can mask substantial differences in completion, answer validity, and generation workload.” AURA’s non-native prefix reconstruction at query time, for instance, is directly comparable in QA score to native streaming systems but incurs a very different workload profile.
Limitations and open questions
Several limitations are visible. The set is 1,240 records, modest for a benchmark intended to separate closely-matched systems along many axes; statistical power for per-dimension comparisons is unclear. Timing and workload are not hardware-normalized, which the authors acknowledge — comparisons across configurations mix model behavior with runtime environment. The proactive semantic judge is a single fixed LLM, and judge sensitivity is not characterized in the reported sections. Trigger annotation, especially for sufficient-evidence tasks, is inherently subjective; inter-annotator agreement on the audited timestamps is not reported in the excerpted text. Finally, JoyAI’s “unverified autonomous” trigger flag hints at the difficulty of enforcing the protocol on end-to-end systems whose internals are opaque.
Why this matters
If streaming-video benchmarks continue to report scalar task scores, they will keep producing rankings that do not reflect operational reality — systems with identical QA accuracy but very different workloads, false-alarm rates, and history requirements. TRACE’s contribution is procedural more than architectural: it forces evidence timing, trigger semantics, and execution conditions into the score, which is a prerequisite for meaningful comparison of streaming video-language systems.
Source: https://arxiv.org/abs/2609.30670
Hacker News Signals
Ember-1
Fireworks AI released Ember-1, a compound AI system targeting agentic coding workflows. The architecture is built around tool use and multi-step reasoning, with the model trained specifically to drive software development tasks — writing, editing, debugging, and navigating codebases. Rather than a single large model doing everything, Ember-1 is designed as a system that orchestrates tool calls (file reads, terminal execution, search) with a loop similar to ReAct or Toolformer-style execution, but tightened for the coding domain.
The technical claims center on latency and cost relative to frontier models. Fireworks positions Ember-1 as competitive with Claude 3.5 Sonnet and GPT-4o on SWE-bench style benchmarks while running on their own inference infrastructure, which allows sub-second tool-call round trips. The system is integrated with their inference platform, so the compound loop executes server-side rather than through client-driven API chaining — this matters for agentic workloads where network round-trip overhead compounds across dozens of tool calls.
The prompt-to-action pipeline uses structured output for tool dispatch, ensuring the model emits JSON-schema-conformant calls rather than free-form text that requires downstream parsing. Training involved supervised fine-tuning on execution traces and likely RLHF or rejection sampling against pass/fail signals from actual code execution, a now-standard recipe for coding agents.
Discussion on HN focused on whether compound systems like this are genuinely more capable or just faster/cheaper than calling a frontier model directly, with skepticism about benchmark reliability for real-world agentic tasks. A recurring point was that SWE-bench numbers are easy to game with scaffolding choices rather than model quality improvements.
No public weights; access is through the Fireworks API.
Source: https://fireworks.ai/blog/ember-1
Thinking Fast and Slow in AI: The Role of Metacognition (2021)
This paper maps Kahneman’s dual-process framework onto AI systems, distinguishing System 1 (fast, associative, pattern-matching) from System 2 (slow, deliberative, rule-governed) and asking what architectural and training properties would be needed for AI to exhibit genuine metacognition — the capacity to monitor and regulate its own reasoning processes.
The core technical argument is that current deep learning systems are predominantly System 1: they perform rapid pattern completion over learned distributions but lack explicit mechanisms for uncertainty-aware self-monitoring, working memory manipulation, or deliberate strategy switching. The authors survey attempts to introduce System 2 properties: scratchpad reasoning (chain-of-thought), neuro-symbolic hybrids, learned optimizers, and memory-augmented networks. Each is evaluated against a taxonomy of metacognitive capacities including monitoring confidence, detecting reasoning errors, allocating compute based on task difficulty, and switching strategies.
The paper is from 2021, predating the explosion of chain-of-thought prompting and RLHF-tuned reasoning models (o1, Deepseek-R1, etc.), which makes it interesting to read retroactively. The metacognitive criteria it defines — self-monitoring of confidence, adaptive compute allocation, explicit strategy representation — are precisely the properties that o1-style models partially instantiate through extended reasoning traces and process reward models. The paper’s framework predicts that these capabilities cannot emerge purely from scale without architectural or training changes that create explicit internal feedback loops.
Limitations of the paper are largely historical: it was written before the empirical evidence from large-scale RLHF and reasoning-trace training became available, so several of its pessimistic conclusions about scale have been partially falsified, while its architectural concerns remain open.
Source: https://arxiv.org/abs/2110.01834
The State of SIMD in Rust in 2026
A detailed survey of where portable SIMD stands in the Rust ecosystem. The author covers three main paths: the std::simd portable SIMD API (still nightly-only as of writing), direct platform intrinsics via std::arch, and autovectorization by relying on LLVM.
The std::simd API stabilization has been slow. The module provides width-agnostic vector types (Simd<f32, 8>, etc.) with lane-wise operations and explicit swizzle primitives, but its relationship with LLVM’s backend representation is imperfect — certain operations that should compile to a single instruction emit suboptimal code. The author benchmarks several patterns and shows cases where hand-written std::arch intrinsics outperform the portable API by 2-4x due to missed vectorization or redundant moves.
For std::arch, the ergonomics remain poor: every intrinsic call is unsafe, the naming is a direct lift from Intel’s intrinsics guide (e.g., _mm256_fmadd_ps), and writing portable fallbacks requires substantial boilerplate. The author recommends the multiversion crate for function-level multiversioning (dispatching at runtime to AVX-512, AVX2, or SSE4.2 implementations) as the most practical path for library authors today.
Autovectorization coverage is patchy. LLVM vectorizes straightforward loops reliably but fails on anything with non-trivial control flow, gather/scatter patterns, or horizontal reductions. Annotating loops with #[target_feature(enable = "avx2")] helps LLVM but loses portability.
The post also covers packed_simd2 (mostly abandoned), wide (a third-party portable SIMD crate with more stable semantics), and SVE/NEON on ARM. The conclusion is that Rust’s SIMD story is usable but requires either accepting unsafe intrinsics or tolerating autovectorization gaps; std::simd stabilization is the key unresolved blocker.
Source: https://shnatsel.github.io/state-of-simd-rust-2026/
Don’t Couple Your Go Code to GitHub
A practical engineering post arguing against using GitHub-specific conventions as first-class constructs in Go module paths and tooling. The central issue is the Go module system’s reliance on VCS-hosting URLs in import paths: a module declared as github.com/user/repo embeds the hosting provider into every import statement across every downstream consumer. Migrating to a self-hosted Gitea instance, GitLab, or any other provider then requires a global rename — a breaking change.
The recommended alternative is to use a custom domain with Go module proxy redirects. Go supports vanity import paths via <meta name="go-import"> HTML tags, so mypkg.example.com/somepkg can resolve to an underlying git repo on any host. The author walks through the nginx or Caddy configuration needed to serve these redirect responses, and how to configure a module proxy (GOPROXY) to cache fetches from the custom domain.
A secondary point concerns GitHub Actions workflows that use actions/checkout and other github.com-namespaced actions directly in CI. These create a softer coupling: switching CI providers requires rewriting all workflow files. The author suggests abstracting common CI steps into Makefile targets or shell scripts so the CI YAML becomes thin glue rather than the authoritative definition of build/test/lint procedures.
There is nothing fundamentally new here — the Go documentation covers vanity imports, and the advice about CI abstraction is standard. But the post is a useful reminder that import paths are a long-lived public interface decision, and that the path of least resistance (using github.com directly) trades short-term convenience for long-term lock-in. HN discussion debated whether this matters in practice, with most disagreement centering on how often projects actually migrate VCS hosts.
Source: https://iain.rocks/blog/dont-couple-your-go-code-to-github
A Single-Function Jev-like Wrapper for LLMs, Including Vision Models
This post presents a minimal Python wrapper that exposes a uniform llm(prompt, ...) interface over multiple LLM backends (OpenAI, Anthropic, local Ollama), with multimodal support for passing images. The design philosophy mirrors Jev (a minimal LLM interface concept circulating in the community): reduce the API surface to a single callable so that the calling code contains no provider-specific logic.
The implementation handles image inputs by base64-encoding them and inserting them into the appropriate message format for each provider — OpenAI’s content array with image_url items, Anthropic’s content array with image source blocks. The abstraction normalizes these differences so callers pass a file path or bytes object and the wrapper handles encoding and schema differences.
For text-only models, the wrapper normalizes system prompts, handles the messages list construction, and extracts the response string from whichever response schema the backend returns. Streaming is optionally supported. The total implementation is under 200 lines.
The engineering tradeoff is typical of any abstraction layer: you gain provider portability but lose access to provider-specific features (structured output schemas, tool-use formats, sampling parameter names vary). The author acknowledges this and recommends the wrapper for prototyping and scripting contexts rather than production deployments where provider-specific capabilities matter.
HN discussion focused on whether this is better than existing libraries like litellm, which provides the same abstraction at larger scale. The counterargument is that a self-contained single-file wrapper with no dependencies is operationally simpler for scripts and personal tooling. The vision model support is the more novel piece, since many wrappers handle text well but paper over image input format differences poorly.
Source: http://allanrbo.blogspot.com/2026/09/a-jev-like-wrapper-for-llms-including.html
Prompting Claude Opus 5.5
Anthropic’s official documentation for prompting Claude Opus 5.5 was posted to HN and generated substantial discussion, not because the content is novel but because it provides a direct window into how the model’s designers expect it to behave and how they recommend shaping that behavior.
The technical substance covers several specific patterns. Claude Opus 5.5 is described as having stronger instruction-following for long-horizon tasks and being more resistant to prompt injection in agentic contexts. The documentation recommends explicit XML-tag delimiters for structuring multi-part prompts (e.g., <instructions>, <context>, <output_format>) rather than relying on markdown or prose separation — Anthropic’s training process appears to have specifically reinforced XML structure as a reliable delimiter.
For chain-of-thought elicitation, the docs recommend asking for reasoning before answers rather than after, consistent with the finding that pre-answer scratchpads improve accuracy on tasks requiring multi-step logic. For agentic workflows, they emphasize instructing the model to confirm ambiguous actions rather than proceeding, and to prefer reversible over irreversible actions — an alignment-level behavioral preference baked into the model but accessible via prompting.
The section on long context is notable: the documentation warns that Opus 5.5 performs differently with instructions at the beginning versus end of long contexts, with end-of-context placement for critical instructions outperforming beginning placement in some task types. This is consistent with known attention sink and recency effects in long-context transformers.
HN discussion questioned how much of this is prompt engineering that transfers to other models versus Claude-specific behavioral artifacts of Anthropic’s RLHF data. No mechanistic explanation is offered in the documentation.
Source: https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/prompting-claude-opus-5-5
Turning GLM-5.3-Flash into a Jev-like Decision Model
PrivateMode AI describes a technique for converting GLM-5.3-Flash (a small, fast open-weight model from Zhipu AI) into a routing/decision layer they call “System One.” The goal is to use the model not for generation but for fast binary or multi-class decisions: given a context, classify what action to take before invoking a more expensive model or tool.
The approach involves fine-tuning on a custom dataset of (context, decision) pairs using supervised instruction tuning, then applying a classification head or constrained decoding to force the output into a small discrete action space rather than free-form text. The “Jev-like” framing refers to using the model as a lightweight planner that routes to heavier compute only when necessary.
Technically, GLM-5.3-Flash is appealing for this use case because it runs inference quickly on modest hardware (sub-100ms latency in 4-bit quantized form) and has a small memory footprint. By constraining generation to a fixed action vocabulary, the effective inference cost drops further since you only need logits over a handful of tokens rather than full autoregressive decoding.
The post reports results on their internal benchmarks showing that the fine-tuned decision model matches a larger baseline on routing accuracy while reducing total system latency by approximately 60% on tasks where most queries can be handled by cheaper downstream paths.
Open questions include how well this generalizes across distribution shifts (the decision model is presumably brittle to input patterns not in fine-tuning data) and whether the fine-tuning data collection process is practical for new application domains. The approach is essentially a learned router, similar in concept to mixture-of-experts gating but applied at the application orchestration layer rather than inside the model.
Source: https://www.privatemode.ai/blog/system-one-from-glm-flash
Is Your Postgres Migration Safe or Not Safe?
safenotsafe.dev is a reference site cataloguing Postgres schema migrations by their safety properties — specifically, whether they acquire locks that block reads or writes, whether they can cause table rewrites, and whether they are safe to run against a live production database without downtime risk.
The site organizes migrations into categories: safe (e.g., adding a nullable column with no default in Postgres 11+, adding an index with CONCURRENTLY), unsafe (e.g., adding a non-null column with a default prior to Postgres 11, which triggers a full table rewrite), and conditional (safe only with specific syntax or Postgres versions). Each entry explains the lock type acquired, the duration, and the failure mode.
The technically interesting cases are the ones where safety depends on version. Adding a column with a non-null default used to require rewriting every row (Postgres pre-11), but Postgres 11 changed this to store the default in catalog metadata for newly added columns and fill in the value lazily, making it safe. Similarly, ADD CONSTRAINT ... NOT VALID combined with a subsequent VALIDATE CONSTRAINT allows splitting a potentially long-running validation into an operation that acquires only a brief SHARE UPDATE EXCLUSIVE lock rather than ACCESS EXCLUSIVE.
Index creation with CREATE INDEX CONCURRENTLY is covered in detail — it performs multiple passes over the table and registers the index as INVALID until complete, which means it can be safely interrupted and resumed but requires handling the cleanup of invalid indexes if the process fails.
HN discussion added context around pg_repack, pglogical, and tooling like strong_migrations (a Rails gem that catches unsafe migrations at the ORM layer). The site serves as a useful quick reference for engineers who need to reason about lock contention without reading the full Postgres documentation on MVCC and locking.
Source: https://safenotsafe.dev/
Noteworthy New Repositories
mizorewww/laya-mlx
A native MLX runtime targeting Laya typed decision models, designed to run short-latency inference on Apple Silicon without any dependency on PyTorch, text generation pipelines, or cloud APIs. The project is explicitly scoped to decision models — structured classification and regression tasks — rather than generative LLMs, which allows the runtime to stay lean and avoid the overhead of token-sampling infrastructure.
The implementation sits directly on Apple’s MLX framework, exploiting unified memory and the M-series Neural Engine to achieve 7–14 ms end-to-end latency on M3 Max hardware. That figure is for a full decision pass, not just a matmul, making it competitive with server-side inference for latency-sensitive on-device applications. There is no Python packaging overhead at call time; the runtime is designed for tight integration into native macOS/iOS application stacks.
The design philosophy mirrors what llama.cpp did for generative models but restricts scope to typed decision workloads — think structured prediction, routing, policy evaluation — where the input/output contract is strongly typed rather than free-form text. This makes it a credible drop-in for edge ML pipelines that currently call out to a remote endpoint. The absence of PyTorch dependency is a meaningful binary-size and cold-start win for app distribution.
Worth watching if you are building on-device decision logic on Apple Silicon and need inference that respects real-time budgets without cloud round-trips.
Source: https://github.com/mizorewww/laya-mlx
MichaelKinsy/PiG
PiG is a line-for-line behavioral port of the Pi coding agent — originally written in TypeScript — to Go. The stated contract is upstream behavioral parity: any divergence from Pi’s TypeScript runtime is treated as a bug in PiG, not a feature. This is a translation project, not a redesign.
The value proposition is deployment ergonomics. Go produces a single statically linked binary with no Node.js or npm runtime dependency, which simplifies CI integration, container sizing, and distribution to engineers who do not want a JavaScript toolchain as a side effect of running a coding agent. Go’s goroutine model also provides a cleaner substrate for the concurrent I/O patterns typical of agent loops — tool calls, subprocess management, streaming responses — without the callback/promise layering that TypeScript necessitates.
Parity-bound translation is a disciplined constraint. It means the maintainer cannot introduce idiomatic Go abstractions that would change observable behavior, which keeps the test surface well-defined: pass the same inputs through both runtimes and diff the outputs. This makes regression testing tractable and gives downstream users a clear upgrade path as Pi’s TypeScript upstream evolves.
Useful for teams that want Pi’s agent behavior inside Go-native infrastructure — monorepos with Go tooling, Kubernetes operators, or CLI tools — without vendoring a Node runtime. The single-binary artifact is also directly deployable as a sidecar or Lambda function.
Source: https://github.com/MichaelKinsy/PiG
mirkovicdev/HFTENGINE
A terminal-mode replay console for market-making backtests built on the hftbacktest Rust crate, operating on real recorded Binance Futures tick data. This is a visualization and analysis layer, not an execution system.
The core value is granularity. Most backtesting frameworks report fill rates and P&L; HFTENGINE exposes the full microstructure state at each simulation tick: the order book at every level, the queue position of each resting limit order, separate feed latency and order-acknowledgment latency channels, and the raw collector feed alongside the simulated fills. This level of detail is essential for diagnosing queue-position effects — a dominant source of slippage in high-frequency market-making that aggregate metrics obscure entirely.
The runner is implemented in Rust, leveraging hftbacktest’s event-driven simulation engine which models latency as explicit message delays rather than a fixed slippage assumption. The front-end is a text-mode TUI dashboard, keeping the dependency footprint minimal and making it usable over SSH on data-center machines where the recorded data lives.
The design is deliberately read-only. There is no order routing, no exchange connectivity, no live trading path. That constraint makes it safer to run against proprietary tick data without accidentally wiring up execution logic. For researchers studying queue dynamics, adverse selection, or inventory management in limit-order-book environments, this fills a gap between raw hftbacktest output and the kind of interactive inspection that reveals why a strategy leaks.
Source: https://github.com/mirkovicdev/HFTENGINE
0xethanq/astra-quant-agent
A multi-agent LLM trading system structured as a simulated investment committee, with connectors to OKX, Binance, and Gate.io. The architecture assigns distinct LLM agents to analyst, risk, and CIO roles; agents debate a position, the CIO agent selects a final decision, and that decision passes through 17 fail-closed risk gates before any order is submitted.
The fail-closed design is the technically interesting part. Each gate is a hard stop rather than a soft warning — if any gate cannot confirm safety (connectivity loss, position limit breach, anomalous volatility, etc.), the trade is blocked rather than passed. This inverts the typical default-allow pattern that makes naive LLM trading systems dangerous.
Prompts are described as self-evolving: the system logs outcomes and rewrites agent instructions based on recent trade results, implementing a lightweight online prompt-optimization loop without full fine-tuning. A complete audit trail records each agent’s reasoning, the gate evaluations, and the final order parameters, which is important for post-hoc analysis of why specific trades were taken or blocked.
The multi-exchange abstraction normalizes order types and position accounting across venues. Crypto quant research often fragments across exchange-specific SDKs; having a unified interface with a structured agent layer above it reduces the plumbing work for strategy experimentation.
Significant caveat: LLM-driven live trading carries model unpredictability risks that no finite set of rule-based gates fully mitigates. Use in production requires treating the risk gates as a necessary but not sufficient safety layer.
Source: https://github.com/0xethanq/astra-quant-agent
timoncool/YuE2-Studio
A native Windows application wrapping YuE2, an open-source music generation model, with a focus on making full-song generation with vocals accessible without a Python environment. The installer bundles the runtime and handles model weights, with an auto-update mechanism to track upstream model changes.
The technically notable addition over raw YuE2 inference is the editable score layer. Generated songs are presented alongside sheet music that reflects the model’s output, and the user can edit that score and regenerate — closing a feedback loop that pure audio generation tools lack. This matters because YuE2, like most audio diffusion models, is stochastic; the score representation gives users a structured handle on the output rather than requiring repeated blind resampling.
Exact replay is supported, meaning a generation seed and parameters are stored and can be re-executed to reproduce a specific output exactly — essential for iterative editing workflows where you want to modify one element without regenerating the whole piece.
Cover generation extends the model beyond original composition: supply a reference track and the system produces a new version in the model’s style. This is implemented through YuE2’s conditioning interface rather than audio transformation.
The no-Python constraint is a real engineering commitment for a Windows desktop app targeting non-ML users. The runtime is packaged as a self-contained executable, which trades update flexibility for deployment simplicity. GPU acceleration is required; the model does not run at useful speed on CPU.
Source: https://github.com/timoncool/YuE2-Studio
jankeesvw/omarchy-meeting-recorder
A meeting recorder built specifically for the Omarchy Linux environment (an opinionated Arch-based desktop setup). The design records microphone and system audio as separate tracks simultaneously — preserving the distinction between the local speaker and remote audio — then runs on-device transcription with speaker diarization, chapter segmentation, and a playback interface.
The two-track capture is the key architectural choice. Most meeting recorders collapse audio to a single mixed stream before transcription, which makes speaker attribution a post-hoc inference problem. By keeping the tracks separate at capture time, the local speaker is always identifiable without diarization heuristics, and remote speakers can be diarized on the combined remote track rather than a mixed signal. This improves transcript quality for multi-speaker meetings.
Transcription runs locally, which keeps meeting content off cloud services — relevant for any organization with data handling constraints. Speaker labels and chapter markers are generated as part of the same pipeline, producing a structured transcript rather than a raw text dump. The integrated player lets you navigate by chapter or speaker segment rather than scrubbing audio manually.
The Omarchy dependency limits the user base but also means the tool can integrate tightly with that environment’s audio routing — PipeWire-based multi-source capture is straightforward on Arch where it is often a configuration challenge on other distributions. Useful for anyone running Omarchy who wants local, structured meeting records without shipping audio to a third-party service.
Source: https://github.com/jankeesvw/omarchy-meeting-recorder
yibie/awesome-jev
A curated index of community projects, integrations, and documented use cases built on Jev, TypeSafe AI’s System One model for typed decision inference. With nearly 1,900 stars, it functions as a community discovery layer for a relatively nascent ecosystem.
The technical substance is in what gets catalogued: integrations connect Jev’s typed decision interface to downstream systems — databases, API gateways, workflow orchestrators — and the projects section surfaces production deployments and open-source tools that compose with the model. The discussions category documents architectural patterns and trade-offs, which is often more useful than raw code when evaluating whether a framework fits a use case.
Typed decision models occupy a specific niche: they enforce a schema on both input features and output decisions, making the model’s contract machine-checkable rather than relying on prompt engineering to produce structured output. Jev exposes this through a type system rather than JSON schema validation bolted onto a generative model, which changes how downstream systems integrate — you get compile-time or schema-validation-time guarantees rather than runtime parsing.
The value of an awesome list for this kind of infrastructure is primarily ecosystem legibility: understanding what problems others have solved with the tool, which integrations are production-tested, and where the sharp edges are. For a typed decision framework, the integration patterns (how it connects to feature stores, how decisions are logged for audit, how it handles schema evolution) are the technically interesting artifacts.
Source: https://github.com/yibie/awesome-jev
yetone/magpie
A macOS menu-bar application that maps coding agent frontends to arbitrary backend model providers. The canonical examples are running Codex-style agents against DeepSeek’s API and Claude Code against Kimi’s endpoint — decoupling the agent UX from the model vendor.
The core mechanism is API compatibility shim: providers like DeepSeek and Kimi expose OpenAI-compatible REST endpoints, and Magpie routes the agent’s outbound requests to the configured provider while preserving the request/response schema the agent expects. This requires no modification to the agent software itself; the shim operates at the network layer or via environment variable injection of the base URL and credentials.
From the menu bar, users switch the active backend per-agent, which is useful when different tasks favor different model cost/capability trade-offs — a fast, cheap model for routine completions and a stronger model for architecture-level tasks. The menu-bar placement keeps this switching lightweight compared to editing config files or environment variables per session.
The practical appeal is cost and latency arbitrage. Claude Code’s UX is well-regarded but Anthropic’s pricing is high at scale; routing to a cost-equivalent provider for lower-stakes tasks without changing the agent workflow reduces burn rate. Similarly, DeepSeek’s code models are competitive on benchmarks at substantially lower API cost than OpenAI, making Codex-fronted workflows cheaper to run at volume.
No model fine-tuning or local inference is involved — this is purely a routing and credential management tool. The limitation is that behavioral parity across providers is assumed but not guaranteed; prompt sensitivity varies between models even with identical API schemas.
Source: https://github.com/yetone/magpie