Daily AI Digest — 2026-07-09
arXiv Highlights
Sparse Delta Memory: Scaling the State of Linear RNNs through Sparsity
Problem
Linear attention and gated linear RNNs (Mamba2, Gated DeltaNet) offer fixed per-token compute and constant memory, but their recurrent state is a dense outer-product matrix \mathbf{M}_t \in \mathbb{R}^{d_{qk}\times d_v} whose size trades directly against FLOPs. On long-context recall this state is too small; enlarging it costs quadratic compute per update because every slot is touched every step. The paper’s premise: the dense delta-rule update is wasteful — most tokens only need to read/write a handful of slots. Sparsifying the addressing decouples state capacity from per-token FLOPs.
Method
Sparse Delta Memory (SDM) modifies the Gated DeltaNet (GDN) recurrence by replacing the dense k_t \otimes v_t update with sparse read/write into an explicit memory table \mathbf{M}_t \in \mathbb{R}^{N \times d_v} with N \gg d_{qk} slots. At each step:
Sparse key selection (PKM-style). Keys and queries are factored into two halves of size \sqrt{N}: k' \to [k_1', k_2'] with k_i' \in \mathbb{R}^{\sqrt N}, and the score matrix is the outer sum k_1' \oplus k_2' \in \mathbb{R}^{\sqrt N \times \sqrt N} interpreted as N slot scores. Top-W selects write indices \mathcal{I}^w_t, top-R selects read indices \mathcal{I}^r_t. Using \text{top}_k(s_1 \oplus s_2) = \text{top}_k(\text{top}_k(s_1) \oplus \text{top}_k(s_2)) avoids materializing the full N-vector, giving O(\sqrt N \cdot d + W^2 + R^2) selection cost.
Gated delta write. For each i \in \mathcal{I}^w_t: \tilde{\mathbf{M}}_t[i] \leftarrow \alpha_t \cdot \mathbf{M}_{t-1}[i], \mathbf{M}_t[i] \leftarrow \tilde{\mathbf{M}}_t[i] + \beta_t \cdot k_t^{(i)} \cdot (v_t - \tilde{\mathbf{M}}_t[i]), with per-head forget gate \alpha_t = \exp(-A \cdot \text{softplus}(W_a x_t + b_{\text{dt}})) and input gate \beta_t = \sigma(W_b x_t). This is the GDN delta rule applied only to the W selected rows.
Sparse read. y_t = \tilde{\mathbf{M}}_t^\top q_t restricted to \mathcal{I}^r_t, then g_t \odot \text{LN}(y_t) and output projection W_o.
Configuration in experiments: W = R = 64, N = (d/4H)^2 chosen so the state-to-parameter ratio is roughly 1:1 (Table 1 shows St:Param ranging 56–150% for SDM vs 0.03–0.16% for GDN at matched parameter count). The architecture is hybrid: 3:1 short:long ratio of sliding-window MHA (window 128) to global SDM/GDN/full-attention layers, GQA with group 2, RoPE \theta=5\!\times\!10^5. The initial memory \mathbf{M}_0 is a learned parameter, giving SDM a parametric-memory interpretation.
Results
Scaling law. Trained on a 160-tokens-per-parameter budget across 8 ladder levels (280M → 1.48B), SDM’s loss-vs-FLOPs curve tracks a power law with R^2 = 0.999. Extrapolating to level-13 (8B, 1.14T tokens), the prediction is confirmed: SDM’s final loss is below GDN’s and below FullAttn’s at the same scale.
At smaller scales SDM already dominates iso-FLOP GDN; the crossover with FullAttn happens as scale grows because FullAttn’s quadratic cost forces smaller models at iso-FLOP. Note the state sizes involved: at level-13 SDM carries a 7.96B-parameter memory state alongside 8.14B model parameters, vs 2.2M state for GDN — roughly 3600\times larger recurrent state at the same FLOPs.
Long-context. On 1M-token code documents (1.4B model, 128k long-context fine-tuned), perplexity by position shows SDM maintaining lower PPL far beyond the training context, whereas GDN degrades earlier.
Ablation on learned \mathbf{M}_0. SDM without a learned initial state still substantially outperforms GDN, isolating capacity as the dominant factor. Learning \mathbf{M}_0 helps SDM further but does nothing measurable for GDN, consistent with GDN having too little state to exploit a parametric prior.
Limitations and open questions
- The 8B result is a single training run confirming an extrapolated scaling law from \leq 1.5B fits; the confidence interval is bootstrapped from the ladder, not from replicates.
- FLOP accounting excludes the sparse embedding memory from parameters. In practice, SDM at level-13 stores ~8B floats of state that must live in HBM and be gathered/scattered every step; wall-clock and memory-bandwidth cost, not reported in the excerpts, is the real deployment question.
- Top-k selection via the factored k_1' \oplus k_2' trick is not exact top-k on the true N-way score; the paper uses the identity that iterated top-k on the two axes recovers the joint top-k, which holds combinatorially but may interact unfavorably with softmax normalization on the selected activations.
- W = R = 64 is fixed; how performance scales with sparsity (W/N ratio) and whether reads and writes should decouple in count remains unstudied here.
- Comparisons are against GDN/Mamba2/FullAttn hybrids; no comparison to other large-state approaches (e.g., Titans, memory-augmented transformers, retrieval-augmented recurrences).
Why this matters
SDM demonstrates that the state-capacity bottleneck of linear RNNs is not intrinsic — sparse addressing with a factored PKM-style key selector lets the recurrent state grow to match the parameter count while keeping per-token FLOPs constant, and this suffices for a hybrid model to beat 8B full attention at iso-FLOP with predictable scaling. It reframes gated delta rules as sparse writes to an explicit key-value store, blurring the line between recurrent state and parametric memory.
Source: https://arxiv.org/abs/2607.07386
Imagined Rollouts are Kinematic, Not Dynamic: A Diagnosis of Long-Horizon World-Model Failure
Long-horizon degradation of learned world models is usually catalogued under “compounding error,” a phrase that treats every deviation from ground truth as fungible. This paper argues the failure has a specific character: latent-space rollouts tend to preserve kinematic plausibility — positions and velocities that look like motion — while violating the dynamical constraints that would link those trajectories to forces, contacts, and regime-dependent behavior. The distinction matters because policies trained inside such rollouts optimize against a physics that is not actually the environment’s physics; when the environment’s dynamics change (friction, mass, actuator saturation), an imagination-based controller has no signal that anything is different.
The diagnostic: imagined Kinematic-Consistency Error
The authors define a per-step imagined Kinematic-Consistency Error (iKCE) that measures the residual between a rollout’s decoded state transitions and a closed-form kinematic null model. The null is deliberately weak: it encodes only kinematic identities such as x_{t+1} \approx x_t + \dot{x}_t \Delta t (and analogous integrations for orientation), with no force model. If a rollout satisfies the kinematic null closely, it is consistent with “things that look like motion” but not necessarily with Newtonian response to control inputs or contact.
Concretely, for a decoded rollout \{\hat{s}_t\}_{t=0}^H, iKCE at step t is
\text{iKCE}_t = \big\lVert \hat{s}_{t+1} - f_{\text{kin}}(\hat{s}_t, \hat{s}_{t-1}) \big\rVert,
where f_{\text{kin}} is the closed-form kinematic predictor (finite-difference velocity extrapolation with the embodiment’s known link lengths and joint parameterization). Real physics rollouts have nonzero iKCE because true dynamics deviate from pure kinematic extrapolation at every contact and actuation event; that residual is the dynamical signature. A world model that imagines “kinematically” will drive iKCE toward zero on its own rollouts even though the reference process is far from kinematic.
The perturbation protocol
The second half of the diagnostic tests whether iKCE responds to a physical regime change. The authors sweep a physical parameter — ground friction on DMC walker-walk — across a range that includes the gait-collapse boundary, i.e., the friction value below which the trained policy can no longer maintain a walking limit cycle. For each friction value they compute (i) the environment’s true reward under the trained policy, and (ii) the imagined iKCE of rollouts produced by the world model conditioned on that friction. A dynamically faithful imagination should show iKCE moving as friction crosses the collapse boundary; a kinematically-biased imagination will not.
Results on DreamerV3, DMC walker-walk
Using a released DreamerV3 checkpoint, the imagined iKCE runs roughly two orders of magnitude above the iKCE of matched real-physics rollouts — the opposite direction of what a “kinematic imagination” naive reading might predict, and the authors’ clarifying point is important: the imagined trajectories are internally kinematic-looking in that their reaction to physical regime is absent, but they are also not consistent with the environment’s kinematic null because the decoder introduces its own drift. What matters diagnostically is the response of iKCE to perturbation, not the raw scale.
Across the friction sweep, the trained policy’s true environment reward collapses through the gait-collapse boundary — a large, monotone drop — while the world model’s imagined iKCE remains statistically flat across the same friction range. This is the kinematic-not-dynamic signature: the rollout distribution is invariant to a physical parameter that the true dynamics are not invariant to. The signature becomes detectable at horizons longer than the embodiment’s gait period, roughly the timescale on which contact and actuator dynamics would have to be integrated correctly for a rollout to show regime dependence.
Limitations and open questions
The instantiation is narrow: a single checkpoint, a single embodiment, a single perturbed parameter. Whether the same signature appears for mass, actuator gain, or contact stiffness perturbations, and whether it generalizes to manipulation tasks where contact is the dominant nonlinearity, is untested. The kinematic null is embodiment-specific and requires knowledge of link geometry; scaling this to pixel-only observations or unknown morphologies requires an additional identification step. The paper also does not propose a training-time remedy — it is a diagnostic, not a fix — leaving open whether penalizing iKCE-invariance during world-model training, or injecting explicit dynamical residuals, would produce imaginations that respond to regime change. Finally, the reward-vs-iKCE decoupling is shown qualitatively; a formal statistical test relating iKCE gradient with respect to a physical parameter to policy-return gradient would tighten the claim.
Why this matters
Reframing world-model failure from “compounding error” to “kinematic-vs-dynamic imagination” gives a mechanistic, testable target: the model can be right about what motion looks like while being wrong about what causes it, and this is precisely the failure mode that breaks sim-to-real and off-policy planning under changed physics. A cheap per-step diagnostic that flags this — without needing counterfactual rollouts of the true environment — is a useful primitive for anyone deploying Dreamer-style agents beyond the training distribution.
Source: https://arxiv.org/abs/2607.05966
Accurate, Interdisciplinary and Transparent Structure-property Understanding with Deep Native Structural Reasoning
Structure-property prediction across proteins, small molecules and inorganic crystals is typically balkanized: each domain has its own representations (residue graphs, SMILES/3D conformers, unit cells with periodic boundary conditions) and its own inductive biases. SciReasoner attempts a unification by treating structural primitives — atomic coordinates, bond topology, periodic connectivity — as discrete tokens inside a single vocabulary that a decoder-only reasoner can attend over and cite. The claim is that domain-native structural tokens, when used as addressable evidence during chain-of-thought reasoning, yield both better accuracy on hard generalization splits and traceable explanations.
Problem
Two failure modes motivate the work. First, most scientific LLMs consume linearized strings (FASTA, SMILES, CIF text) that discard geometric and periodic information, so predictions collapse under distribution shift (e.g., low-homology proteins, out-of-distribution scaffolds, new space groups). Second, specialized structural models (GNNs, equivariant nets) predict well but do not produce a reasoning trace that references specific structural features. The joint requirement is: (i) preserve structural evidence at the token level; (ii) let a reasoning process point to particular tokens as justifications tied to physical constraints (stereochemistry, symmetry, energetics).
Method
The core construction is a structure-aware vocabulary that discretizes three heterogeneous inputs into a common token stream:
- Proteins: residues plus discretized backbone/side-chain geometry (torsions \phi, \psi, \chi and C_\alpha contact tokens).
- Small molecules: atom identity, bond order, stereo descriptors, and quantized 3D coordinates.
- Crystals: atom types, Wyckoff positions, lattice parameters, and periodic connectivity tokens encoding the unit-cell adjacency graph modulo translations.
Coordinates are quantized (VQ-style) into a shared codebook so that “geometry” is a symbol addressable by the LM. Formally, an input structure S becomes a sequence (t_1, \ldots, t_n) with t_i \in \mathcal{V}_{\text{struct}} \cup \mathcal{V}_{\text{text}}, and the model factorizes
p_\theta(y, r \mid S) = \prod_{k} p_\theta(u_k \mid u_{<k}, t_{1:n})
where r is a reasoning trace whose tokens are trained to reference structural token indices (attention-based citations) and y is the property label. Training combines: (a) next-token prediction on scientific corpora with interleaved structure tokens; (b) supervised reasoning traces that annotate which structural tokens support intermediate claims; (c) task-specific fine-tuning for GO annotation, reaction outcomes, materials properties, etc. The “native” aspect is that periodicity and stereochemistry survive tokenization rather than being flattened into strings.
Reasoning is transparent in that each generated justification span carries pointers back to the structural tokens it used, giving per-prediction attributions over residues, functional groups or Wyckoff sites.
Results
The headline evaluation is homology-controlled Gene Ontology prediction, where sequence identity to training is capped to isolate low-homology and orphan-like proteins — the regime where pure sequence models degrade. SciReasoner improves Cellular Component F_{\max} in this regime (the abstract is truncated mid-number, but the direction and setting are the important claims: gains concentrate on low-homology and orphan splits rather than easy homologs). The framing implies smaller or negligible gains on high-homology splits, which is the expected pattern when structural priors matter most where sequence signal is weakest.
Beyond proteins, the paper positions SciReasoner as a single model across molecular property prediction and crystal property regression, arguing that the shared structural vocabulary allows cross-domain transfer (e.g., stereochemistry priors learned on molecules aiding protein side-chain reasoning; symmetry priors from crystals aiding assembly reasoning). Without the full tables in the provided excerpt, the quantitative claims to weigh are: (i) improved F_{\max} on the hardest GO splits; (ii) attribution traces that a domain expert can audit against known mechanistic features (active-site residues, pharmacophores, coordination environments).
Limitations and open questions
Several concerns follow from the design. Coordinate quantization introduces a discretization error whose effect on properties sensitive to sub-Ångström geometry (binding affinity, band gaps near degeneracies) is unclear; the codebook size and reconstruction fidelity are the key knobs. Periodic-connectivity tokens encode the graph but not exact lattice vectors — properties depending on strain or anisotropy may need auxiliary continuous heads. The reasoning traces are supervised, so their faithfulness depends on trace-generation quality; without an intervention-based faithfulness test (e.g., ablating cited tokens and checking prediction change), “transparent” is a training objective rather than a guarantee. Finally, homology-controlled GO is one hard split; broader OOD benchmarks (TAPE, MatBench with structure-based splits, therapeutic OOD) would sharpen the claim of interdisciplinary generalization.
Open questions: how does the shared codebook scale as new modalities (surfaces, MOFs, polymers) are added; can equivariance be enforced post-tokenization; and does the reasoning trace produce actionable hypotheses (e.g., predicted mutations) that experiments confirm?
Why this matters
Unifying protein, molecule and crystal reasoning inside a single tokenized decoder — with structural tokens as citable evidence — is a plausible route to scientific foundation models that generalize where sequence-only or string-only LLMs fail. If the attribution traces hold up under faithfulness probes, the approach turns structure-property prediction from a black-box regression into an auditable inference over named structural features.
Source: https://arxiv.org/abs/2607.07708
Dual Latent Memory in Vision-Language-Action Models for Robotic Manipulation
Standard VLA policies factor manipulation as a Markovian map \bm{a}_{t:t+H-1} = \bm{\Pi}_\theta(\bm{o}_t, \bm{I}), which loses temporal state whenever the current frame is insufficient to disambiguate progress (e.g., “has the cube been picked up already?”). Prior memory-augmented VLAs either widen the observation window or retrieve raw history from an external bank and concatenate it as policy-side context. Both leave memory outside the VLA’s native embedding space, so historical evidence cannot participate in cross-modal reasoning through the same attention pathways as the instruction and current view.
LaMem-VLA reframes memory as context-native: historical experience is compressed into latent tokens that live in the same embedding space as vision-language reasoning tokens, and are injected via attention into the action-generation stream.
Method
The system decomposes into four modules on top of a 7B Prismatic VLM and a ~300M-parameter diffusion action expert producing 7-DoF end-effector chunks of length 16.
Curator (dual vaults). Historical experience is partitioned into two vaults, each of maximum capacity L=16: a short-term vault holding fine-grained recent embeddings (visuomotor detail: object contact, gripper transitions) and a long-term vault holding coarser, more semantic units that summarize task progress. The split lets the model separately preserve high-frequency perceptual state and slow-changing task state, rather than compressing both through a single retention policy.
Seeker. Given the current multimodal representation \bm{h}_t from the VLM encoder, a query builder \mathcal{B} (a two-layer masked-attention transformer) forms a query used to retrieve the top K=8 units from each vault. Retrieval is over the same latent space that VLM reasoning operates in, so relevance is measured in the model’s own semantic geometry rather than by an external similarity metric.
Condenser. Two “memory formers” \mathcal{F}_v (visual/short-term) and \mathcal{F}_c (contextual/long-term), also two-layer masked-attention blocks, reconstruct the retrieved evidence into fixed-length latent memory tokens: L_s=8 short-term tokens and L_l=4 long-term tokens. Fixed length is critical for stable interleaving with the reasoning sequence and for compute predictability.
Weaver. The L_s + L_l = 12 memory tokens are inserted into the VLM’s reasoning sequence alongside instruction and current-observation tokens; the diffusion action expert then denoises action chunks conditioned on the memory-grounded hidden states, using DDIM with 10 denoising steps at inference.
Training uses 8 H800 GPUs, per-GPU batch 32 (global 256), learning rate 2\times 10^{-5}, single third-person RGB at 224\times 224.
Results
On SimplerEnv-Bridge with the WidowX robot, LaMem-VLA reaches 73.9% average success across four tasks (Spoon-on-Towel, Carrot-on-Plate, Stack-Cube, Eggplant-in-Basket). This surpasses the strongest reported baselines: MemoryVLA at 71.9%, \pi_0 at 69.2%, SemanticVLA at 65.1%, CronusVLA at 60.4%, and CogACT at 57.3%. OpenVLA sits at 4.2% and RT-1-X at 1.1% on the same suite, showing how brittle Markovian policies are on these compositional tasks.
Per-task, LaMem-VLA is competitive or best on Carrot-on-Plate (75.0%, tied with MemoryVLA), Stack-Cube (41.7%, best), and Spoon-on-Towel (83.3%, matching SemanticVLA’s 83.6% and \pi_0’s 83.8%). Eggplant-in-Basket at 95.8% trails the 100.0% ceiling several methods reach, suggesting the remaining error is not memory-limited on the easiest task.
The token-count ablation (Figure 3) is what makes the design claim concrete: the fixed-length compression to (L_s, L_l) = (8, 4) is not a trivial choice. Both under- and over-allocation degrade performance, indicating that the condenser is doing meaningful bandwidth-limited summarization rather than acting as an unstructured cache.
Limitations and open questions
- Only a single third-person RGB view is used; it is unclear whether latent memory yields the same gains under wrist-cam or multi-view inputs where redundancy already provides temporal cues.
- The dual-vault split is prescribed rather than learned; there is no analysis of what actually populates the short- vs. long-term vaults, so the interpretability claim (“perceptual vs. semantic”) is architectural, not empirical.
- Retrieval uses top-K over the current encoder embedding, which risks distribution shift between train-time and long-rollout inference embeddings; robustness across horizon length is not reported.
- Vault capacity is capped at L=16, which is small; scaling behavior for truly long-horizon tasks (hundreds of steps) is not shown.
- No real-robot numbers are reported in the excerpt, only SimplerEnv; sim-to-real transfer of the latent memory mechanism remains open.
Why this matters
Moving memory from a policy-side auxiliary bank into the VLM’s native token stream is a small architectural shift with a clear mechanistic justification: retrieved history now participates in the same attention computations as the instruction and observation, rather than being appended as a separate conditioning signal. The 2-point gain over MemoryVLA at matched scale is modest but the ablation supports the interpretation that compression bandwidth, not just retrieval, is doing the work.
Source: https://arxiv.org/abs/2607.07608
RoboDojo: A Unified Sim-and-Real Benchmark for Comprehensive Evaluation of Generalist Robot Manipulation Policies
Problem
Benchmarks for generalist manipulation policies are fragmented: most are simulation-only or real-only, cover narrow skill sets, and use short-horizon pick-and-place variants that fail to distinguish policies on distinct capability axes (precision, memory, long-horizon composition, open-vocabulary instruction following). This makes it hard to diagnose which capability is bottlenecking a given VLA model, and cross-lab comparison is impaired by hardware, workspace, and protocol drift. RoboDojo proposes a single benchmark that couples 42 simulation tasks (Isaac Sim / Isaac Lab) with 18 standardized real-world tasks, plus a cloud-accessible evaluation harness (RoboDojo-RealEval) and a shared policy interface (XPolicyLab).
Benchmark design
The simulation suite is built on the ARX X5 bimanual platform (0.6 m base separation) and partitions tasks into five capability dimensions: Generalization (12 tasks; distribution shift over backgrounds, lighting, clutter, and target objects), Precision (fine-grained contact-rich control), Long-Horizon (multi-stage compositions), Memory (persistent scene state), and Open (open-vocabulary instructions). Compared with RoboTwin 2.0 (≤10 clutter objects), RoboDojo randomizes up to 25 clutter objects and ships 100 auxiliary DLC pick-and-place trajectories under randomized backgrounds/lighting to enable data-level augmentation without leaking task solutions.
Tasks are specified via modular YAML: assets, layouts, initialization distributions, randomization ranges, and success predicates. Scene construction, object sampling, articulation state, clutter placement, lighting, and background textures are drawn under deterministic seeds — the same runtime executes rigid, articulated, and deformable assets. Vectorized execution uses Isaac Lab’s interface, extending the MagicSim modular manager architecture. The important engineering detail is “heterogeneous parallel simulation”: different task configurations run concurrently in a single vectorized session rather than replicating one env, which is what makes running 30 policies × 5 dimensions × 3 seeds × 50 trials tractable.
Real-world evaluation (RoboDojo-RealEval) standardizes hardware, workspace layout, lighting, reset procedure, and provides remote cloud access — so results across labs are executed on the same physical rigs rather than “similar” ones.
Evaluation protocol
For each policy: 3 seeds × 50 trials/task. For Generalization, the 50 trials split into 25 standard + 25 randomized. Total 150 trials per task per policy. Metrics are mean success rate and a continuous score (partial credit for sub-stage completion), reported per dimension with std, then averaged across dimensions. Three expert VR teleoperators (>1000 h each) provide a human reference under identical horizons and success criteria.
Main results
The leaderboard (frozen 3 Jul 2026, 30 policies) shows generalist manipulation is far from solved.
- Best policy: Hy-Embodied-0.5-VLA at 13.07 score / 8.80\% success averaged across dimensions.
- Human teleoperation: 80.42 / 76.03\%. Roughly a 9\times gap in success rate.
- Leading cluster (avg. score): Hy-Embodied-0.5-VLA (13.07), Spatial Forcing (12.38), \pi_{0.5} (11.41), X-VLA (10.13), X-WAM (7.69), Xiaomi-Robotics-0 (6.93), StarVLA-\alpha (6.40), GigaWorld-Policy (6.20).
Capability leadership is fragmented: Spatial Forcing tops Generalization (14.12 / 9.33\%), X-VLA tops Precision (18.32 / 12.00\%), \pi_{0.5} is competitive on Open (1.98 / 1.67\%), and Hy-Embodied-0.5-VLA leads Long-Horizon (25.74 / 14.92\%) and Memory (13.37 / 12.11\%). No policy dominates across dimensions.
Absolute numbers per dimension expose severity:
- Precision: best success 12.00\% (X-VLA) — indicating that fine-grained contact modeling is essentially unsolved.
- Memory: best 12.11\% — persistent scene state tracking fails broadly.
- Open (open-vocabulary): best 1.67\% (\pi_{0.5}) — instruction generalization collapses at deployment; scores below 1 for most policies.
- Long-Horizon and Generalization are relatively “stronger” but still cap at \sim 14.92\% and \sim 9.33\% respectively.
The Generalization dimension confirms Finding 2 in the paper: scene-level randomization (background, lighting, clutter distribution, object appearance) triggers broad performance collapse, and visual-spatial grounding methods (e.g., Spatial Forcing) only partially recover robustness.
Limitations and open questions
- Score compression: with best average success at 8.80\%, the leaderboard has limited resolution for ranking within the leading cluster; standard deviations across 3 seeds may be comparable to differences between neighboring methods.
- The five capability dimensions are hand-designed; whether they factorize cleanly (or overlap, e.g., Long-Horizon subsuming Memory) is not empirically validated.
- Real-world coverage is a single embodiment (ARX X5); dexterous hands, humanoids, tactile, and mobile manipulation are deferred to future extensions.
- Sim-to-real transfer numbers per policy are not summarized here — the value of the “unified” claim depends on how correlated the two leaderboards are, which the paper does not fully quantify.
- Cloud real-world evaluation introduces a shared physical resource contention model whose fairness (queueing, environment drift over time) will matter as the leaderboard scales.
Why this matters
If the numbers hold up, RoboDojo says the frontier of generalist manipulation is at roughly 9\% average task success versus 76\% for humans, with essentially no policy above 2\% on open-vocabulary tasks — a much less rosy picture than headline demos suggest. Capability-factored evaluation with a shared real-world rig is the right instrument for triaging where scaling data, changing architectures, or adding tactile/force sensing actually helps.
Source: https://arxiv.org/abs/2607.04434
WildCity: A Real-World City-Scale Testbed for Rendering, Simulation, and Spatial Intelligence
Problem
Scene reconstruction and neural rendering have matured on room- and block-scale captures, but scaling to a coherent city-scale digital twin — the natural substrate for embodied agents, driving policies, and spatial memory research — is bottlenecked by data. Existing autonomous-driving datasets (nuScenes, Waymo, Argoverse) contain short clips (20 s to a few minutes) at isolated locations, which precludes studying long-horizon spatial consistency, loop closure at kilometer scales, and cross-district generalization. WildCity targets this gap: 18 trajectories averaging 83.7 km each, captured by a production autonomous fleet, retaining the in-the-wild pathologies (dynamics, exposure drift, imperfect poses) that clean academic captures avoid.
Data and sensor stack
The rig is a roof LiDAR, six RGB cameras (three narrow-FoV forward, three wide-FoV lateral/rear), an IMU, and GPS. Extrinsics are calibrated to the ego frame; intrinsics and distortion are estimated via AprilCal. The distinguishing property is trajectory length and continuity — logs are continuous drives rather than curated snippets, which is what enables the study of scalability and extrapolation as first-class axes.
The coverage plot makes the long-tail structure of the dataset explicit: several logs exceed 100 km, and the per-city distribution supports intra-city loop-closure experiments as well as cross-city transfer.
Method: region-aware urban 3DGS
The reconstruction baseline builds on 3D Gaussian Splatting. The scene is a set of anisotropic Gaussians \mathcal{G} = \{g_i\}_{i=1}^N, each parameterized by opacity, mean, rotation, scale, and view-dependent color (SH). Rendering alpha-composites projected Gaussians in depth order:
C = \sum_{i \in \mathcal{N}} c_i\, \alpha_i \prod_{j=1}^{i-1}(1 - \alpha_j).
Vanilla 3DGS collapses in urban captures for well-known reasons: (i) dynamic vehicles and pedestrians pollute the static reconstruction; (ii) unbounded sky induces floaters and misuses SH capacity; (iii) large planar road surfaces are under-constrained by photometric loss alone; (iv) camera poses from GPS/IMU/odometry are only approximate. WildCity’s baseline addresses these with a region-aware pipeline driven by semantic masks.
Three mask streams gate the loss and the geometry priors:
- Dynamic-object masks, refined by 3D tracking cuboids so that parked cars (stationary instances of a movable class) are retained. This distinction matters: naive class-level masking of vehicles removes a large fraction of static appearance in urban scenes.
- Sky masks, used to route those pixels to a dedicated far-field sky model rather than allowing Gaussians to drift to infinity.
- Ground masks, used to apply road-surface regularization on planar regions where photometric gradients are weak.
Because masks are auto-generated, the authors validate them against manual annotations.
The visual agreement is corroborated by the IoU numbers reported in Table 2 of the paper, and it justifies pushing the auto-mask pipeline to full-scale logs where manual labeling is infeasible.
Experiments
Evaluation uses two sequences under the same sensor rig at contrasting scales: Ann Arbor-0.5k (local, sub-kilometer) and Atlanta-5k (long-horizon, 5 km class). Metrics are PSNR, SSIM, and LPIPS on static regions defined by the semantic mask, plus Depth L1 (meters) on pixels with valid LiDAR returns. VGGT-based baselines are aligned to the metric LiDAR point cloud via an optimized global similarity transform so that Depth L1 is comparable. Training is on H200 hardware, run until validation stabilizes.
The paper frames results around three failure modes rather than a single leaderboard number:
- Scalability. Methods that are competitive on the 0.5 km sequence degrade as the trajectory extends to 5 km, both in photometric metrics and in Depth L1. The gap between local and long-horizon performance is the empirical signature of the problem the dataset is designed to expose.
- Extrapolation. When rendering from viewpoints displaced from the training trajectory (e.g., lateral shifts for closed-loop simulation), quality drops sharply — the standard failure mode of overfit 3DGS on narrow driving corridors.
- Uncertainty. The reconstructions lack calibrated uncertainty, which is problematic for downstream policy learning that must decide when to trust the simulator.
Limitations and open questions
The baseline is a competent urban-3DGS pipeline, not a solution: extrapolation error and scalability degradation remain open. Pose noise is preserved rather than corrected — the dataset does not ship refined SfM poses at city scale, which is itself an open problem. The closed-loop simulator inherits any residual dynamics leakage and view-dependent artifacts of the underlying reconstruction. Semantic masks are IoU-validated but not perfect; propagation of mask errors into geometry (especially at ground/vehicle boundaries) is not quantified in the sections provided. Finally, evaluation on two sequences characterizes the axes of difficulty but is not yet a broad benchmark; a leaderboard spanning all 18 logs would sharpen method comparisons.
Why this matters
City-scale reconstruction is the missing substrate for embodied AI research on spatial memory, long-horizon navigation, and closed-loop driving evaluation. WildCity operationalizes this by pairing 83.7-km-average logs with a region-aware 3DGS baseline and by explicitly benchmarking scalability, extrapolation, and uncertainty rather than only static-view PSNR — the axes that actually determine whether a reconstruction is simulation-ready.
Source: https://arxiv.org/abs/2607.06838
Single-Rollout Asynchronous Optimization for Agentic Reinforcement Learning
Problem
RL post-training for agentic LLMs (long tool-using trajectories, 100k+ tokens, many turns) is bottlenecked by synchronous group-based samplers such as GRPO. GRPO requires G rollouts per prompt to estimate a group-relative baseline; the slowest trajectory in a group determines the step latency, which is catastrophic when trajectory lengths vary by orders of magnitude. Asynchronous alternatives — where each finished trajectory is consumed immediately — recover throughput but expose two problems: (i) group-relative advantage estimators no longer fit naturally, and (ii) the rollout policy drifts arbitrarily far from the training policy because the engine may be updated many times during a single generation, breaking standard PPO-style importance sampling that assumes a single tractable \pi_{\theta_\text{old}}.
Method
SAO combines three ingredients: single-rollout sampling with a learned value model, direct double-sided importance sampling (DIS) for token-level trust-region control, and length-adaptive GAE.
Single rollout. Group size G=1: one trajectory per prompt. This eliminates the head-of-line blocking from long trajectories in a group and removes the need for a group-relative baseline. To retain a variance-reduced advantage, SAO trains an explicit value model V_\phi (initialized from the SFT policy) with learning rate 5\times10^{-6}, GAE \lambda_\text{critic}=1, and K=2 value updates per batch to keep the critic ahead of the fast-drifting policy.
Direct double-sided importance sampling (DIS). Because \pi_{\theta_\text{old}} is not well-defined under many rollout-engine updates per trajectory, SAO drops it and uses the rollout log-probabilities (already available from the inference engine) as the behavior proxy:
r_t(\theta) = \exp\bigl(\log\pi_\theta(a_t\mid s_t) - \log\pi_\text{rollout}(a_t\mid s_t)\bigr).
The trust region is enforced by a strict two-sided mask [1-\epsilon_\ell, 1+\epsilon_h]: any token with r_t(\theta) outside this interval is entirely removed from the gradient (rather than PPO’s asymmetric clip that only zeros the gradient conditional on the sign of \hat A_t). The objective is
L(\theta) = \hat{\mathbb{E}}_t\bigl[f(r_t(\theta), \epsilon_\ell, \epsilon_h)\,\hat A_t \log\pi_\theta(a_t\mid s_t)\bigr],
with f acting as a hard mask over the trust region. This is similar in spirit to IcePop but avoids maintaining old-policy inference. For math with tool calls, \epsilon_\ell=0.3, \epsilon_h=5.0; for coding agents, \epsilon_\ell=0.8, \epsilon_h=3.0 — the asymmetry reflects the empirical distribution of r_t(\theta) under strong rollout-training drift.
Length-adaptive GAE. Policy uses \lambda_\text{policy}=1-\frac{1}{\alpha l} with \alpha=1.5, scaling with trajectory length l so long rollouts do not collapse to Monte Carlo returns.
Training uses batch size 128 (so GRPO baselines use 16 prompts × 8 rollouts), policy LR 1\times10^{-6}, and 128k-token context.
Results
Backbone is Qwen3-30B-A3B-Thinking-2507, SFT’d on GPT-OSS-120B-generated tool-integrated reasoning traces before RL.
On math reasoning with Python tools (Pass@1, averaged over 16 runs for most benchmarks):
- AIME2025: SFT 80.4, GRPO 84.2, SAO 97.3 (GPT-5 High 94.6, GLM-4.7 95.7).
- BeyondAIME: SFT 53.3, GRPO 54.8, SAO 74.8 (GPT-5 High 74.0).
- HMMT Nov 2025: SFT 75.2, GRPO 76.0, SAO 88.3.
- IMOAnswerBench: SFT 53.3, GRPO 55.8, SAO 74.0.
Ablations isolate the contributions: GRPO + DIS reaches 93.5 / 70.8 / 84.0 / 70.0 on the four benchmarks, and SAO with DIS only (no length-adaptive GAE or fast-value updates) reaches 94.2 / 71.5 / 86.7 / 71.3. The remaining ~3 point gap to full SAO comes from single-rollout + value-model design; the bulk of the gap from vanilla GRPO comes from DIS.
On SWE-Bench Verified with OpenHands scaffold (300 turns max): Qwen3-30B-A3B 23.0, +GRPO(DIS) 27.0, +SAO 29.8.
The paper claims stable training over 1000 steps, whereas asynchronous GRPO variants collapse without the strict two-sided mask. The training curves show SAO’s advantage is not merely a final checkpoint effect — it holds throughout optimization.
Limitations and open questions
- The comparison holds total batch size fixed at 128, but SAO uses 128 unique prompts vs GRPO’s 16 prompts × 8 samples. Part of SAO’s gain is therefore attributable to prompt diversity per step, not just to asynchrony or clipping; the two are entangled in the reported ablations.
- Dropping \pi_{\theta_\text{old}} is theoretically aggressive: r_t(\theta) = \pi_\theta/\pi_\text{rollout} conflates off-policy correction and rollout-training mismatch into a single term, and correctness relies on the hard mask trimming any token where this approximation is poor. The chosen \epsilon_h = 5.0 is very loose, suggesting most tokens are trusted; behavior under smaller trust regions is not characterized.
- Value-model quality on 128k-context long-horizon agentic trajectories is notoriously fragile, and no critic diagnostics (explained variance, TD error) are reported.
- Only Qwen3-30B-A3B is studied; whether DIS-style mask thresholds transfer to dense models or to shorter-horizon RLHF is unclear.
Why this matters
Asynchronous RL for long-horizon agents has, until now, been a systems win with unclear algorithmic footing. SAO shows that a single-rollout critic-based setup with an aggressive two-sided token mask on \pi_\theta/\pi_\text{rollout} is sufficient to train stably for 1000+ steps and to push a 30B open model past GPT-5 High on AIME2025 and BeyondAIME in the tool-augmented setting — evidence that the group-relative baseline is not essential and that policy-lag correction can be reduced to a hard trust-region mask.
Source: https://arxiv.org/abs/2607.07508
Hacker News Signals
Cloudflare Meerkat - Globally distributed consensus
Meerkat is Cloudflare’s internal globally distributed consensus system, described in a technical introduction post. The core problem: existing consensus protocols (Raft, Multi-Paxos) assume a single datacenter or a small cluster with sub-millisecond links. Cloudflare needs strong consistency across hundreds of PoPs spanning continents, where round-trip times are hundreds of milliseconds. Meerkat’s approach separates leader election (infrequent, can tolerate latency) from the steady-state log replication path. The post describes a hierarchical quorum design: regional clusters form sub-groups that each run a local consensus round, with a global coordinator aggregating regional decisions. This reduces the critical path from O(\text{global RTT}) to O(\text{regional RTT}) for common cases. Failure handling promotes a regional leader to global coordinator using a Paxos-style lease mechanism. The post does not publish formal proofs or detailed pseudocode, but the architecture resembles Flexible Paxos with geography-aware quorum assignment. Latency numbers are not published but the post claims p99 commit latencies acceptable for distributed locking and configuration storage use cases. The main open question is how the system behaves under network partitions that split regional clusters unevenly, which the post does not address in depth.
Rewriting Bun in Rust
The Bun team published a post explaining a partial migration of Bun’s codebase from Zig to Rust. Bun was originally written in Zig for fine-grained memory control and compile-time features, but the post cites several practical problems: Zig’s toolchain instability across versions, limited ecosystem for safe concurrent data structures, and difficulty onboarding contributors familiar with systems languages. The migration is not a full rewrite — the JavaScript engine (JavaScriptCore via C++) and hot paths remain in Zig; Rust is being introduced for new subsystems and for rewriting components where ownership semantics map cleanly to Rust’s borrow checker. The post is candid about interop costs: Zig-to-Rust FFI requires careful ABI alignment and the team uses #[repr(C)] structs extensively at boundaries. Build system integration uses a custom Makefile layer that compiles both languages and links the resulting static libraries. Performance regression testing is described qualitatively rather than with published benchmarks, which is a gap. The technically interesting takeaway is that the team found Rust’s async/await and tokio a better fit for I/O-heavy subsystems than Zig’s async, which is being deprecated upstream. This is a case study in incremental language migration in a performance-critical runtime rather than a ground-up rewrite.
Source: https://bun.com/blog/bun-in-rust
30papers.com – Ilya’s 30 essential ML papers, in a beginner friendly format
This site presents structured summaries of the approximately 30 papers Ilya Sutskever reportedly recommended as foundational ML reading — a list that circulated after his OpenAI departure. The papers span attention mechanisms, scaling laws, contrastive learning, RL from human feedback, and early deep learning theory. Each entry includes an accessible explanation, key equations rendered legibly, and links to the original PDFs. The technical value for a PhD-level reader is modest as a primary resource, but the curation is useful as a reference for what Sutskever considers load-bearing intellectual foundations of modern large-model work. The list skews toward papers that established conceptual primitives (e.g., the original Transformer, “Attention is All You Need”; Sutskever’s own sequence-to-sequence work; the AlexNet paper) rather than recent engineering papers. Notably absent: anything about mixture-of-experts scaling, retrieval-augmented generation, or inference-time compute — reflecting that the list predates those as central concerns. The site is a pedagogical artifact; its value is in aggregation and accessibility rather than novel analysis.
Source: https://30papers.com/
Microsoft Flint: a visualization language for AI agents
Flint is a domain-specific language and renderer from Microsoft Research designed for AI agents to produce structured, interactive visualizations rather than raw text or markdown tables. The core idea is that LLMs are poor at generating correct Vega-Lite or D3 JSON from scratch due to verbose syntax and tight semantic constraints, but they can emit a simpler declarative spec that Flint compiles down to a chart. The language exposes chart primitives (bar, line, scatter, etc.) with typed fields, and the renderer handles layout and interactivity. The technical implementation uses a grammar defined in TypeScript with a parser that validates field bindings against an inferred schema from the data payload passed alongside the spec. From an agent perspective, the output token budget for a valid Flint spec is substantially smaller than equivalent Vega-Lite JSON, which reduces both hallucination surface area and latency. The GitHub Pages demo shows an agent-in-the-loop workflow where the LLM emits a Flint block, the runtime renders it inline in a chat UI, and the user can ask follow-up queries that mutate the visualization state. Limitations: Flint currently supports a restricted chart vocabulary and has no support for geospatial or network graph types. Whether the abstraction generalizes to complex dashboards is an open question.
SWE-1.7: Near GPT-4.5 and Claude Opus Intelligence
Cognition’s SWE-1.7 is an update to their Devin-derived software engineering agent, claiming benchmark scores approaching GPT-4.5 and Claude 3 Opus on SWE-bench Verified and related coding evaluations. The post describes improvements to the agent’s long-context task execution: specifically, a revised memory architecture that maintains a compressed representation of repository state across tool calls, reducing context window thrashing on large codebases. The scaffolding uses a hierarchical task decomposition — a high-level planner emits subtask specs, and lower-level execution agents handle file edits, test runs, and error triage. The post reports 53.6% on SWE-bench Verified, compared to Devin 1.0’s earlier ~13% and GPT-4o’s ~49% (these numbers have been in flux as the benchmark itself has evolved). Key technical detail: the team uses a fine-tuned backbone rather than pure prompting, trained on trajectories of successful bug fixes labeled with intermediate reasoning steps. The limitation is that SWE-bench Verified tests a narrow distribution of GitHub issues — mostly Python, mostly well-scoped bugs — and does not measure performance on greenfield development, refactoring, or cross-language tasks, which are harder and less saturated.
Postgres rewritten in Rust, now passing 100% of the Postgres regression tests
The pgrust project is a ground-up reimplementation of PostgreSQL in Rust, with the headline claim of passing the full Postgres regression test suite. The approach is a clean-room rewrite rather than a transpilation: the author reimplements the storage engine, query executor, parser, and wire protocol independently, using the regression tests as a conformance target. The Rust implementation leverages tokio for async I/O and uses Rust’s type system to enforce invariants around buffer pool management and transaction isolation that in the C codebase are enforced only by convention and code review. The storage layer implements the same heap file format as Postgres, enabling data file compatibility — a significant constraint that rules out storage-level redesign. The repository is in early stages; passing regression tests is a correctness baseline, not a performance claim. The project does not yet implement extensions, logical replication, or many contrib modules. The practical value at this stage is as a research artifact and a test of how much of Postgres’s correctness surface can be captured by the type system alone. The main engineering question is whether the async I/O model will require fundamental changes to how locks and latches are managed across await points — a known hard problem in async Rust for database workloads.
GPT-Live
OpenAI’s GPT-Live is a real-time conversational interface that integrates the Realtime API with persistent memory and visual input over a live video stream. Technically, it extends the existing gpt-4o-realtime model with a streaming video input path: the client sends H.264 frames at reduced resolution alongside the audio stream, and the model processes interleaved audio-visual tokens to enable visually-grounded conversation. The architecture reuses the multimodal tokenization from GPT-4o’s vision pipeline but must operate under strict latency budgets — the post implies sub-500ms end-to-end response latency targets. Memory is handled via a retrieval layer that injects compressed summaries of past sessions into the system prompt, similar to the existing ChatGPT memory feature but scoped to the live interaction context. The main technical challenge is managing the token budget when simultaneous audio, video, and memory context compete for context window space; the system appears to use dynamic frame sampling that drops visual frames during high speech-activity periods. From a systems perspective, the client SDK handles the media pipeline, and the server side likely runs the same speculative decoding infrastructure used for gpt-4o-mini to meet latency targets. Limitations: the video path is currently constrained to a narrow field of view and does not support document-quality OCR mid-conversation.
Noteworthy New Repositories
Dicklesworthstone/franken_ocr
A pure-Rust, CPU-only inference engine targeting Baidu’s Unlimited-OCR model, a 3B-parameter Mixture-of-Experts Vision-Language Model derived from DeepSeek-OCR. The project’s core premise is zero external ML framework dependency: no PyTorch, no ONNX Runtime, no GPU requirement. Custom int4 and int8 quantization kernels are written directly in Rust, keeping the binary self-contained and portable to any x86-64 or ARM machine. The single-model design is a deliberate constraint — one fixed set of weights, one fixed tokenizer, no configurability overhead.
The codebase is currently pre-Phase-0, meaning the scaffold structure is established but end-to-end inference is not yet functional. What exists is the architectural skeleton: tensor primitives, quantized weight loading stubs, and the planned pipeline for feeding image patches through the MoE routing logic. The int4 kernel design matters here because MoE models with sparse expert activation are memory-bound; int4 packing reduces the effective bandwidth requirement enough to make CPU inference plausible at 3B scale.
The target use case is air-gapped or resource-constrained deployments where pulling in a Python stack or requiring NVIDIA hardware is a non-starter — embedded systems, secure enclaves, CI pipelines doing document processing. Once functional, the Rust-native approach should also offer deterministic latency absent GPU scheduling variance. Follow this if you need OCR inference in a Rust binary with no runtime dependencies.
Source: https://github.com/Dicklesworthstone/franken_ocr
NotASithLord/peerd
Peerd implements a full agent loop as a browser extension, with no backend server and no telemetry. The architecture is genuinely different from most agent frameworks: instead of a remote orchestrator calling browser-automation APIs, the agent runs inside the extension process itself and directly drives tabs via the standard WebExtensions API. Compute sandboxes are spun up client-side — JavaScript notebooks for scripted data work, and WASM-compiled Linux VMs for heavier tasks that would normally require a server.
The peer-to-peer sharing layer lets the agent distribute artifacts it builds (notebooks, small apps, processed data) directly to other browser peers without a central host, using WebRTC or a similar browser-native transport. BYOK (bring your own key) means LLM calls go directly from the browser to the provider endpoint; the extension never sees credentials server-side because there is no server side.
The engineering interest is in the sandboxing model: running untrusted agent-generated code inside a WASM Linux VM that itself runs inside a browser extension process creates a layered isolation boundary without requiring OS-level containers. The main limitations are the constraints of the WebExtensions API itself — cross-origin restrictions, tab permission prompts, and memory limits on the extension background worker. This is an interesting existence proof that agent infrastructure does not inherently require cloud compute, and worth watching for anyone thinking about privacy-preserving or offline-capable agent deployments.
Source: https://github.com/NotASithLord/peerd
eli-labz/Third-Eye
Third-Eye is a production-oriented OSINT aggregation platform that consolidates multiple intelligence domains — social media, domain/IP intelligence, public records, and related feeds — into a unified situational-awareness interface. The “production-grade” framing suggests it targets security operations teams and researchers who need to correlate signals across sources rather than query each tool individually.
The platform architecture appears to follow a modular collector design: each intelligence domain is a separate ingestion module that normalizes output into a common schema, enabling cross-domain correlation queries. This is the standard approach for OSINT tooling at scale, but the value is in implementation quality — how reliably collectors handle rate limits, API changes, and data format drift from upstream sources.
Practical use cases include threat actor attribution (correlating infrastructure across registrar data, BGP announcements, and social identifiers), incident response triage (rapidly enumerating exposure surface for a given organization or IP range), and competitive or geopolitical monitoring. The high star count for a relatively new repository suggests it filled a gap — most existing OSINT frameworks are either single-domain CLI tools or expensive commercial products. Third-Eye occupies the middle ground: multi-domain, open-source, and structured for operational use rather than ad-hoc querying. Worth evaluating if you need a self-hosted OSINT backbone that can be extended with custom collectors.
Source: https://github.com/eli-labz/Third-Eye
SmileLikeYe/agent-chief
Agent-Chief addresses a specific failure mode of multi-agent systems: notification fatigue and the lack of a unified interrupt-or-continue decision layer. As the number of running agents, alert sources, and data feeds increases, the cognitive load on the operator grows super-linearly unless something aggregates and prioritizes. Chief positions itself as that local-first aggregation layer.
The core mechanism appears to be a routing and triage engine that ingests events from agents and feeds, applies a priority model, and emits a binary decision: interrupt the user or suppress. The “local-first” constraint means the triage logic runs on the operator’s own machine, avoiding both latency from round-tripping to a cloud classifier and the privacy concern of forwarding all agent activity to an external service.
The practical architecture likely involves a lightweight event bus where agent outputs post structured messages, a configurable priority model (rule-based, ML-scored, or both), and a unified notification surface. The hardest design problem here is calibrating the interrupt threshold — too aggressive and you replicate the original noise problem, too conservative and urgent signals are missed. Open questions include how the priority model is trained or calibrated per user, whether it handles temporal context (urgency decay over time), and how it integrates with existing agent frameworks like LangChain or AutoGen. Useful for anyone running more than a handful of concurrent agents and needing operator-in-the-loop control without drowning in pings.
Source: https://github.com/SmileLikeYe/agent-chief
MihaiCiprianChezan/Agentic-First-Enterprises
This repository is a reference architecture and operating model document rather than a runnable codebase, targeting organizations that want to systematically replace or augment human roles with AI agents. The central design constraint is that every process must be built agent-first — meaning the workflow specification, data interfaces, and handoff protocols are designed assuming an agent will execute them, with human involvement as an optional override rather than the default path.
The “pause-adjust-resume” pattern is the most technically interesting concept here: every agentic workflow must support a checkpoint-and-handoff mechanism that lets a human inspect intermediate state, modify it, and return control to the agent without restarting from scratch. This requires stateful workflow representations rather than fire-and-forget task invocations, pushing toward something like durable execution frameworks (Temporal, Restate) as the underlying infrastructure.
The role-agnostic framing — any role can be human or agent — implies the process definitions must be abstract enough that the executor identity is a runtime parameter, not a design-time assumption. This has implications for how you model authorization, accountability, and audit trails. The repository is most useful as a structured thinking framework for enterprise architecture teams evaluating agent adoption, rather than as deployable software. It fills a gap between high-level AI strategy literature and concrete implementation guides, though the lack of runnable reference implementations limits its immediate engineering utility.
Source: https://github.com/MihaiCiprianChezan/Agentic-First-Enterprises
uzairansaruzi/hermex
Hermex is a native iOS application that provides a mobile interface for interacting with a Hermes-based AI agent. The Hermes framing likely refers to a locally-run or self-hosted LLM instance using one of the NousResearch Hermes model family variants, which are fine-tuned for instruction-following and tool-use. The native iPhone implementation distinguishes this from web-based or React Native wrappers — UIKit or SwiftUI rendering, direct access to iOS system APIs, and the performance characteristics of compiled Swift code.
The technical interest is in how it bridges the iOS sandbox with an agent that needs to invoke tools. Mobile agent applications face hard constraints: no persistent background processes, strict network permission requirements, and limited local compute for anything beyond model quantization at very small scales. Hermex most likely operates in a client-server mode where the Hermes agent runs on a home server or cloud instance and the iOS app provides the conversational UI and tool-result rendering, rather than running the model on-device.
If local inference is involved, it would require Core ML conversion of the Hermes weights or use of llama.cpp via a C bridge, which imposes significant memory pressure on device. The peer-to-peer or BYOK model implied by the description suggests the backend is user-controlled. Worth watching for anyone building mobile-native interfaces to self-hosted LLM agents, particularly for the UI patterns around streaming responses and tool-call visualization on small screens.
Source: https://github.com/uzairansaruzi/hermex
simonlin1212/Vibe-Research
Vibe-Research is a personal trading research agent covering A-share (mainland China), US, and Hong Kong equity markets. It provides six functional modules: daily review (post-session recaps), information radar (news and announcement monitoring), individual stock data (fundamentals, price data), sector center (sector rotation analysis), portfolio tracking, and research notes. The “vibe-research” framing positions it as an AI-driven alternative to subscribing to brokerage research.
The architecture is agent-driven: rather than static dashboards, each module is backed by an LLM agent that can query data sources, synthesize information, and generate natural-language analysis on demand. The multi-market scope is technically non-trivial — A-share data requires different API integrations (Tushare, AKShare, or similar) than US equities (yfinance, Polygon, Alpha Vantage), and regulatory data formats differ significantly across markets.
The most challenging component is the daily review module, which requires ingesting session data, cross-referencing news events with price movements, and producing coherent attribution analysis — a task that pushes the limits of current LLM reliability for quantitative reasoning. The portfolio module adds a personalization layer: the agent can contextualize market analysis against the user’s actual holdings, which requires maintaining persistent state across sessions. BYOK is implied by the local-first design. Useful for individual traders and researchers who want an AI assistant that understands multi-market context without paying for Bloomberg Terminal-tier infrastructure.
Source: https://github.com/simonlin1212/Vibe-Research
dongshuyan/compass-skills
Compass-Skills (司南) is a personalized task orchestration layer for AI agents, framed as a “Personal Alignment Skills OS.” The core idea is that different users have different working styles, priorities, and domain contexts, and a generic agent task router does not account for this — compass-skills provides the personalization substrate that aligns agent behavior to individual preferences and workflows.
The “skills” abstraction appears to be a modular capability registry: discrete, composable task handlers that the orchestrator can assemble into workflows based on the current goal and user context. This is similar in spirit to the skills layer in Microsoft’s Semantic Kernel, but with heavier emphasis on the personal-alignment dimension — the system learns or is configured to know which skill combinations suit which user in which situations.
The “总控” (general control) framing suggests a meta-orchestrator pattern: compass-skills sits above individual agents or tools and decides how to decompose tasks, which skills to invoke, and in what order, rather than being a skill itself. The personalization mechanism is the open question — whether this is explicit configuration, learned preference modeling, or a combination. For developers building multi-agent systems that need to serve users with heterogeneous working styles, this kind of personalization layer is a real missing piece in most current frameworks. The Chinese-language documentation and A-share focus in related projects suggest this is primarily targeting the Chinese developer ecosystem, but the architectural concepts transfer broadly.