authored by Keagan Chern, Charles Hong, Sophia Shao, and Alvin Cheung
In December 2025, UC Berkeley’s Sophia Shao and Alvin Cheung won $50k USD in Gemini compute credits from the RISE Project to dedicate research using AI to speed up the RISC-V software ecosystem. They set out to solve one of RISC-V’s biggest bottlenecks: software optimization and kernel translation.
RISC-V adoption is growing across AI, edge, and high-performance systems, bringing more attention to RISC-V Vector (RVV) and its open-standard, vector-length-agnostic design. Yet RVV software support is still catching up: compiler auto-vectorization often falls short of hand-tuned performance, while manually writing optimized kernels requires expertise that is difficult to scale across workloads. Arm Neon, by comparison, has a mature ecosystem of hand-written SIMD kernels.
The translation from Neon to RVV is easy to get only half right. Staying close to the Neon source may preserve correctness but also carry fixed-width assumptions into RVV, leaving much of the target hardware unused. A more aggressive rewrite can run faster, but it also creates opportunities for subtle errors in vector lengths, lane ordering, arithmetic behavior, and tail handling.
SALTyRN addresses both problems in two stages: it first translates and verifies the kernel, then optimizes the verified RVV baseline and checks the final result again. Across 35 XNNPACK microkernels, the optimized output reached a 1.40× geometric-mean speedup over handwritten RVV.
The same computation, but a different execution model
To see why Neon-to-RVV translation is more than swapping intrinsic names, consider a simple kernel that multiplies an array of 32-bit floats by a constant.
Neon uses fixed 128-bit vectors, so each instruction processes four floats. Any remaining elements require a separate tail path:
while (remaining >= 4) { load 4 floats; multiply 4 floats; store 4 floats; remaining -= 4; } handle the final 1–3 elements; |
RVV instead determines the active vector length at runtime, letting each iteration process as many elements as the hardware supports:
while (remaining > 0) { vl = vsetvl(remaining); load vl floats; multiply vl floats; store vl floats; remaining -= vl; } |
The final iteration naturally receives a smaller vector length(vl), allowing the same loop to handle the tail and adapt to the processor’s hardware vector length.
A correct port must also reconsider pointer updates, tails, lane operations, floating-point behavior, widening and narrowing conversions, and RVV’s element-width and register-grouping rules. Many Neon intrinsics also have no direct one-to-one RVV equivalent.
There is a performance trap as well. A translation that stays too close to Neon may remain limited to four-float chunks even when the RVV processor can handle many more. Preserving the computation does not automatically mean using the target architecture effectively.
A two-phase pipeline: verify the port, then optimize it
SALTyRN separates correctness-oriented translation from target-oriented optimization. Phase 1 generates a RVV, repairs compilation and runtime failures, and checks it against a Neon source using bounded symbolic equivalence. Phase 2 starts from that verified baseline, searches for a faster implementation tuned to the target hardware, and re-verifies the winner.
This separation lets translation favor code whose behavior is easier to preserve, while optimization can make larger structural changes with the verified baseline as a correctness reference.
Phase 1: getting to a verified RVV baseline
The LLM receives the Neon kernel together with structured information about Neon and RVV semantics, including lane behavior, floating-point arithmetic, widening and narrowing, and legal element-width and LMUL combinations. The model reproduces the Neon kernel’s behavior using RVV’s execution model and emits a candidate RVV implementation. That candidate then enters the first validation gate.
First gate: does it compile and run?
SALTyRN first compiles the candidate through Zephyr RTOS toolchain and runs it on Spike, a RISC-V functional simulator. This catches invented intrinsics, illegal RVV configuration, incorrect function signatures, and runtime failures. Errors are returned to the LLM for repair until the candidate runs successfully or reaches the retry limit.
Second gate: is behavior perfectly equivalent?
SALTyRN then executes the Neon and RVV kernels symbolically. Inputs represent all values of their modeled types, while symbolic versions of the intrinsics construct SMT expressions for loads, arithmetic, lane operations, and stores.
The solver is then asked a counterexample question:
Is there any input that makes the Neon and RVV outputs differ?
A counterexample guides another semantics repair attempt. If none exists, the candidate is accepted as equivalent within the bounded input size, VLEN, and parameters. This becomes the verified RVV baseline for Phase 2.
Phase 2: turning correct RVV into fast and correct RVV
SALTyRN extends Autocomp with an RVV backend targeting Saturn. The LLM can combine strategies from the optimization menu or propose a kernel-specific idea during planning. Representative strategies include:
- Vector utilization and configuration: tune LMUL, reduce instruction count, and move vsetvl out of inner loops where possible.
- Register reuse: apply register blocking and multiple accumulators to reduce reloads, hide latency, and avoid spills.
- Memory and data movement: prefer unit-stride or segmented accesses and use slides instead of expensive gathers when possible.
- Instruction selection: use immediate and scalar-vector forms, widening multiply-accumulate, and saturating narrowing operations where appropriate.
- Scheduling and dependencies: overlap memory, vector arithmetic, and scalar bookkeeping; restructure reductions; and use agnostic policies when old destination values need not be preserved.
- Different kernels need different combinations. Elementwise kernels may mostly benefit from larger LMUL and fewer vsetvl operations, while convolution and GEMM kernels may require register blocking, multiple accumulators, data reuse, and careful scheduling.
How does the search choose an optimization?
SALTyRN uses a bounded beam search. At each iteration, it retains a small set of promising implementations and asks the LLM to generate several plans for each one.
Candidates that fail compilation or representative functional checks against the verified baseline are removed. The survivors are measured on Saturn through FireSim, and only the fastest remain in the beam. In the evaluation, the beam size is four, with four plans per candidate over eight iterations.
This search is measurement-driven. Increasing LMUL may reduce loop iterations but increase register pressure; unrolling may expose parallelism but cause spills; instruction reordering may improve overlap or lengthen a dependency chain. FireSim determines whether a proposed optimization actually helps. Running SMT verification on every candidate would be too expensive, so intermediate candidates use faster functional checks. The winner is then symbolically re-verified against the verified RVV baseline.
Results
We evaluated SALTyRN on 35 XNNPACK microkernels spanning elementwise operations, reductions, pooling, conversions, convolution, and GEMM-style workloads. Each had both a Neon implementation and an expert-written RVV baseline. Translation and optimization used Gemini 3.1 Pro, while performance was measured cycle-accurately through FireSim on a Saturn vector unit with a 512-bit VLEN, a 256-bit datapath, and a Shuttle scalar core.
All 35 kernels were translated and passed bounded equivalence checking across the configurations and growing input sizes. Each configuration fixes VLEN and relevant parameters while leaving input values symbolic. Simpler kernels completed broader loop and tail cases, while convolution and GEMM produced more expensive solver queries.
How much did optimization help?
The following graph compares each implementation with handwritten XNNPACK RVV. Each per-kernel value is the geometric mean across evaluated batch sizes, and the dashed line marks 1× parity.
- SALTyRN (Translated): verified Phase 1 output
- SALTyRN (Optimized): final Phase 2 output
- Autovec: GCC auto-vectorization of scalar C by identifying vectorizable loops in scalar C and generating RVV instructions.
- Neon2RVV: mechanical Neon-RVV intrinsic mapping, only supports VLEN=128
The direct translations achieved 0.78× handwritten RVV performance by geometric mean. After optimization, that rose to 1.40×. Some translated kernels were already competitive, while others required substantial RVV-specific restructuring.
Autovec achieved only 0.25× handwritten RVV performance, making optimized SALTyRN 5.6× faster by geometric mean. Neon2RVV produced working RVV code but retained Neon’s 128-bit execution granularity, leaving much of Saturn’s 512-bit vector width unused. Across its 32 supported kernels, optimized SALTyRN achieved a median speedup of 8.9×, ranging from 1.0× to 91.5×.
These comparisons show the limits of current automated approaches: auto-vectorization may miss hand-tuned SIMD structure, while mechanical mapping preserves Neon too literally. SALTyRN preserves the source behavior while allowing the implementation to be reorganized for RVV.
Upstream impact on XNNPack
This work has already produced three merged XNNPACK contributions:
- PR #9879 added a new RVV f32-vmulcaddc-minmax microkernel.
- PR #9974 added a new RVV f16-vmulcaddc microkernel.
- PR #9996 optimized the RVV x32-transposec microkernel, achieving a 1.25× geometric-mean speedup and up to 1.54× on Saturn.
The first two expanded XNNPACK’s RVV coverage, while the third showed that the optimization workflow could improve an existing upstream implementation. All 35 optimized kernels are available in the optimized RVV kernels repository
Conclusion and Future Work
Cross-ISA kernel translation forces a tradeoff between correctness and performance. Staying close to Neon makes behavior easier to preserve but can underuse RVV. Moving farther from the source can unlock performance while increasing the risk of subtle bugs. SALTyRN addresses that tension by first establishing a verified baseline and then using it as a safety net for target-aware optimization.
We plan to continue upstreaming RVV kernels and optimizations into XNNPACK. Longer-term directions include moving beyond bounded verification toward equivalence across arbitrary input lengths and vector configurations, and optimizing at the operator-graph or full-model level, where tensor layouts, fusion, memory traffic, and scheduling can matter as much as any individual microkernel.
The broader goal is to extend the same idea that drives SALTyRN today: establish a trustworthy semantic baseline, then optimize aggressively without giving up correctness.
Acknowledgments
We would like to thank the XNNPack maintainers, in particular Dillon Sharlet and Ken Unger, for their help in preparing and reviewing our upstream contributions, as well as Martin Maas for his continued support. We would also like to thank Google and the RISC-V Software Ecosystem (RISE) project for their support of this work.
References
- SALTyRN — Neon-to-RVV translation and bounded verification pipeline
- Autocomp — LLM-driven optimization framework extended with an RVV backend
- XNNPACK — source microkernels and hand-written RVV baselines
- Saturn — open source RVV-compliant vector processor
- Chipyard and FireSim — infrastructure used for Saturn vector processor evaluation
- RVV — RISC-V’s vector-length-agnostic SIMD extension
- Neon — Arm’s fixed-width 128-bit SIMD extension
