by Tristan Matthews

As part of the RISE Project’s ongoing mission to ensure high-performance, production-ready multimedia libraries on RISC-V hardware, work under initiative RP020 focused on bringing vector optimizations and modern validation infrastructure to SpeexDSP.

Through a multi-stage approach encompassing build system modernization, integration of automated testing harnesses, and hand-written RISC-V Vector (RVV) assembly, SpeexDSP now delivers optimized real-time audio processing performance across RISC-V platforms.

Background & Real-World Impact of SpeexDSP

Originally created by Jean-Marc Valin as part of the Speex speech codec released in 2003, SpeexDSP was spun off as an independent helper library with version 1.2.0 in 2016. It serves as a foundational component for real-time audio applications—particularly Voice over IP (VoIP) and system-level audio routing.

SpeexDSP provides critical signal processing primitives, including an audio resampler, acoustic echo canceller, jitter buffer, and preprocessor supporting noise suppression, automatic gain control (AGC), and voice activity detection (VAD) in both floating-point and fixed-point builds. This allows it to be deployed on a wide range of hardware, including VLC Media Player, Firefox, Wireshark, PulseAudio, Mumble, Asterisk and many others.

Due to its heavy use of vector math, SpeexDSP is particularly well suited for SIMD (Single Instruction, Multiple Data) optimization, which allows processors to perform the same operation on multiple data points simultaneously. Earlier optimization targets included x86 SSE/SSE2 (using intrinsics) and ARM NEON/aarch64 (using GCC inline assembly) for the resampler.

However, as there was no standardized benchmark or validation harness for these implementations, adding additional hardware-specific optimizations was deemed too risky and unwieldy. Recognizing this gap, Google engineers—working in collaboration with the RISE (RISC-V Software Ecosystem) project—embarked on an initiative to modernize the codebase, establish robust testing harnesses, and deliver high-performance RISC-V Vector (RVV) implementations.

Addressing Legacy SIMD Gaps with Meson and Checkasm

To establish a verified baseline for RISC-V optimizations, the RP020 team first introduced the Meson build system alongside existing Autotools and CMake configurations. Meson provided seamless integration with Checkasm—a specialized validation and benchmarking harness originally developed in FFmpeg/dav1d and maintained as a standalone project by Niklas Haas and Martin Storsjö.

Adding checkasm coverage for existing SIMD code bases immediately yielded unexpected dividends. While verifying correctness, we discovered that in some legacy configurations, the highly-optimized SIMD code was actually slower than standard, scalar C!

Specifically, we uncovered an 8-year-old data dependency bug in the SSE2 intrinsic code for the inner_product_double function. Because of this dependency, the SSE2 code ran at a sluggish 0.58x to 0.75x the speed of scalar C. By refactoring the kernel to use two accumulators instead of one (thereby breaking the instruction dependency chain), performance was restored to 1.04x to 1.07x the speed of C, turning a silent performance regression into an active optimization win.

Hand-Written RVV Assembly Implementation

When evaluating implementation strategies for RISC-V Vector (RVV) support, hand-written assembly was chosen over vector intrinsics. Following patterns established in projects like FFmpeg and dav1d, hand-written assembly provides precise instruction scheduling control and eliminates reliance on compiler codegen variations. We wanted to leverage CPU runtime detection to ensure that the RVV code was run wherever possible (the existing SIMD in SpeexDSP did not use RTCD).

The defining advantage of the RISC-V Vector (RVV) extension is its length-agnostic design. Unlike traditional vector architectures (such as Intel SSE/AVX or ARM NEON) that require rewriting assembly when register widths expand, a single RVV assembly implementation dynamically scales across different hardware designs.

Whether the underlying silicon implements a 128-bit, 256-bit, or even larger Vector Length (VLEN), the same compiled binary runs optimally without recompilation. This represents a massive win for long-term software maintenance and ecosystem portability. To prove this flexibility, we validated the new RVV code across two distinct physical architectures: the Banana Pi BPI-F3 SpacemiT K1 (VLEN=256-bit) and the Canaan Kendryte K230 (VLEN=128-bit).

Benchmark Results on SpaceMIT K1

While comparing hand-written assembly against scalar C is useful, a far more rigorous engineering baseline is comparing against compiler autovectorized C (compiled using modern toolchains like Clang 18 and GCC 13). This confirms whether manual optimization provides a meaningful uplift over modern compilers.

Our benchmarking showed remarkable speedups across the board. In particular, we compared the performance of our hand-written RVV kernels against the best compiler-vectorized C:

• RVV Float resampler kernels: Achieved a 3.71x speedup under Clang 18 and a 3.59x speedup under GCC 13.
• RVV Fixed-point resampler kernels: Achieved a 2.69x speedup under Clang 18 and a massive 4.88x speedup under GCC 13 (where the compiler failed to autovectorize and fell back to scalar C).

More interesting than just comparing against scalar C, we made sure to compare against autovectorized C to ensure there was still a benefit to writing assembly instead of just relying on compilation:

Aside from checkasm, we also wrote end-to-end benchmarks to ensure real-world use cases were represented:

Aside from the resampler, the optimization work extended into the echo canceler (FFT) and pre-processor, yielding these results for bench_echo the end-to-end benchmark:

Thanks to the RISE project’s sponsorship, we’ve been able to significantly optimize the SpeexDSP library for RISC-V while also ensuring that the maintenance and further optimization of this library will be significantly easier and safer going forward.