Data & spreadsheets

Benchmarking C versus Rust for SQLite number parsing extensions

Simon Willison compared C and Rust implementations of a SQLite parse_number() extension, finding optimized C achieved 16-19 ns per row.

Illustration comparing a small C gear and a larger Rust gear processing data tokens
Illustration created for this article

Developer Simon Willison published a technical benchmark on November 8, 2026, comparing the performance and complexity of building a custom SQLite extension in C versus Rust. The study focused on a parse_number() function designed to clean messy string data into numeric values, revealing that a highly optimized C implementation could outperform Rust equivalents in raw speed while maintaining a significantly smaller binary footprint.

What happened

Willison initiated the experiment while walking his dog, curious about the practical differences between writing a SQLite extension in C or Rust. He used an AI assistant to generate initial prototypes for both languages, aiming to create a function that strips currency symbols, commas, and percentage signs from strings like -$1,234.56 or '12.5%'. The goal was to return a valid numeric value for summation and sorting, or NULL if the input could not be parsed. The initial comparison showed that the Rust extensions were larger in file size but surprisingly faster than the naive C implementation, which relied on the standard library’s strtod function.

To address the performance gap, Willison directed the AI to write a custom C parsing function with a compact fast path, bypassing the slower standard library calls for decimal handling. This optimized C version closed the gap and ultimately surpassed the Rust implementations in speed. The final results demonstrated that the new C code processed mixed data in 19 nanoseconds per row, compared to 41-46 nanoseconds for the Rust versions. The experiment concluded with the AI generating a full research report formatted for Willison’s public repository, documenting the code, tests, and performance metrics.

Key details

  • The optimized C extension achieved a processing time of 16–19 nanoseconds per row, roughly twice as fast as the Rust versions.
  • File size differed significantly: the C extension was 14 KB, while Rust versions ranged from 298 KB to 321 KB.
  • Initial plain C code was slower on decimals due to reliance on strtod, but a custom fast path resolved this bottleneck.
  • All four implementations passed 8 million fuzz checks, ensuring consistent behavior and accuracy across edge cases.
  • The Rust implementation using rusqlite took 46 ns for mixed data, while sqlite-loadable-rs took 41 ns.
  • The final C implementation handled integers in 16 ns and decimals in 18 ns, outperforming Rust’s 32-36 ns and 47-55 ns respectively.

Background

SQLite is a widely used embedded database engine that allows developers to extend its functionality through custom functions written in compiled languages like C or Rust. These extensions run directly within the database process, making them ideal for high-performance data transformation tasks such as cleaning imported records. When choosing a language for these extensions, developers often balance ease of development and memory safety against raw execution speed and binary size. C offers minimal overhead and tiny binaries but requires manual memory management, while Rust provides strong safety guarantees at the cost of larger binaries and potentially different runtime characteristics.

The parse_number() function addressed a common data engineering problem: converting human-formatted strings into machine-readable numbers. Standard SQL functions like replace() can strip characters but are often slow when applied row-by-row over large datasets. Custom extensions bypass some of these overheads by operating closer to the metal. However, writing efficient parsers in C requires careful optimization to avoid the pitfalls of general-purpose conversion functions, which may not handle specific formatting quirks efficiently.

Why it matters

For teams managing self-hosted databases or building data-intensive applications, this benchmark highlights the tangible trade-offs between development convenience and operational efficiency. While Rust offers a safer development experience and robust tooling, the performance penalty in this specific micro-benchmark was significant. For high-volume data ingestion pipelines where every nanosecond counts, the ability to squeeze maximum performance out of C can reduce infrastructure costs and improve throughput. However, this comes with the responsibility of managing memory safety and potential bugs manually.

The study also underscores the growing role of AI assistants in low-level systems programming. By leveraging AI to generate and optimize complex C code, developers can access performance levels that might otherwise require deep expertise in systems programming. This democratizes access to high-performance database extensions, allowing smaller teams to implement custom optimizations without hiring specialized C engineers. It suggests that the barrier to entry for writing efficient database plugins is lowering, provided one knows how to guide the tooling effectively.

What you can do

  • Evaluate your current data cleaning pipelines to identify bottlenecks where custom SQLite extensions could replace slower SQL functions.
  • Consider using C for performance-critical extensions if your team has the expertise to manage memory safety and testing rigorously.
  • Use Rust for extensions where development speed, memory safety, and maintainability are higher priorities than raw nanosecond-level performance.
  • Implement comprehensive fuzz testing for any custom parsing logic to ensure it handles unexpected or malformed input gracefully.
  • Benchmark your specific use cases rather than relying on general assumptions, as performance can vary based on data distribution and hardware.
  • Leverage AI tools to prototype and optimize low-level code, but always verify the generated output with rigorous testing and code review.

More news

All news