# Benchmarks

Four suites compare ry's runtime with GNU R. Two run code that others wrote: the programs that
R compiler research measures itself on, and the test suites of CRAN packages. The other two run
code we wrote: small numeric kernels, each also written in typed ry, and programs in the shape of
package and business code. Each suite's geometric mean of ry's speed-ups over GNU R:

| Suite | What it is | ry vs GNU R | ry typed |
| --- | --- | ---: | ---: |
| Public R JIT benchmarks | 54 standard programs from R compiler research (Ř, rcp) | 2.90x | not typed |
| Package test suites | 26 pinned CRAN packages, each running its own test suite | 1.93x | not typed |
| Numeric kernels (ours) | 16 small programs we wrote whose time goes into loops, recursion and arithmetic on numbers | 5.26x | 11x |
| Application workloads (ours) | 15 programs we wrote in the shape of package and business code: calls, dispatch, conditions, records, tables | 1.84x | 2.35x |

Geometric means of the speed-ups over GNU R; above 1x means ry is faster.

- Public R JIT benchmarks: measured on 2026-11-11 at ry 4d96a342b with GNU R 4.6.2 on Intel(R) Xeon(R) Processor @ 2.30GHz, 4 cores, Linux 6.18.44-fc-v378; 10 paired rounds.
- Package test suites: measured on 2026-11-11 at ry 46191c4 with GNU R 4.6.2 on Intel(R) Xeon(R) Processor @ 2.30GHz, 4 cores, Linux 6.18.44-fc-v378; 10 paired rounds.
- Numeric kernels (ours): measured on 2026-11-11 at ry 4d96a342b with GNU R 4.6.2 on Intel(R) Xeon(R) Processor @ 2.30GHz, 4 cores, Linux 6.18.44-fc-v378; 10 paired rounds.
- Application workloads (ours): measured on 2026-11-11 at ry 4d96a342b with GNU R 4.6.2 on Intel(R) Xeon(R) Processor @ 2.30GHz, 4 cores, Linux 6.18.44-fc-v378; 10 paired rounds.

Every number on this page is rendered from the results files in `tools/bench/`, and each chart names
the date, the GNU R version, the machine and the ry commit of its measurement.

## Public R JIT benchmarks

These are the programs that the Ř (rir) and rcp R compilers are measured on, from the suite of
Kocourek, Křikava and Vitek, "Copy-and-Patch Just-in-Time Compiler for R" (VMIL 2025): all 57 but
three that cannot be sized to a few seconds. They are vendored unchanged with their MIT licence, and
each file ends with a driver that runs the program once at a fixed size and prints a checksum and
its time. The sizes make GNU R 4.6 take about half a second to two seconds per program. Every run
is a fresh process, so the time includes GNU R's byte compilation and ry's JIT tier-up, though not
start-up.

Public R JIT benchmarks: Are We Fast Yet, speed-up over GNU R:

| program | ry |
| --- | ---: |
| Mandelbrot set, scalar loops (`mandelbrot`) | 2.88x (2.68x to 3.03x) |
| Tree allocation (`storage`) | 2.12x (2.05x to 2.28x) |
| Bouncing balls, named fields (`bounce`) | 1.95x (1.80x to 2.10x) |
| Bouncing balls, positional fields (`bounce_nonames`), wide interval on ry | 1.88x (1.69x to 2.05x) |
| Bouncing balls, positional fields, simple setup (`bounce_nonames_simple`) | 1.82x (1.75x to 1.95x) |

A program marked wide interval varied so much between rounds that its ratio's interval is wide; read its median with care. Measured on 2026-11-11 at ry 4d96a342b with GNU R 4.6.2 on Intel(R) Xeon(R) Processor @ 2.30GHz, 4 cores, Linux 6.18.44-fc-v378; 10 paired rounds.

Public R JIT benchmarks: Benchmarks Game, speed-up over GNU R:

| program | ry |
| --- | ---: |
| Spectral norm (`spectralnorm`), too short on ry | 14x (14x to 15x) |
| Spectral norm, scalar loops (`spectralnorm_naive`), too short on ry, wide interval on ry | 13x (11x to 14x) |
| Reverse complement, whole file (`reversecomplement_2`) | 5.97x (5.48x to 6.55x) |
| Reverse complement (`reversecomplement`) | 4.95x (4.61x to 5.37x) |
| K-nucleotide, named vector counts (`knucleotide_brute_3`) | 4.28x (4.17x to 4.46x) |
| Reverse complement, scalar lookup (`reversecomplement_naive`) | 3.85x (3.62x to 4.06x) |
| K-nucleotide, attribute counts (`knucleotide_brute`) | 3.55x (3.39x to 3.76x) |
| Fasta, scalar linear search (`fasta_naive`), wide interval on ry | 3.50x (3.04x to 3.67x) |
| Fasta-redux (`fastaredux`), wide interval on ry | 3.28x (2.79x to 3.46x) |
| Spectral norm, matrix rebuilt per product (`spectralnorm_alt_2`) | 3.23x (2.94x to 3.41x) |
| Fasta-redux, scalar lookup (`fastaredux_naive`), wide interval on ry | 3.10x (2.91x to 3.51x) |
| Fasta, vectorized binary search (`fasta_3`) | 2.91x (2.62x to 3.16x) |
| K-nucleotide, attribute counts, growing vector (`knucleotide_brute_2`) | 2.85x (2.62x to 3.06x) |
| Fasta, vectorized linear search (`fasta_2`) | 2.61x (2.37x to 2.75x) |
| K-nucleotide, hashed environment (`knucleotide`) | 2.54x (2.42x to 2.69x) |
| Fannkuch-redux, scalar swaps (`fannkuchredux_naive`) | 2.53x (2.33x to 2.58x) |
| Fasta (`fasta`) | 2.48x (2.31x to 2.59x) |
| Fasta, scalar binary search (`fasta_naive_2`) | 2.41x (2.19x to 2.60x) |
| Mandelbrot, scalar points (`mandelbrot_naive_ascii`) | 2.23x (2.14x to 2.40x) |
| Regex DNA (`regexdna`), wide interval on ry | 1.91x (1.78x to 2.19x) |
| N-body, matrix state (`nbody_2`), wide interval on ry | 1.72x (1.47x to 1.80x) |
| Spectral norm, precomputed matrix (`spectralnorm_alt`) | 1.67x (1.62x to 1.79x) |
| Pi digits (`pidigits`) | 1.67x (1.52x to 1.73x) |
| Mandelbrot, vectorized rows (`mandelbrot_ascii`) | 1.55x (1.43x to 1.68x) |
| Mandelbrot, vectorized rows, no output (`mandelbrot_noout`) | 1.45x (1.39x to 1.58x) |
| Spectral norm, eigenvalues (`spectralnorm_math`) | 1.35x (1.24x to 1.37x) |
| N-body (`nbody`) | 1.34x (1.21x to 1.39x) |
| Mandelbrot, whole-plane matrix (`mandelbrot_noout_naive`) | 1.29x (1.25x to 1.34x) |
| Binary trees (`binarytrees`) | 1.20x (1.10x to 1.27x) |
| Binary trees, inline arithmetic (`binarytrees_2`) | 1.11x (1.04x to 1.22x) |
| Binary trees, explicit loop (`binarytrees_naive`) | 1.09x (1.01x to 1.15x) |
| Fannkuch-redux (`fannkuchredux`) | 1.03x (0.94x to 1.08x) |
| Spectral norm, element-wise matrix fill (`spectralnorm_alt_3`) | 1.02x (0.97x to 1.04x) |

A time marked too short on a ry engine is under 0.1 s. That speed is what the suite measures, so it counts in the geometric mean. A program marked wide interval varied so much between rounds that its ratio's interval is wide; read its median with care. Measured on 2026-11-11 at ry 4d96a342b with GNU R 4.6.2 on Intel(R) Xeon(R) Processor @ 2.30GHz, 4 cores, Linux 6.18.44-fc-v378; 10 paired rounds.

Public R JIT benchmarks: rcp micro-benchmarks and convolutions, speed-up over GNU R:

| program | ry |
| --- | ---: |
| Scalar assignment, while loop (`scalar-while`), too short on ry, wide interval on ry | 15x (13x to 17x) |
| Scalar assignment, for loop (`scalar-for`) | 12x (11x to 12x) |
| Counting while loop (`empty-while`), too short on ry | 8.86x (8.12x to 9.66x) |
| Many local assignments (`bytecodes`), too short on ry | 6.97x (6.34x to 7.43x) |
| Convolution, scalar loops (`convolution`) | 5.91x (5.62x to 6.29x) |
| Vector element assignment, while loop (`vector-while`) | 5.68x (5.45x to 5.96x) |
| Superassignment loop (`super-while`) | 4.84x (4.76x to 5.02x) |
| Growing a list, for loop (`list-for`), wide interval on ry | 4.49x (3.99x to 4.90x) |
| Growing a list, while loop (`list-while`), wide interval on ry | 4.40x (3.94x to 4.98x) |
| Convolution, [[ indexing (`convolution_slow`) | 4.25x (4.11x to 4.54x) |
| Vector element assignment, for loop (`vector-for`) | 4.18x (3.95x to 4.47x) |
| Empty for loop (`empty-for`) | 3.06x (2.96x to 3.32x) |
| lapply with an extra argument (`lapply-dots`) | 2.54x (2.37x to 2.68x) |
| Matrix element assignment (`matrix-for`), wide interval on ry | 2.43x (2.00x to 2.57x) |
| lapply with a closure (`lapply`) | 2.35x (2.19x to 2.56x) |
| Convolution, vectorized (`convolution_v`) | 1.48x (1.42x to 1.60x) |
| geometric mean of all groups | 2.90x (2.86x to 2.92x) |

A time marked too short on a ry engine is under 0.1 s. That speed is what the suite measures, so it counts in the geometric mean. A program marked wide interval varied so much between rounds that its ratio's interval is wide; read its median with care. Measured on 2026-11-11 at ry 4d96a342b with GNU R 4.6.2 on Intel(R) Xeon(R) Processor @ 2.30GHz, 4 cores, Linux 6.18.44-fc-v378; 10 paired rounds.

Scalar loops gain the most, because the JIT compiles them to machine code while GNU R interprets
bytecode. Programs that spend their time allocating gain the least.

### Time

| program | GNU R | ry | ry, speed-up |
| --- | ---: | ---: | ---: |
| Bouncing balls, named fields | 0.955 s | 0.463 s | 1.95x (1.80x to 2.10x) |
| Bouncing balls, positional fields | 0.773 s | 0.397 s | 1.88x (1.69x to 2.05x) (wide interval on ry) |
| Bouncing balls, positional fields, simple setup | 0.766 s | 0.406 s | 1.82x (1.75x to 1.95x) |
| Mandelbrot set, scalar loops | 0.866 s | 0.294 s | 2.88x (2.68x to 3.03x) |
| Tree allocation | 1.000 s | 0.471 s | 2.12x (2.05x to 2.28x) |
| Binary trees | 1.325 s | 1.091 s | 1.20x (1.10x to 1.27x) |
| Binary trees, inline arithmetic | 0.515 s | 0.480 s | 1.11x (1.04x to 1.22x) |
| Binary trees, explicit loop | 1.216 s | 1.113 s | 1.09x (1.01x to 1.15x) |
| Fannkuch-redux | 1.857 s | 1.875 s | 1.03x (0.94x to 1.08x) |
| Fannkuch-redux, scalar swaps | 1.017 s | 0.394 s | 2.53x (2.33x to 2.58x) |
| Fasta | 0.705 s | 0.277 s | 2.48x (2.31x to 2.59x) |
| Fasta, vectorized linear search | 0.750 s | 0.294 s | 2.61x (2.37x to 2.75x) |
| Fasta, vectorized binary search | 0.990 s | 0.347 s | 2.91x (2.62x to 3.16x) |
| Fasta, scalar linear search | 1.155 s | 0.325 s | 3.50x (3.04x to 3.67x) (wide interval on ry) |
| Fasta, scalar binary search | 1.542 s | 0.643 s | 2.41x (2.19x to 2.60x) |
| Fasta-redux | 0.915 s | 0.279 s | 3.28x (2.79x to 3.46x) (wide interval on ry) |
| Fasta-redux, scalar lookup | 1.117 s | 0.327 s | 3.10x (2.91x to 3.51x) (wide interval on ry) |
| K-nucleotide, hashed environment | 1.079 s | 0.419 s | 2.54x (2.42x to 2.69x) |
| K-nucleotide, attribute counts | 0.666 s | 0.183 s | 3.55x (3.39x to 3.76x) |
| K-nucleotide, attribute counts, growing vector | 1.095 s | 0.377 s | 2.85x (2.62x to 3.06x) |
| K-nucleotide, named vector counts | 0.872 s | 0.199 s | 4.28x (4.17x to 4.46x) |
| Mandelbrot, vectorized rows | 0.737 s | 0.460 s | 1.55x (1.43x to 1.68x) |
| Mandelbrot, scalar points | 1.169 s | 0.536 s | 2.23x (2.14x to 2.40x) |
| Mandelbrot, vectorized rows, no output | 0.601 s | 0.420 s | 1.45x (1.39x to 1.58x) |
| Mandelbrot, whole-plane matrix | 1.016 s | 0.783 s | 1.29x (1.25x to 1.34x) |
| N-body | 1.109 s | 0.817 s | 1.34x (1.21x to 1.39x) |
| N-body, matrix state | 1.177 s | 0.702 s | 1.72x (1.47x to 1.80x) (wide interval on ry) |
| Pi digits | 0.632 s | 0.403 s | 1.67x (1.52x to 1.73x) |
| Regex DNA | 0.743 s | 0.398 s | 1.91x (1.78x to 2.19x) (wide interval on ry) |
| Reverse complement | 0.626 s | 0.122 s | 4.95x (4.61x to 5.37x) |
| Reverse complement, whole file | 0.803 s | 0.135 s | 5.97x (5.48x to 6.55x) |
| Reverse complement, scalar lookup | 1.349 s | 0.350 s | 3.85x (3.62x to 4.06x) |
| Spectral norm | 0.974 s | 0.068 s | 14x (14x to 15x) (too short on ry) |
| Spectral norm, precomputed matrix | 1.516 s | 0.898 s | 1.67x (1.62x to 1.79x) |
| Spectral norm, matrix rebuilt per product | 1.885 s | 0.571 s | 3.23x (2.94x to 3.41x) |
| Spectral norm, element-wise matrix fill | 1.394 s | 1.371 s | 1.02x (0.97x to 1.04x) |
| Spectral norm, eigenvalues | 1.317 s | 1.051 s | 1.35x (1.24x to 1.37x) |
| Spectral norm, scalar loops | 0.996 s | 0.074 s | 13x (11x to 14x) (too short on ry, wide interval on ry) |
| Many local assignments | 0.493 s | 0.070 s | 6.97x (6.34x to 7.43x) (too short on ry) |
| Convolution, scalar loops | 0.755 s | 0.127 s | 5.91x (5.62x to 6.29x) |
| Convolution, [[ indexing | 0.831 s | 0.192 s | 4.25x (4.11x to 4.54x) |
| Convolution, vectorized | 1.361 s | 0.891 s | 1.48x (1.42x to 1.60x) |
| Empty for loop | 0.543 s | 0.172 s | 3.06x (2.96x to 3.32x) |
| Counting while loop | 0.474 s | 0.056 s | 8.86x (8.12x to 9.66x) (too short on ry) |
| lapply with a closure | 0.725 s | 0.314 s | 2.35x (2.19x to 2.56x) |
| lapply with an extra argument | 0.915 s | 0.351 s | 2.54x (2.37x to 2.68x) |
| Growing a list, for loop | 1.320 s | 0.285 s | 4.49x (3.99x to 4.90x) (wide interval on ry) |
| Growing a list, while loop | 1.173 s | 0.268 s | 4.40x (3.94x to 4.98x) (wide interval on ry) |
| Matrix element assignment | 0.495 s | 0.206 s | 2.43x (2.00x to 2.57x) (wide interval on ry) |
| Scalar assignment, for loop | 1.238 s | 0.107 s | 12x (11x to 12x) |
| Scalar assignment, while loop | 1.202 s | 0.072 s | 15x (13x to 17x) (too short on ry, wide interval on ry) |
| Superassignment loop | 1.004 s | 0.204 s | 4.84x (4.76x to 5.02x) |
| Vector element assignment, for loop | 0.916 s | 0.213 s | 4.18x (3.95x to 4.47x) |
| Vector element assignment, while loop | 0.942 s | 0.166 s | 5.68x (5.45x to 5.96x) |

Each time is the median over the rounds. Each speed-up is the median of the per-round ratios, with its 95% interval in parentheses.

### Memory

| program | GNU R | ry |
| --- | ---: | ---: |
| Bouncing balls, named fields | 77 MiB | 81 MiB |
| Bouncing balls, positional fields | 89 MiB | 86 MiB |
| Bouncing balls, positional fields, simple setup | 88 MiB | 86 MiB |
| Mandelbrot set, scalar loops | 71 MiB | 67 MiB |
| Tree allocation | 106 MiB | 87 MiB |
| Binary trees | 75 MiB | 66 MiB |
| Binary trees, inline arithmetic | 70 MiB | 58 MiB |
| Binary trees, explicit loop | 77 MiB | 62 MiB |
| Fannkuch-redux | 74 MiB | 79 MiB |
| Fannkuch-redux, scalar swaps | 78 MiB | 85 MiB |
| Fasta | 127 MiB | 91 MiB |
| Fasta, vectorized linear search | 119 MiB | 90 MiB |
| Fasta, vectorized binary search | 122 MiB | 94 MiB |
| Fasta, scalar linear search | 125 MiB | 91 MiB |
| Fasta, scalar binary search | 117 MiB | 95 MiB |
| Fasta-redux | 141 MiB | 99 MiB |
| Fasta-redux, scalar lookup | 128 MiB | 98 MiB |
| K-nucleotide, hashed environment | 73 MiB | 79 MiB |
| K-nucleotide, attribute counts | 71 MiB | 62 MiB |
| K-nucleotide, attribute counts, growing vector | 71 MiB | 65 MiB |
| K-nucleotide, named vector counts | 131 MiB | 64 MiB |
| Mandelbrot, vectorized rows | 125 MiB | 94 MiB |
| Mandelbrot, scalar points | 140 MiB | 68 MiB |
| Mandelbrot, vectorized rows, no output | 124 MiB | 60 MiB |
| Mandelbrot, whole-plane matrix | 405 MiB | 404 MiB |
| N-body | 70 MiB | 61 MiB |
| N-body, matrix state | 73 MiB | 65 MiB |
| Pi digits | 70 MiB | 74 MiB |
| Regex DNA | 119 MiB | 97 MiB |
| Reverse complement | 148 MiB | 94 MiB |
| Reverse complement, whole file | 157 MiB | 109 MiB |
| Reverse complement, scalar lookup | 147 MiB | 110 MiB |
| Spectral norm | 71 MiB | 72 MiB |
| Spectral norm, precomputed matrix | 310 MiB | 430 MiB |
| Spectral norm, matrix rebuilt per product | 169 MiB | 111 MiB |
| Spectral norm, element-wise matrix fill | 81 MiB | 76 MiB |
| Spectral norm, eigenvalues | 127 MiB | 114 MiB |
| Spectral norm, scalar loops | 70 MiB | 67 MiB |
| Many local assignments | 84 MiB | 69 MiB |
| Convolution, scalar loops | 71 MiB | 79 MiB |
| Convolution, [[ indexing | 71 MiB | 80 MiB |
| Convolution, vectorized | 187 MiB | 151 MiB |
| Empty for loop | 64 MiB | 51 MiB |
| Counting while loop | 65 MiB | 51 MiB |
| lapply with a closure | 189 MiB | 157 MiB |
| lapply with an extra argument | 187 MiB | 157 MiB |
| Growing a list, for loop | 220 MiB | 195 MiB |
| Growing a list, while loop | 220 MiB | 193 MiB |
| Matrix element assignment | 168 MiB | 194 MiB |
| Scalar assignment, for loop | 77 MiB | 51 MiB |
| Scalar assignment, while loop | 77 MiB | 51 MiB |
| Superassignment loop | 78 MiB | 60 MiB |
| Vector element assignment, for loop | 293 MiB | 170 MiB |
| Vector element assignment, while loop | 141 MiB | 132 MiB |

Peak memory is the peak resident set size of the whole process, R's startup included.

Measured on 2026-11-11 at ry 4d96a342b with GNU R 4.6.2 on Intel(R) Xeon(R) Processor @ 2.30GHz, 4 cores, Linux 6.18.44-fc-v378; 10 paired rounds.

### Are We Fast Yet

R translations of programs from the Are We Fast Yet suite, which compares language implementations on the same small object and loop workloads. Bounce comes in three variants that differ in how a ball's fields are indexed. Source: https://github.com/PRL-PRG/r-compile-server/tree/b2b13a3150bddcc2d6dae8c37ecbbf1007170940/client/rsh/inst/benchmarks/areWeFast Licence: MIT.

- **Bouncing balls, named fields** ([`bounce`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/awfy/bounce.R), 1.95x): Moves 3,000 balls for 50 steps with lapply, reading and writing each ball's fields by name (ball['xVel'])
- **Bouncing balls, positional fields** ([`bounce_nonames`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/awfy/bounce_nonames.R), 1.88x): Moves 10,000 balls for 50 steps with lapply, indexing each ball's fields by position and counting bounces through <<-
- **Bouncing balls, positional fields, simple setup** ([`bounce_nonames_simple`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/awfy/bounce_nonames_simple.R), 1.82x): Same as the positional variant, but each ball's four random numbers are drawn into locals before c() builds it
- **Mandelbrot set, scalar loops** ([`mandelbrot`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/awfy/mandelbrot.R), 2.88x): Computes a 450 x 450 Mandelbrot bitmap checksum with nested scalar while loops, double arithmetic and bitwShiftL/bitwXor
- **Tree allocation** ([`storage`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/awfy/storage.R), 2.12x): Recursively builds a depth-10 tree of 4-element lists with random leaves, stressing allocation, recursion and <<-

### Benchmarks Game

R ports of the Computer Language Benchmarks Game programs, each in several variants that range from idiomatic vectorized R to naive scalar loops. The input-reading programs read a FASTA file that the driver generates with fasta.R before timing starts. Source: https://github.com/PRL-PRG/r-compile-server/tree/b2b13a3150bddcc2d6dae8c37ecbbf1007170940/client/rsh/inst/benchmarks/shootout Licence: MIT.

- **Binary trees** ([`binarytrees`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/shootout/binarytrees.R), 1.20x): Allocates and walks recursive list trees up to depth 12, with integer arithmetic and sapply over the iterations
- **Binary trees, inline arithmetic** ([`binarytrees_2`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/shootout/binarytrees_2.R), 1.11x): Binary trees with the child items computed inline in the recursive call and double rather than integer depths
- **Binary trees, explicit loop** ([`binarytrees_naive`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/shootout/binarytrees_naive.R), 1.09x): Binary trees that sums each depth's checks in a for loop instead of sapply
- **Fannkuch-redux** ([`fannkuchredux`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/shootout/fannkuchredux.R), 1.03x): Counts pancake flips over all permutations of 9 elements, reversing prefixes with vectorized rev and slice assignment
- **Fannkuch-redux, scalar swaps** ([`fannkuchredux_naive`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/shootout/fannkuchredux_naive.R), 2.53x): Fannkuch-redux that reverses prefixes and rotates the permutation one element at a time with [[ swaps
- **Fasta** ([`fasta`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/shootout/fasta.R), 2.48x): Generates DNA sequences with an integer LCG, picking each line's codes at once with outer() and colSums
- **Fasta, vectorized linear search** ([`fasta_2`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/shootout/fasta_2.R), 2.61x): Fasta that picks each line's codes by a vectorized linear search, advancing indices with which() until all match
- **Fasta, vectorized binary search** ([`fasta_3`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/shootout/fasta_3.R), 2.91x): Fasta that picks each line's codes by a vectorized binary search over the cumulative probabilities
- **Fasta, scalar linear search** ([`fasta_naive`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/shootout/fasta_naive.R), 3.50x): Fasta that picks each character with a scalar linear search loop and joins every line with paste
- **Fasta, scalar binary search** ([`fasta_naive_2`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/shootout/fasta_naive_2.R), 2.41x): Fasta that picks each character with a scalar binary search loop and joins every line with paste
- **Fasta-redux** ([`fastaredux`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/shootout/fastaredux.R), 3.28x): Fasta with a 4,096-entry lookup table that maps a random number to a code, corrected for a whole line at a time
- **Fasta-redux, scalar lookup** ([`fastaredux_naive`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/shootout/fastaredux_naive.R), 3.10x): Fasta-redux that does the table lookup and its correction one character at a time in a scalar loop
- **K-nucleotide, hashed environment** ([`knucleotide`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/shootout/knucleotide.R), 2.54x): Reads a FASTA file and counts k-mer frequencies in a hashed environment with exists, get and assign
- **K-nucleotide, attribute counts** ([`knucleotide_brute`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/shootout/knucleotide_brute.R), 3.55x): K-nucleotide that keeps each k-mer's count as an attribute of an integer vector, read with attr and set with attr<-
- **K-nucleotide, attribute counts, growing vector** ([`knucleotide_brute_2`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/shootout/knucleotide_brute_2.R), 2.85x): Attribute-count k-nucleotide that also grows the underlying vector by doubling each time a new k-mer appears
- **K-nucleotide, named vector counts** ([`knucleotide_brute_3`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/shootout/knucleotide_brute_3.R), 4.28x): K-nucleotide that keeps the counts in a named integer vector, indexed and assigned by k-mer name
- **Mandelbrot, vectorized rows** ([`mandelbrot_ascii`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/shootout/mandelbrot_ascii.R), 1.55x): Iterates complex vectors one 1,300-point row at a time, packs the bits into raw bytes and prints them
- **Mandelbrot, scalar points** ([`mandelbrot_naive_ascii`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/shootout/mandelbrot_naive_ascii.R), 2.23x): Iterates each point as a complex scalar with an escape test, packing bits into bytes one point at a time
- **Mandelbrot, vectorized rows, no output** ([`mandelbrot_noout`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/shootout/mandelbrot_noout.R), 1.45x): The vectorized-row Mandelbrot without printing the bitmap, so the time is complex vector arithmetic
- **Mandelbrot, whole-plane matrix** ([`mandelbrot_noout_naive`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/shootout/mandelbrot_noout_naive.R), 1.29x): Iterates a 1,000 x 1,000 complex matrix 20 times, storing exp(-abs(Z)) into slices of a 3-D array
- **N-body** ([`nbody`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/shootout/nbody.R), 1.34x): Simulates five planets with one coordinate vector per axis, using outer() and %*% at each of 40,000 steps
- **N-body, matrix state** ([`nbody_2`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/shootout/nbody_2.R), 1.72x): N-body with positions in 3 x 5 matrices and pairwise distances filled into a 3-D array by nested loops, then aperm
- **Pi digits** ([`pidigits`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/shootout/pidigits.R), 1.67x): Prints 55 digits of pi with a spigot over a hand-written bignum on base-10,000 integer vectors
- **Regex DNA** ([`regexdna`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/shootout/regexdna.R), 1.91x): Counts nine DNA patterns with gregexpr and expands IUB codes with gsub over one large string
- **Reverse complement** ([`reversecomplement`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/shootout/reversecomplement.R), 4.95x): Reads a FASTA file line by line and maps each line's characters through a named vector in one vectorized lookup
- **Reverse complement, whole file** ([`reversecomplement_2`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/shootout/reversecomplement_2.R), 5.97x): Reverse complement that reads the whole file with one readLines call before mapping each line
- **Reverse complement, scalar lookup** ([`reversecomplement_naive`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/shootout/reversecomplement_naive.R), 3.85x): Reverse complement that maps one character at a time with [[ on the named vector in a for loop
- **Spectral norm** ([`spectralnorm`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/shootout/spectralnorm.R), 14x): Multiplies by an 800 x 800 matrix once, filling each row with scalar closure calls and taking a %*% dot product
- **Spectral norm, precomputed matrix** ([`spectralnorm_alt`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/shootout/spectralnorm_alt.R), 1.67x): Builds the 3,000 x 3,000 matrix once with outer() and runs 10 power iterations as matrix products
- **Spectral norm, matrix rebuilt per product** ([`spectralnorm_alt_2`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/shootout/spectralnorm_alt_2.R), 3.23x): Spectral norm that rebuilds the 1,200 x 1,200 matrix with outer() for every matrix product
- **Spectral norm, element-wise matrix fill** ([`spectralnorm_alt_3`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/shootout/spectralnorm_alt_3.R), 1.02x): Spectral norm that copies a precomputed table into a fresh matrix with [[i, j]]<- in nested loops for every product
- **Spectral norm, eigenvalues** ([`spectralnorm_math`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/shootout/spectralnorm_math.R), 1.35x): Computes the spectral norm directly as the largest eigenvalue of t(A) %*% A with eigen(), so LAPACK dominates
- **Spectral norm, scalar loops** ([`spectralnorm_naive`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/shootout/spectralnorm_naive.R), 13x): Spectral norm with every matrix-vector product as nested scalar loops that call a closure per element

### rcp micro-benchmarks and convolutions

The 13 micro-benchmarks of the simple/ directory, each one tight loop that isolates an interpreter cost such as variable assignment, vector subassignment or closure calls, and the 3 convolution programs of RealThing/, which compute the same convolution with scalar loops or vector operations. Source: https://github.com/PRL-PRG/r-compile-server/tree/b2b13a3150bddcc2d6dae8c37ecbbf1007170940/client/rsh/inst/benchmarks Licence: MIT.

- **Many local assignments** ([`bytecodes`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/rcp/bytecodes.R), 6.97x): A while loop that assigns the counter to 24 local variables per iteration, 500,000 times
- **Convolution, scalar loops** ([`convolution`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/rcp/convolution.R), 5.91x): Convolves two 1,000-element vectors 10 times with nested for loops and [ indexing, as in Writing R Extensions
- **Convolution, [[ indexing** ([`convolution_slow`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/rcp/convolution_slow.R), 4.25x): Convolution with nested loops over seq(length =), [[ indexing and the outer element hoisted into a local
- **Convolution, vectorized** ([`convolution_v`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/rcp/convolution_v.R), 1.48x): Convolves 1,500-element vectors with outer(), rbind, reshaping through length<- and dim<-, and rowSums
- **Empty for loop** ([`empty-for`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/rcp/empty-for.R), 3.06x): Runs a for loop with an empty body over 1:1e8, measuring pure loop overhead
- **Counting while loop** ([`empty-while`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/rcp/empty-while.R), 8.86x): Increments a double scalar in a while loop up to 2e7
- **lapply with a closure** ([`lapply`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/rcp/lapply.R), 2.35x): Calls a one-argument closure through lapply over 1e6 integers
- **lapply with an extra argument** ([`lapply-dots`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/rcp/lapply-dots.R), 2.54x): Calls a two-argument closure through lapply over 1e6 integers, passing the second argument through ...
- **Growing a list, for loop** ([`list-for`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/rcp/list-for.R), 4.49x): Grows an empty list to 1.5e6 elements by [[<- in a for loop
- **Growing a list, while loop** ([`list-while`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/rcp/list-while.R), 4.40x): Grows an empty list to 1.5e6 elements by [[<- in a while loop with a double index
- **Matrix element assignment** ([`matrix-for`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/rcp/matrix-for.R), 2.43x): Assigns every element of a 3,000 x 3,000 matrix with [[i, j]]<- in nested for loops
- **Scalar assignment, for loop** ([`scalar-for`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/rcp/scalar-for.R), 12x): Assigns the loop variable to a local in a for loop over 1:5e7
- **Scalar assignment, while loop** ([`scalar-while`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/rcp/scalar-while.R), 15x): Assigns and increments two double scalars in a while loop, 2.5e7 times
- **Superassignment loop** ([`super-while`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/rcp/super-while.R), 4.84x): A closure increments a variable of its enclosing function with <<- in a while loop, 6e6 times
- **Vector element assignment, for loop** ([`vector-for`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/rcp/vector-for.R), 4.18x): Fills a 3e7-element vector by [[<- in a for loop, the first write coercing it from logical to integer
- **Vector element assignment, while loop** ([`vector-while`](https://github.com/felix-andreas/ry/blob/main/tools/bench/public/rcp/vector-while.R), 5.68x): Fills a 2e7-element integer vector by [[<- in a while loop with an integer counter, then sums it

## Package test suites

This suite times the test suites of packages from ry's benchmark dependency set. Each package and
every dependency it installs is pinned by version and by the SHA-256 of its source tarball
(`tools/bench/packages/lock.tsv`), so two machines measure the same code. GNU R installs them into
one library that both engines load, because ry's `libR` is ABI-compatible with GNU R's. The tests
run the way `R CMD check` runs them, offline. Each time is the wall time of a whole process, or
for a package whose `tests/` holds plain scripts, the sum over one process per script, so it
includes start-up and loading the package and its imports, as real use does.

The suite also counts the tests that pass, fail and raise errors on each engine. A package whose
counts differ between the engines is flagged, and a package whose tests do not run to the end on an
engine is marked as failing there and has no speed-up.

Package test suites, speed-up over GNU R:

| program | ry |
| --- | ---: |
| abind | 3.09x (2.92x to 3.29x) |
| progress | 2.58x (2.50x to 2.82x) |
| R6 | 2.58x (2.46x to 2.77x) |
| brio | 2.50x (2.25x to 2.72x) |
| DBI | 2.43x (2.25x to 2.56x) |
| assertthat | 2.42x (2.24x to 2.66x) |
| rappdirs | 2.40x (2.18x to 2.51x) |
| cachem | 2.35x (2.21x to 2.56x) |
| zoo | 2.34x (2.27x to 2.51x) |
| memoise | 2.27x (2.18x to 2.36x) |
| zeallot, wide interval on ry | 2.26x (2.02x to 2.44x) |
| PerformanceAnalytics | 2.19x (2.10x to 2.29x) |
| withr, tests differ on ry: GNU R: 355 passed, 0 failed, 0 errors; ry: 354 passed, 0 failed, 0 errors | 2.14x (1.99x to 2.29x) |
| lubridate | 1.98x (1.89x to 2.16x) |
| lgr | 1.81x (1.70x to 1.96x) |
| digest | 1.78x (1.62x to 1.92x) |
| stringr | 1.73x (1.57x to 1.79x) |
| rlang, tests differ on ry: GNU R: 4010 passed, 0 failed, 0 errors; ry: 3995 passed, 0 failed, 12 errors | 1.73x (1.66x to 1.77x) |
| jsonlite | 1.66x (1.56x to 1.74x) |
| purrr | 1.57x (1.46x to 1.67x) |
| cli | 1.54x (1.50x to 1.61x) |
| fs | 1.53x (1.44x to 1.60x) |
| snakecase | 1.44x (1.33x to 1.58x) |
| data.table, tests differ on ry: GNU R: 10 passed, 0 failed, 0 errors; ry: 9 passed, 1 failed, 0 errors | 1.35x (1.23x to 1.37x) |
| xts | 1.20x (1.13x to 1.29x) |
| foreach | 1.08x (1.07x to 1.09x) |
| geometric mean | 1.93x (1.91x to 1.96x) |

A program marked wide interval varied so much between rounds that its ratio's interval is wide; read its median with care. A package marked tests differ passed a different number of tests on ry than on GNU R; the flag gives both counts. Measured on 2026-11-11 at ry 46191c4 with GNU R 4.6.2 on Intel(R) Xeon(R) Processor @ 2.30GHz, 4 cores, Linux 6.18.44-fc-v378; 10 paired rounds.

### Time

| program | GNU R | ry | ry, speed-up |
| --- | ---: | ---: | ---: |
| abind 1.4-8 | 0.801 s | 0.257 s | 3.09x (2.92x to 3.29x) |
| assertthat 0.2.1 | 0.835 s | 0.341 s | 2.42x (2.24x to 2.66x) |
| brio 1.1.5 | 0.808 s | 0.333 s | 2.50x (2.25x to 2.72x) |
| cachem 1.1.0 | 0.905 s | 0.376 s | 2.35x (2.21x to 2.56x) |
| cli 3.6.6 | 25.939 s | 16.591 s | 1.54x (1.50x to 1.61x) |
| data.table 1.18.6.1 | 31.518 s | 23.861 s | 1.35x (1.23x to 1.37x) (tests differ on ry: GNU R: 10 passed, 0 failed, 0 errors; ry: 9 passed, 1 failed, 0 errors) |
| DBI 1.3.0 | 1.111 s | 0.436 s | 2.43x (2.25x to 2.56x) |
| digest 0.6.39 | 0.422 s | 0.233 s | 1.78x (1.62x to 1.92x) |
| foreach 1.5.2 | 11.560 s | 10.783 s | 1.08x (1.07x to 1.09x) |
| fs 2.1.0 | 10.215 s | 6.744 s | 1.53x (1.44x to 1.60x) |
| jsonlite 2.0.0 | 3.143 s | 1.865 s | 1.66x (1.56x to 1.74x) |
| lgr 0.5.2 | 5.936 s | 3.269 s | 1.81x (1.70x to 1.96x) |
| lubridate 1.9.5 | 15.470 s | 7.633 s | 1.98x (1.89x to 2.16x) |
| memoise 2.0.1 | 0.709 s | 0.311 s | 2.27x (2.18x to 2.36x) |
| PerformanceAnalytics 2.1.0 | 2.877 s | 1.198 s | 2.19x (2.10x to 2.29x) |
| progress 1.2.3 | 0.557 s | 0.213 s | 2.58x (2.50x to 2.82x) |
| purrr 1.2.2 | 13.544 s | 8.710 s | 1.57x (1.46x to 1.67x) |
| R6 2.6.1 | 3.724 s | 1.460 s | 2.58x (2.46x to 2.77x) |
| rappdirs 0.3.4 | 0.736 s | 0.303 s | 2.40x (2.18x to 2.51x) |
| rlang 1.3.0 | 64.120 s | 37.309 s | 1.73x (1.66x to 1.77x) (tests differ on ry: GNU R: 4010 passed, 0 failed, 0 errors; ry: 3995 passed, 0 failed, 12 errors) |
| snakecase 0.11.1 | 1.865 s | 1.240 s | 1.44x (1.33x to 1.58x) |
| stringr 1.6.0 | 3.913 s | 2.413 s | 1.73x (1.57x to 1.79x) |
| withr 3.0.3 | 2.661 s | 1.244 s | 2.14x (1.99x to 2.29x) (tests differ on ry: GNU R: 355 passed, 0 failed, 0 errors; ry: 354 passed, 0 failed, 0 errors) |
| xts 0.14.3 | 4.495 s | 3.695 s | 1.20x (1.13x to 1.29x) |
| zeallot 0.2.0 | 0.657 s | 0.294 s | 2.26x (2.02x to 2.44x) (wide interval on ry) |
| zoo 1.9-1 | 1.813 s | 0.744 s | 2.34x (2.27x to 2.51x) |

Each time is the median over the rounds. Each speed-up is the median of the per-round ratios, with its 95% interval in parentheses.

### Memory

| program | GNU R | ry |
| --- | ---: | ---: |
| abind 1.4-8 | 111 MiB | 114 MiB |
| assertthat 0.2.1 | 113 MiB | 93 MiB |
| brio 1.1.5 | 127 MiB | 99 MiB |
| cachem 1.1.0 | 115 MiB | 91 MiB |
| cli 3.6.6 | 181 MiB | 207 MiB |
| data.table 1.18.6.1 | 429 MiB | 377 MiB |
| DBI 1.3.0 | 124 MiB | 118 MiB |
| digest 0.6.39 | 97 MiB | 84 MiB |
| foreach 1.5.2 | 174 MiB | 143 MiB |
| fs 2.1.0 | 228 MiB | 202 MiB |
| jsonlite 2.0.0 | 156 MiB | 134 MiB |
| lgr 0.5.2 | 155 MiB | 136 MiB |
| lubridate 1.9.5 | 183 MiB | 193 MiB |
| memoise 2.0.1 | 114 MiB | 91 MiB |
| PerformanceAnalytics 2.1.0 | 133 MiB | 136 MiB |
| progress 1.2.3 | 117 MiB | 94 MiB |
| purrr 1.2.2 | 171 MiB | 150 MiB |
| R6 2.6.1 | 117 MiB | 110 MiB |
| rappdirs 0.3.4 | 116 MiB | 98 MiB |
| rlang 1.3.0 | 227 MiB | 250 MiB |
| snakecase 0.11.1 | 121 MiB | 120 MiB |
| stringr 1.6.0 | 128 MiB | 132 MiB |
| withr 3.0.3 | 188 MiB | 152 MiB |
| xts 0.14.3 | 338 MiB | 452 MiB |
| zeallot 0.2.0 | 115 MiB | 95 MiB |
| zoo 1.9-1 | 122 MiB | 110 MiB |

Peak memory is the peak resident set size of the whole process, R's startup included.

### Tests

| package | GNU R: passed, failed, errors | ry: passed, failed, errors |
| --- | ---: | ---: |
| abind 1.4-8 | 5, 0, 0 | 5, 0, 0 |
| assertthat 0.2.1 | 159, 0, 0 | 159, 0, 0 |
| brio 1.1.5 | 81, 0, 0 | 81, 0, 0 |
| cachem 1.1.0 | 90, 0, 0 | 90, 0, 0 |
| cli 3.6.6 | 1527, 0, 0 | 1527, 0, 0 |
| data.table 1.18.6.1 | 10, 0, 0 | 9, 1, 0 |
| DBI 1.3.0 | 101, 0, 0 | 101, 0, 0 |
| digest 0.6.39 | 324, 0, 0 | 324, 0, 0 |
| foreach 1.5.2 | 570, 0, 0 | 570, 0, 0 |
| fs 2.1.0 | 3435, 0, 0 | 3435, 0, 0 |
| jsonlite 2.0.0 | 1524, 0, 0 | 1524, 0, 0 |
| lgr 0.5.2 | 522, 1, 1 | 522, 1, 1 |
| lubridate 1.9.5 | 3035, 0, 0 | 3035, 0, 0 |
| memoise 2.0.1 | 160, 0, 0 | 160, 0, 0 |
| PerformanceAnalytics 2.1.0 | 69, 0, 0 | 69, 0, 0 |
| progress 1.2.3 | 17, 0, 0 | 17, 0, 0 |
| purrr 1.2.2 | 735, 0, 0 | 735, 0, 0 |
| R6 2.6.1 | 625, 0, 0 | 625, 0, 0 |
| rappdirs 0.3.4 | 45, 0, 0 | 45, 0, 0 |
| rlang 1.3.0 | 4010, 0, 0 | 3995, 0, 12 |
| snakecase 0.11.1 | 292, 0, 0 | 292, 0, 0 |
| stringr 1.6.0 | 445, 0, 0 | 445, 0, 0 |
| withr 3.0.3 | 355, 0, 0 | 354, 0, 0 |
| xts 0.14.3 | 4038, 0, 0 | 4038, 0, 0 |
| zeallot 0.2.0 | 100, 0, 0 | 100, 0, 0 |
| zoo 1.9-1 | 6, 0, 0 | 6, 0, 0 |

Measured on 2026-11-11 at ry 46191c4 with GNU R 4.6.2 on Intel(R) Xeon(R) Processor @ 2.30GHz, 4 cores, Linux 6.18.44-fc-v378; 10 paired rounds.

- **abind** ([`abind`](https://github.com/felix-andreas/ry/blob/main/tools/bench/packages), 3.09x): Combines multidimensional arrays, with scripts that check dimnames and array slicing output
- **assertthat** ([`assertthat`](https://github.com/felix-andreas/ry/blob/main/tools/bench/packages), 2.42x): Easy pre and post assertions, tested through many small predicate calls
- **brio** ([`brio`](https://github.com/felix-andreas/ry/blob/main/tools/bench/packages), 2.50x): Basic R input and output, with tests that read and write files through C code
- **cachem** ([`cachem`](https://github.com/felix-andreas/ry/blob/main/tools/bench/packages), 2.35x): Caches R objects with automatic pruning, in memory and on disk
- **cli** ([`cli`](https://github.com/felix-andreas/ry/blob/main/tools/bench/packages), 1.54x): Helpers for command line interfaces, with snapshot tests of styled and formatted text
- **data.table** ([`data.table`](https://github.com/felix-andreas/ry/blob/main/tools/bench/packages), 1.35x): Extension of data.frame, with a large script suite of table operations backed by C code
- **DBI** ([`DBI`](https://github.com/felix-andreas/ry/blob/main/tools/bench/packages), 2.43x): The R database interface, a set of S4 generics, with tests of SQL quoting, interpolation and a dummy driver
- **digest** ([`digest`](https://github.com/felix-andreas/ry/blob/main/tools/bench/packages), 1.78x): Compact hash digests of R objects, with tests that compare many hash algorithms against known values
- **foreach** ([`foreach`](https://github.com/felix-andreas/ry/blob/main/tools/bench/packages), 1.08x): The foreach looping construct, with tests of iterators, result combiners, nesting and error handling
- **fs** ([`fs`](https://github.com/felix-andreas/ry/blob/main/tools/bench/packages), 1.53x): Cross-platform file system operations based on libuv, with tests that create and inspect real files
- **jsonlite** ([`jsonlite`](https://github.com/felix-andreas/ry/blob/main/tools/bench/packages), 1.66x): A JSON parser and generator, with round-trip tests over many R data types
- **lgr** ([`lgr`](https://github.com/felix-andreas/ry/blob/main/tools/bench/packages), 1.81x): A logging framework built on R6 classes, with appenders, layouts and filters
- **lubridate** ([`lubridate`](https://github.com/felix-andreas/ry/blob/main/tools/bench/packages), 1.98x): Date and time handling, with tests of parsing, time zones and period arithmetic
- **memoise** ([`memoise`](https://github.com/felix-andreas/ry/blob/main/tools/bench/packages), 2.27x): Memoisation of functions, with tests of in-memory and file system caches
- **PerformanceAnalytics** ([`PerformanceAnalytics`](https://github.com/felix-andreas/ry/blob/main/tools/bench/packages), 2.19x): Econometric tools for performance and risk analysis of xts return series
- **progress** ([`progress`](https://github.com/felix-andreas/ry/blob/main/tools/bench/packages), 2.58x): Terminal progress bars, with tests of their rendering from R and from C++
- **purrr** ([`purrr`](https://github.com/felix-andreas/ry/blob/main/tools/bench/packages), 1.57x): Functional programming tools, with tests of map functions and their error handling built on rlang and vctrs
- **R6** ([`R6`](https://github.com/felix-andreas/ry/blob/main/tools/bench/packages), 2.58x): Encapsulated classes with reference semantics, with tests of inheritance, cloning and active bindings
- **rappdirs** ([`rappdirs`](https://github.com/felix-andreas/ry/blob/main/tools/bench/packages), 2.40x): Determines where to save data, caches and logs, with tests over platform and environment variables
- **rlang** ([`rlang`](https://github.com/felix-andreas/ry/blob/main/tools/bench/packages), 1.73x): Functions for base types and core R features, with a large suite of tests on environments, quosures and conditions
- **snakecase** ([`snakecase`](https://github.com/felix-andreas/ry/blob/main/tools/bench/packages), 1.44x): Converts strings into any case, with tests that run many string transformations
- **stringr** ([`stringr`](https://github.com/felix-andreas/ry/blob/main/tools/bench/packages), 1.73x): Consistent wrappers for common string operations, built on stringi
- **withr** ([`withr`](https://github.com/felix-andreas/ry/blob/main/tools/bench/packages), 2.14x): Runs code with temporarily modified global state, with tests of options, environment variables and on.exit handlers
- **xts** ([`xts`](https://github.com/felix-andreas/ry/blob/main/tools/bench/packages), 1.20x): Extensible time series built on zoo, with tests of merging, subsetting and periodicity
- **zeallot** ([`zeallot`](https://github.com/felix-andreas/ry/blob/main/tools/bench/packages), 2.26x): Multiple, unpacking and destructuring assignment with the %<-% operator
- **zoo** ([`zoo`](https://github.com/felix-andreas/ry/blob/main/tools/bench/packages), 2.34x): S3 infrastructure for regular and irregular time series, with scripts compared against saved output

## Numeric kernels (ours)

These are small programs we wrote whose time goes into arithmetic on numbers and numeric vectors:
recursion, scalar loops, element loops over vectors, whole-vector arithmetic and a few kernels from
the finance and biostatistics code bases. A process runs each program once cold and then five
times warm, and the median warm run is its time, so the time leaves out start-up, GNU R's byte
compilation and ry's first tier-up. Most programs also exist in typed ry, as a `<name>.typed.R`
beside the plain R, which runs in [strict mode](/guide.md#strict-mode) as the ry typed column. Every
engine must print the same checksum as GNU R, or the runner refuses to store the results.

Numeric kernels (ours), speed-up over GNU R:

| program | ry | ry typed |
| --- | ---: | ---: |
| Recursive Fibonacci (`fib`), too short on ry, too short on ry typed, wide interval on ry | 83x (76x to 93x) | 94x (91x to 102x) |
| Collatz chains (`integer_loop`), too short on ry | 13x (12x to 14x) | n/a |
| Drawdown and volatility (`drawdown`), too short on ry, too short on ry typed, wide interval on ry typed | 9.37x (9.17x to 9.51x) | 25x (22x to 27x) |
| Counted loop sum (`loop_sum`), too short on ry, too short on ry typed | 7.71x (7.49x to 7.79x) | 23x (22x to 24x) |
| Element map (`array_map`), too short on ry, too short on ry typed | 6.67x (6.53x to 6.80x) | 26x (24x to 27x) |
| Scale and shift (`vector_loop`), too short on ry | 5.82x (5.62x to 6.04x) | n/a |
| Dot product (`array_dot`), too short on ry, too short on ry typed, wide interval on ry typed | 5.62x (5.30x to 5.77x) | 38x (32x to 40x) |
| Counting filter (`array_filter`), too short on ry, too short on ry typed | 5.39x (5.17x to 5.56x) | 13x (12x to 13x) |
| Element sum (`array_sum`), too short on ry, too short on ry typed | 5.20x (5.05x to 5.38x) | 8.89x (8.34x to 9.22x) |
| Pairwise distances (`nested_loops`), too short on ry | 5.15x (5.08x to 5.42x) | n/a |
| Closures and lapply (`closures`), too short on ry typed, wide interval on ry typed | 3.28x (3.18x to 3.47x) | 152x (118x to 172x) |
| Path simulation (`simulation`) | 3.17x (2.93x to 3.36x) | n/a |
| Vector arithmetic (`vector_arith`), too short on ry, too short on ry typed | 2.82x (2.63x to 3.03x) | 3.08x (2.94x to 3.37x) |
| Vasicek rate paths (`vasicek`), typed code partly refused: `vasicek` runs in the interpreter | 1.91x (1.77x to 1.99x) | 2.17x (2.05x to 2.28x) |
| Jackknife estimate (`jackknife`) | 1.87x (1.83x to 1.94x) | n/a |
| Log-rank statistic (`log_rank`), typed code partly refused: `log_rank` runs in the interpreter; `count` runs in the interpreter | 1.36x (1.30x to 1.49x) | 0.05x (0.05x to 0.06x) |
| geometric mean | 5.26x (5.19x to 5.33x) | 11x (11x to 11x) |

A time marked too short on a ry engine is under 0.1 s. That speed is what the suite measures, so it counts in the geometric mean. A program marked typed code partly refused has code the typed core refused, which runs in the interpreter instead; the flag gives the refusal. A program marked wide interval varied so much between rounds that its ratio's interval is wide; read its median with care. Measured on 2026-11-11 at ry 4d96a342b with GNU R 4.6.2 on Intel(R) Xeon(R) Processor @ 2.30GHz, 4 cores, Linux 6.18.44-fc-v378; 10 paired rounds.

Recursion and scalar loops gain the most from the JIT. Kernels whose time goes into vector and
statistical routines gain the least, because those routines run in C on GNU R too. Typed scalar
loops and recursion gain more again, because a typed function in strict mode compiles to native
code before its first call.

### Time

| program | GNU R | ry | ry typed | ry, speed-up | ry typed, speed-up |
| --- | ---: | ---: | ---: | ---: | ---: |
| Dot product | 0.155 s | 0.028 s | 0.004 s | 5.62x (5.30x to 5.77x) (too short on ry) | 38x (32x to 40x) (too short on ry typed, wide interval on ry typed) |
| Counting filter | 0.120 s | 0.022 s | 0.009 s | 5.39x (5.17x to 5.56x) (too short on ry) | 13x (12x to 13x) (too short on ry typed) |
| Element map | 0.193 s | 0.029 s | 0.008 s | 6.67x (6.53x to 6.80x) (too short on ry) | 26x (24x to 27x) (too short on ry typed) |
| Element sum | 0.400 s | 0.076 s | 0.045 s | 5.20x (5.05x to 5.38x) (too short on ry) | 8.89x (8.34x to 9.22x) (too short on ry typed) |
| Closures and lapply | 0.491 s | 0.150 s | 0.003 s | 3.28x (3.18x to 3.47x) | 152x (118x to 172x) (too short on ry typed, wide interval on ry typed) |
| Drawdown and volatility | 0.209 s | 0.023 s | 0.009 s | 9.37x (9.17x to 9.51x) (too short on ry) | 25x (22x to 27x) (too short on ry typed, wide interval on ry typed) |
| Recursive Fibonacci | 0.500 s | 0.006 s | 0.005 s | 83x (76x to 93x) (too short on ry, wide interval on ry) | 94x (91x to 102x) (too short on ry typed) |
| Collatz chains | 0.427 s | 0.033 s | n/a | 13x (12x to 14x) (too short on ry) | n/a |
| Jackknife estimate | 0.363 s | 0.188 s | n/a | 1.87x (1.83x to 1.94x) | n/a |
| Log-rank statistic | 0.262 s | 0.190 s | 5.000 s | 1.36x (1.30x to 1.49x) | 0.05x (0.05x to 0.06x) (typed code partly refused: `log_rank` runs in the interpreter; `count` runs in the interpreter) |
| Counted loop sum | 0.162 s | 0.021 s | 0.007 s | 7.71x (7.49x to 7.79x) (too short on ry) | 23x (22x to 24x) (too short on ry typed) |
| Pairwise distances | 0.134 s | 0.026 s | n/a | 5.15x (5.08x to 5.42x) (too short on ry) | n/a |
| Path simulation | 0.364 s | 0.121 s | n/a | 3.17x (2.93x to 3.36x) | n/a |
| Vasicek rate paths | 0.218 s | 0.120 s | 0.101 s | 1.91x (1.77x to 1.99x) | 2.17x (2.05x to 2.28x) (typed code partly refused: `vasicek` runs in the interpreter) |
| Vector arithmetic | 0.153 s | 0.053 s | 0.050 s | 2.82x (2.63x to 3.03x) (too short on ry) | 3.08x (2.94x to 3.37x) (too short on ry typed) |
| Scale and shift | 0.207 s | 0.037 s | n/a | 5.82x (5.62x to 6.04x) (too short on ry) | n/a |

Each time is the median over the rounds. Each speed-up is the median of the per-round ratios, with its 95% interval in parentheses.

### Memory

| program | GNU R | ry | ry typed |
| --- | ---: | ---: | ---: |
| Dot product | 101 MiB | 119 MiB | 85 MiB |
| Counting filter | 101 MiB | 113 MiB | 77 MiB |
| Element map | 123 MiB | 102 MiB | 77 MiB |
| Element sum | 100 MiB | 116 MiB | 77 MiB |
| Closures and lapply | 139 MiB | 121 MiB | 58 MiB |
| Drawdown and volatility | 93 MiB | 135 MiB | 140 MiB |
| Recursive Fibonacci | 70 MiB | 49 MiB | 50 MiB |
| Collatz chains | 70 MiB | 55 MiB | n/a |
| Jackknife estimate | 125 MiB | 85 MiB | n/a |
| Log-rank statistic | 126 MiB | 76 MiB | 115 MiB |
| Counted loop sum | 71 MiB | 50 MiB | 50 MiB |
| Pairwise distances | 71 MiB | 75 MiB | n/a |
| Path simulation | 173 MiB | 141 MiB | n/a |
| Vasicek rate paths | 161 MiB | 97 MiB | 62 MiB |
| Vector arithmetic | 139 MiB | 79 MiB | 77 MiB |
| Scale and shift | 172 MiB | 192 MiB | n/a |

Peak memory is the peak resident set size of the whole process, R's startup included.

Measured on 2026-11-11 at ry 4d96a342b with GNU R 4.6.2 on Intel(R) Xeon(R) Processor @ 2.30GHz, 4 cores, Linux 6.18.44-fc-v378; 10 paired rounds.

### The programs

Each program as plain R and, where it exists, typed. The number beside the name is ry's speed-up
on the plain R.

- **Dot product** ([`array_dot`](https://github.com/felix-andreas/ry/blob/main/tools/bench/kernels/array_dot.R), 5.62x): An element loop over a million doubles summing `x[[i]] * y[[i]]`, five times: scalar indexing and arithmetic.

  Plain R:

  ```r
  array_dot <- function(x, y, times) {
    total <- 0
    n <- length(x)
    for (k in seq_len(times)) {
      for (i in seq_len(n)) total <- total + x[[i]] * y[[i]]
    }
    total
  }
  x <- seq(0, 1, length.out = 1000000L)
  y <- rev(x)
  measure("array_dot", function() array_dot(x, y, 5L))
  ```

  Typed:

  ```r
  array_dot <- function(x: List<F64>, y: List<F64>, times: I64) -> F64 {
    total: F64 <- 0
    n <- List.len(x)
    for (k in List.range(1, times)) {
      for (i in List.range(1, n)) total <- total + x[[i]] * y[[i]]
    }
    total
  }
  x <- List.from_r(seq(0, 1, length.out = 1000000L))
  y <- List.reverse(x)
  measure("array_dot", function() array_dot(x, y, 5))
  ```
- **Counting filter** ([`array_filter`](https://github.com/felix-andreas/ry/blob/main/tools/bench/kernels/array_filter.R), 5.39x): An element loop over a million doubles counting those above one half: indexing and comparison in a loop.

  Plain R:

  ```r
  array_filter <- function(x, y, times) {
    total <- 0
    n <- length(x)
    for (k in seq_len(times)) {
      for (i in seq_len(n)) if (x[[i]] > 0.5) total <- total + 1
    }
    total
  }
  x <- seq(0, 1, length.out = 1000000L)
  y <- rev(x)
  measure("array_filter", function() array_filter(x, y, 5L))
  ```

  Typed:

  ```r
  array_filter <- function(x: List<F64>, times: I64) -> F64 {
    total: F64 <- 0
    for (k in List.range(1, times)) {
      for (item in x) if (item > 0.5) total <- total + 1
    }
    total
  }
  x <- List.from_r(seq(0, 1, length.out = 1000000L))
  measure("array_filter", function() array_filter(x, 5))
  ```
- **Element map** ([`array_map`](https://github.com/felix-andreas/ry/blob/main/tools/bench/kernels/array_map.R), 6.67x): An element loop writing `x[[i]] * 2 + 1` into a preallocated vector of a million doubles: subassignment in a loop.

  Plain R:

  ```r
  array_map <- function(x, y, times) {
    total <- 0
    n <- length(x)
    out <- numeric(n)
    for (k in seq_len(times)) {
      for (i in seq_len(n)) out[[i]] <- x[[i]] * 2 + 1
      total <- total + sum(out)
    }
    total
  }
  x <- seq(0, 1, length.out = 1000000L)
  y <- rev(x)
  measure("array_map", function() array_map(x, y, 5L))
  ```

  Typed:

  ```r
  array_map <- function(x: List<F64>, times: I64) -> F64 {
    total <- 0
    for (k in List.range(1, times)) total <- total + List.sum(x * 2 + 1)
    total
  }
  x <- List.from_r(seq(0, 1, length.out = 1000000L))
  measure("array_map", function() array_map(x, 5))
  ```
- **Element sum** ([`array_sum`](https://github.com/felix-andreas/ry/blob/main/tools/bench/kernels/array_sum.R), 5.20x): An element loop summing a million doubles, twenty times: the cheapest indexed loop.

  Plain R:

  ```r
  array_sum <- function(x, y, times) {
    total <- 0
    n <- length(x)
    for (k in seq_len(times)) {
      for (i in seq_len(n)) total <- total + x[[i]]
    }
    total
  }
  x <- seq(0, 1, length.out = 1000000L)
  y <- rev(x)
  measure("array_sum", function() array_sum(x, y, 20L))
  ```

  Typed:

  ```r
  array_sum <- function(x: List<F64>, times: I64) -> F64 {
    total: F64 <- 0
    for (k in List.range(1, times)) {
      for (item in x) total <- total + item
    }
    total
  }
  x <- List.from_r(seq(0, 1, length.out = 1000000L))
  measure("array_sum", function() array_sum(x, 20))
  ```
- **Closures and lapply** ([`closures`](https://github.com/felix-andreas/ry/blob/main/tools/bench/kernels/closures.R), 3.28x): Small closures called in a loop, then lapply and vapply over a sequence: closure calls and argument matching.

  Plain R:

  ```r
  closures <- function(n) {
    add <- function(a, b) a + b
    scale <- 3
    times_scale <- function(a) a * scale
    s <- 0
    for (i in 1:n) s <- add(s, times_scale(i))
    squares <- lapply(seq_len(n), function(x) x * (x + 0.5))
    halves <- vapply(seq_len(n), function(x) x / 2, numeric(1))
    squared <- unlist(squares)
    s + sum(squared) + sum(halves)
  }
  measure("closures", function() closures(300000L))
  ```

  Typed:

  ```r
  closures <- function(n: I64) -> F64 {
    add <- function(a: F64, b: F64) -> F64 a + b
    scale <- 3
    times_scale <- function(a: F64) -> F64 a * scale
    s: F64 <- 0
    for (i in List.range(1, n)) s <- add(s, times_scale(Num.to_f64(i)))
    squares <- List.map(List.range(1, n), function(x: I64) -> F64 Num.to_f64(x) * (Num.to_f64(x) + 0.5))
    halves <- List.map(List.range(1, n), function(x: I64) -> F64 Num.to_f64(x) / 2)
    s + List.sum(squares) + List.sum(halves)
  }
  measure("closures", function() closures(300000))
  ```
- **Drawdown and volatility** ([`drawdown`](https://github.com/felix-andreas/ry/blob/main/tools/bench/kernels/drawdown.R), 9.37x): Maximum drawdown and EWMA volatility in one pass over a return series: scalar state carried through a loop.

  Plain R:

  ```r
  risk_report <- function(returns, lambda = 0.94) {
    wealth <- 1
    peak <- 1
    worst <- 0
    variance <- returns[1] * returns[1]
    for (i in seq_along(returns)) {
      r <- returns[i]
      wealth <- wealth * (1 + r)
      if (wealth > peak) peak <- wealth
      drawdown <- (peak - wealth) / peak
      if (drawdown > worst) worst <- drawdown
      variance <- lambda * variance + (1 - lambda) * r * r
    }
    c(max_drawdown = worst, volatility = sqrt(variance * 252))
  }
  returns <- sin(seq_len(2000000L) * 0.37) * 0.01 + 0.0002
  measure("drawdown", function() sum(risk_report(returns)))
  ```

  Typed:

  ```r
  risk_report <- function(returns: List<F64>, lambda: F64 = 0.94) -> F64 {
    wealth <- 1
    peak <- 1
    worst <- 0
    first <- returns[[1]]
    variance <- first * first
    for (r in returns) {
      wealth <- wealth * (1 + r)
      if (wealth > peak) peak <- wealth
      drawdown <- (peak - wealth) / peak
      if (drawdown > worst) worst <- drawdown
      variance <- lambda * variance + (1 - lambda) * r * r
    }
    worst + Num.sqrt(variance * 252)
  }
  returns <- List.from_r(sin(seq_len(2000000L) * 0.37) * 0.01 + 0.0002)
  measure("drawdown", function() risk_report(returns))
  ```
- **Recursive Fibonacci** ([`fib`](https://github.com/felix-andreas/ry/blob/main/tools/bench/kernels/fib.R), 83x): fib(30), about 2.7 million calls of a small recursive function: call overhead.

  Plain R:

  ```r
  fib <- function(n) if (n < 2) n else fib(n - 1) + fib(n - 2)
  measure("fib", function() fib(30))
  ```

  Typed:

  ```r
  fib <- function(n: I64) -> I64 if (n < 2) n else fib(n - 1) + fib(n - 2)
  measure("fib", function() Num.to_f64(fib(30)))
  ```
- **Collatz chains** ([`integer_loop`](https://github.com/felix-andreas/ry/blob/main/tools/bench/kernels/integer_loop.R), 13x): Collatz chain lengths up to 30,000 in nested while loops: integer arithmetic and branches.
- **Jackknife estimate** ([`jackknife`](https://github.com/felix-andreas/ry/blob/main/tools/bench/kernels/jackknife.R), 1.87x): A jackknife standard error through vapply and a closure that drops one element: vector copies per call.
- **Log-rank statistic** ([`log_rank`](https://github.com/felix-andreas/ry/blob/main/tools/bench/kernels/log_rank.R), 1.36x): The log-rank test statistic, a loop over event times with whole-vector comparisons in its body.

  Plain R:

  ```r
  log_rank <- function(data) {
    times <- sort(unique(data$time[data$event == 1L]))
    observed <- 0
    expected <- 0
    variance <- 0
    for (t in times) {
      risk <- data$time >= t
      n <- sum(risk)
      n1 <- sum(risk & data$arm == "treated")
      d <- sum(data$time == t & data$event == 1L)
      d1 <- sum(data$time == t & data$event == 1L & data$arm == "treated")
      observed <- observed + d1
      expected <- expected + d * n1 / n
      if (n > 1) variance <- variance + d * (n1 / n) * (1 - n1 / n) * (n - d) / (n - 1)
    }
    (observed - expected)^2 / variance
  }
  subjects <- 16000L
  data <- list(
    time = (seq_len(subjects) * 7919L) %% 1000L + 1L,
    event = as.integer(seq_len(subjects) %% 3L != 0L),
    arm = ifelse(seq_len(subjects) %% 2L == 0L, "treated", "control")
  )
  measure("log_rank", function() log_rank(data))
  ```

  Typed:

  ```r
  alias Trial = Record{time: List<I64>, event: List<Bool>, treated: List<Bool>}
  
  log_rank <- function(data: Trial) -> F64 {
    count <- function(mask: List<Bool>) -> F64 Num.to_f64(List.count_if(mask, function(b) b))
    times <- List.sort_asc(Set.to_list(Set.from_list(data$time[data$event])))
    observed <- 0
    expected <- 0
    variance <- 0
    for (t in times) {
      risk <- data$time >= t
      died <- data$time == t & data$event
      n <- count(risk)
      n1 <- count(risk & data$treated)
      d <- count(died)
      d1 <- count(died & data$treated)
      observed <- observed + d1
      expected <- expected + d * n1 / n
      if (n > 1) variance <- variance + d * (n1 / n) * (1 - n1 / n) * (n - d) / (n - 1)
    }
    (observed - expected) * (observed - expected) / variance
  }
  subjects <- List.range(1, 16000)
  data <- (
    time = List.map(subjects, function(i) (i * 7919) %% 1000 + 1),
    event = List.map(subjects, function(i) i %% 3 != 0),
    treated = List.map(subjects, function(i) i %% 2 == 0)
  )
  measure("log_rank", function() log_rank(data))
  ```
- **Counted loop sum** ([`loop_sum`](https://github.com/felix-andreas/ry/blob/main/tools/bench/kernels/loop_sum.R), 7.71x): A scalar double accumulator in a counted loop of ten million steps: the bare loop.

  Plain R:

  ```r
  loop_sum <- function(n) {
    s <- 0
    for (i in 1:n) s <- s + i * 0.5
    s
  }
  measure("loop_sum", function() loop_sum(10000000L))
  ```

  Typed:

  ```r
  loop_sum <- function(n: I64) -> F64 {
    s <- 0
    for (i in List.range(1, n)) s <- s + Num.to_f64(i) * 0.5
    s
  }
  measure("loop_sum", function() loop_sum(10000000))
  ```
- **Pairwise distances** ([`nested_loops`](https://github.com/felix-andreas/ry/blob/main/tools/bench/kernels/nested_loops.R), 5.15x): An O(n^2) pairwise distance sum in two nested loops: scalar indexing in an inner loop.
- **Path simulation** ([`simulation`](https://github.com/felix-andreas/ry/blob/main/tools/bench/kernels/simulation.R), 3.17x): Simulated price paths with matrix products, apply over rows, quantiles and sorting: vectorized model code.
- **Vasicek rate paths** ([`vasicek`](https://github.com/felix-andreas/ry/blob/main/tools/bench/kernels/vasicek.R), 1.91x): Vasicek interest-rate paths, one matrix row per step: vector arithmetic and matrix subassignment.

  Plain R:

  ```r
  vasicek_paths <- function(paths = 200L, steps = 20000L, r0 = 0.03, kappa = 0.5, theta = 0.04, sigma = 0.01, dt = 1 / 12, seed = 9L) {
    set.seed(seed)
    rates <- matrix(r0, nrow = steps + 1L, ncol = paths)
    for (t in seq_len(steps)) {
      shock <- stats::rnorm(paths)
      previous <- rates[t, ]
      rates[t + 1L, ] <- previous + kappa * (theta - previous) * dt + sigma * sqrt(dt) * shock
    }
    rates
  }
  measure("vasicek", function() sum(vasicek_paths()))
  ```

  Typed:

  ```r
  vasicek <- function(paths: I64 = 200, steps: I64 = 20000, r0: F64 = 0.03, kappa: F64 = 0.5, theta: F64 = 0.04, sigma: F64 = 0.01, dt: F64 = 1f64 / 12f64, seed: I64 = 9) -> F64 {
    set.seed(Num.to_r(seed))
    rates <- List.repeat(r0, paths)
    total <- List.sum(rates)
    for (t in List.range(1, steps)) {
      shock <- List.from_r(stats::rnorm(Num.to_r(paths)))
      rates <- rates + kappa * (theta - rates) * dt + sigma * Num.sqrt(dt) * shock
      total <- total + List.sum(rates)
    }
    total
  }
  measure("vasicek", function() vasicek())
  ```
- **Vector arithmetic** ([`vector_arith`](https://github.com/felix-andreas/ry/blob/main/tools/bench/kernels/vector_arith.R), 2.82x): Whole-vector arithmetic on a million doubles, repeated in a loop: allocation of large temporaries.

  Plain R:

  ```r
  vector_arith <- function(x, times) {
    total <- 0
    for (k in seq_len(times)) {
      y <- x * 2 + x / 3 - 1
      total <- total + sum(y)
    }
    total
  }
  x <- seq(0, 1, length.out = 1000000L)
  measure("vector_arith", function() vector_arith(x, 20L))
  ```

  Typed:

  ```r
  vector_arith <- function(x: List<F64>, times: I64) -> F64 {
    total <- 0
    for (k in List.range(1, times)) total <- total + List.sum(x * 2 + x / 3 - 1)
    total
  }
  x <- List.from_r(seq(0, 1, length.out = 1000000L))
  measure("vector_arith", function() vector_arith(x, 20))
  ```
- **Scale and shift** ([`vector_loop`](https://github.com/felix-andreas/ry/blob/main/tools/bench/kernels/vector_loop.R), 5.82x): Element-wise reads and writes of a double vector in a loop: `y[i] <- x[i] * 2 + 1`.

## Application workloads (ours)

These are programs we wrote in the shape of the code that packages and analyses run: validation at
API boundaries, condition handling, S3, S4 and R6 objects, records as named lists, string building,
data frames and data.table, time series and a testthat suite. Their time goes into R's object and
data machinery rather than into arithmetic. They are measured like the kernels, and six of them
also exist in typed ry.

Application workloads (ours), speed-up over GNU R:

| program | ry | ry typed |
| --- | ---: | ---: |
| testthat suite (`testthat`), wide interval on ry | 3.38x (2.93x to 3.58x) | n/a |
| Argument validation (`validation`) | 2.81x (2.58x to 2.92x) | n/a |
| Records as lists (`records`) | 2.24x (2.03x to 2.28x) | n/a |
| String handling (`strings`) | 2.12x (1.99x to 2.21x) | n/a |
| Sales table pipeline (`sales`), too short on ry, too short on ry typed, wide interval on ry, typed code partly refused: `pipeline` runs in the interpreter | 2.04x (1.82x to 2.16x) | 2.60x (2.50x to 2.79x) |
| Order invoicing (`orders`), typed code partly refused: refusing `.ry_record`; no code for [I64]: a base function the typed core does not know; `price_orders` runs in the interpreter; refusing `.ry_tag`; `region_of` runs in the interpreter; `tax_rate` runs in the interpreter | 1.85x (1.81x to 2.00x) | 0.79x (0.75x to 0.83x) |
| Bond book pricing (`bond_book`), too short on ry typed, wide interval on ry | 1.83x (1.69x to 2.01x) | 9.07x (8.30x to 9.64x) |
| Functional style (`functional`), wide interval on ry | 1.78x (1.70x to 1.96x) | n/a |
| Object systems (`objects`) | 1.77x (1.67x to 1.93x) | n/a |
| Expression tree (`expr_tree`), wide interval on ry, typed code partly refused: refusing `.ry_tag`; no code for [I64, I64]: a base function the typed core does not know; `build` runs in the interpreter; `evaluate` runs in the interpreter | 1.76x (1.55x to 1.84x) | 1.35x (1.26x to 1.47x) |
| Row validation (`team_scores`), wide interval on ry, wide interval on ry typed, typed code partly refused: no code for [Vector { item: Str, array: true }]: a mapped function that is not a local function; `team_scores` runs in the interpreter; refusing `Result.with_default`; no code for [Str]: a base function the typed core does not know; `score` runs in the interpreter | 1.62x (1.51x to 1.81x) | 2.18x (1.89x to 2.35x) |
| Report formatting (`report`), wide interval on ry, wide interval on ry typed, typed code partly refused: refusing `.ry_text`; no code for [I64]: a base function the typed core does not know; `report` runs in the interpreter; `padded` runs in the interpreter | 1.54x (1.36x to 1.74x) | 3.07x (2.72x to 3.47x) |
| Time series and dates (`timeseries`) | 1.40x (1.33x to 1.44x) | n/a |
| Condition handling (`conditions`) | 1.28x (1.16x to 1.32x) | n/a |
| Tables (`tables`) | 1.27x (1.18x to 1.36x) | n/a |
| geometric mean | 1.84x (1.80x to 1.87x) | 2.35x (2.25x to 2.43x) |

A time marked too short on a ry engine is under 0.1 s. That speed is what the suite measures, so it counts in the geometric mean. A program marked typed code partly refused has code the typed core refused, which runs in the interpreter instead; the flag gives the refusal. A program marked wide interval varied so much between rounds that its ratio's interval is wide; read its median with care. Measured on 2026-11-11 at ry 4d96a342b with GNU R 4.6.2 on Intel(R) Xeon(R) Processor @ 2.30GHz, 4 cores, Linux 6.18.44-fc-v378; 10 paired rounds.

These programs gain less than the kernels, because call setup, method dispatch, condition handling
and complex assignment decide their time, not loops the JIT compiles. Typing gains the most where
the typed core compiles the whole program. Where it refuses part of a program, that part runs in
the interpreter, and the flag under the bar names what was refused.

### Time

| program | GNU R | ry | ry typed | ry, speed-up | ry typed, speed-up |
| --- | ---: | ---: | ---: | ---: | ---: |
| Bond book pricing | 0.433 s | 0.228 s | 0.046 s | 1.83x (1.69x to 2.01x) (wide interval on ry) | 9.07x (8.30x to 9.64x) (too short on ry typed) |
| Condition handling | 0.441 s | 0.336 s | n/a | 1.28x (1.16x to 1.32x) | n/a |
| Expression tree | 0.274 s | 0.157 s | 0.203 s | 1.76x (1.55x to 1.84x) (wide interval on ry) | 1.35x (1.26x to 1.47x) (typed code partly refused: refusing `.ry_tag`; no code for [I64, I64]: a base function the typed core does not know; `build` runs in the interpreter; `evaluate` runs in the interpreter) |
| Functional style | 0.496 s | 0.269 s | n/a | 1.78x (1.70x to 1.96x) (wide interval on ry) | n/a |
| Object systems | 0.581 s | 0.329 s | n/a | 1.77x (1.67x to 1.93x) | n/a |
| Order invoicing | 0.371 s | 0.199 s | 0.467 s | 1.85x (1.81x to 2.00x) | 0.79x (0.75x to 0.83x) (typed code partly refused: refusing `.ry_record`; no code for [I64]: a base function the typed core does not know; `price_orders` runs in the interpreter; refusing `.ry_tag`; `region_of` runs in the interpreter; `tax_rate` runs in the interpreter) |
| Records as lists | 0.407 s | 0.185 s | n/a | 2.24x (2.03x to 2.28x) | n/a |
| Report formatting | 0.433 s | 0.280 s | 0.140 s | 1.54x (1.36x to 1.74x) (wide interval on ry) | 3.07x (2.72x to 3.47x) (wide interval on ry typed, typed code partly refused: refusing `.ry_text`; no code for [I64]: a base function the typed core does not know; `report` runs in the interpreter; `padded` runs in the interpreter) |
| Sales table pipeline | 0.105 s | 0.052 s | 0.040 s | 2.04x (1.82x to 2.16x) (too short on ry, wide interval on ry) | 2.60x (2.50x to 2.79x) (too short on ry typed, typed code partly refused: `pipeline` runs in the interpreter) |
| String handling | 0.642 s | 0.300 s | n/a | 2.12x (1.99x to 2.21x) | n/a |
| Tables | 0.287 s | 0.234 s | n/a | 1.27x (1.18x to 1.36x) | n/a |
| Row validation | 0.674 s | 0.399 s | 0.315 s | 1.62x (1.51x to 1.81x) (wide interval on ry) | 2.18x (1.89x to 2.35x) (wide interval on ry typed, typed code partly refused: no code for [Vector { item: Str, array: true }]: a mapped function that is not a local function; `team_scores` runs in the interpreter; refusing `Result.with_default`; no code for [Str]: a base function the typed core does not know; `score` runs in the interpreter) |
| testthat suite | 0.529 s | 0.150 s | n/a | 3.38x (2.93x to 3.58x) (wide interval on ry) | n/a |
| Time series and dates | 0.933 s | 0.668 s | n/a | 1.40x (1.33x to 1.44x) | n/a |
| Argument validation | 0.548 s | 0.195 s | n/a | 2.81x (2.58x to 2.92x) | n/a |

Each time is the median over the rounds. Each speed-up is the median of the per-round ratios, with its 95% interval in parentheses.

### Memory

| program | GNU R | ry | ry typed |
| --- | ---: | ---: | ---: |
| Bond book pricing | 71 MiB | 63 MiB | 51 MiB |
| Condition handling | 71 MiB | 65 MiB | n/a |
| Expression tree | 185 MiB | 154 MiB | 88 MiB |
| Functional style | 122 MiB | 105 MiB | n/a |
| Object systems | 83 MiB | 102 MiB | n/a |
| Order invoicing | 72 MiB | 65 MiB | 69 MiB |
| Records as lists | 149 MiB | 109 MiB | n/a |
| Report formatting | 74 MiB | 70 MiB | 85 MiB |
| Sales table pipeline | 183 MiB | 148 MiB | 318 MiB |
| String handling | 108 MiB | 106 MiB | n/a |
| Tables | 97 MiB | 134 MiB | n/a |
| Row validation | 124 MiB | 128 MiB | 131 MiB |
| testthat suite | 101 MiB | 83 MiB | n/a |
| Time series and dates | 123 MiB | 123 MiB | n/a |
| Argument validation | 71 MiB | 63 MiB | n/a |

Peak memory is the peak resident set size of the whole process, R's startup included.

Measured on 2026-11-11 at ry 4d96a342b with GNU R 4.6.2 on Intel(R) Xeon(R) Processor @ 2.30GHz, 4 cores, Linux 6.18.44-fc-v378; 10 paired rounds.

- **Bond book pricing** ([`bond_book`](https://github.com/felix-andreas/ry/blob/main/tools/bench/applications/bond_book.R), 1.83x): Prices 200,000 bonds, each a short vector computation through seq_len, rep, exp and sum: many small base calls.

  Plain R:

  ```r
  bond_price <- function(coupon, rate, years) {
    times <- seq_len(years)
    flows <- rep(coupon, years)
    flows[[years]] <- flows[[years]] + 100
    sum(flows * exp(-rate * times))
  }
  price_book <- function(n) {
    total <- 0
    for (i in seq_len(n)) {
      total <- total + bond_price(2 + i %% 5L, 0.01 + (i %% 7L) * 0.003, 5L + i %% 20L)
    }
    round(total, 6)
  }
  measure("bond_book", function() price_book(200000L))
  ```

  Typed:

  ```r
  bond_price <- function(coupon: F64, rate: F64, years: I64) -> F64 {
    List.walk(List.range(1, years), 0, function(total, year) {
      flow <- if (year == years) coupon + 100 else coupon
      total + flow * Num.pow(Num.e, -rate * Num.to_f64(year))
    })
  }
  price_book <- function(n: I64) -> F64 {
    total <- 0
    for (i in List.range(1, n)) {
      total <- total + bond_price(Num.to_f64(2 + i %% 5), 0.01 + Num.to_f64(i %% 7) * 0.003, 5 + i %% 20)
    }
    Num.to_f64(Num.round(total * 1000000)) / 1000000
  }
  measure("bond_book", function() price_book(200000))
  ```
- **Condition handling** ([`conditions`](https://github.com/felix-andreas/ry/blob/main/tools/bench/applications/conditions.R), 1.28x): tryCatch, withCallingHandlers, custom condition classes, retries and on.exit around fallible steps.
- **Expression tree** ([`expr_tree`](https://github.com/felix-andreas/ry/blob/main/tools/bench/applications/expr_tree.R), 1.76x): Builds and evaluates a tree of sums, maxima and negations: recursion over nested lists.

  Plain R:

  ```r
  build <- function(depth, seed) {
    if (depth == 0L) return(list(kind = "num", value = seed %% 10L + 1))
    left <- build(depth - 1L, (seed * 7L) %% 1009L + 1L)
    if (seed %% 5L == 0L) return(list(kind = "neg", inner = left))
    right <- build(depth - 1L, (seed * 13L) %% 1013L + 3L)
    if (seed %% 2L == 0L) list(kind = "add", left = left, right = right) else list(kind = "max", left = left, right = right)
  }
  evaluate <- function(node) {
    switch(node$kind,
      num = node$value,
      neg = -evaluate(node$inner),
      add = evaluate(node$left) + evaluate(node$right),
      max = max(evaluate(node$left), evaluate(node$right))
    )
  }
  tree <- build(21L, 5L)
  measure("expr_tree", function() evaluate(tree) + evaluate(build(14L, 8L)))
  ```

  Typed:

  ```r
  type Node = :Num(value: F64) | :Neg(inner: Node) | :Add(left: Node, right: Node) | :Max(left: Node, right: Node)
  
  build <- function(depth: I64, seed: I64) -> Node {
    if (depth == 0) return(@Node(:Num(value = Num.to_f64(seed %% 10 + 1))))
    left <- build(depth - 1, (seed * 7) %% 1009 + 1)
    if (seed %% 5 == 0) return(@Node(:Neg(inner = left)))
    right <- build(depth - 1, (seed * 13) %% 1013 + 3)
    if (seed %% 2 == 0) @Node(:Add(left = left, right = right)) else @Node(:Max(left = left, right = right))
  }
  evaluate <- function(node: Node) -> F64 {
    match (node) {
      :Num(value = v) => v
      :Neg(inner = e) => -evaluate(e)
      :Add(left = a, right = b) => evaluate(a) + evaluate(b)
      :Max(left = a, right = b) => Num.max(evaluate(a), evaluate(b))
    }
  }
  tree <- build(21, 5)
  measure("expr_tree", function() evaluate(tree) + evaluate(build(14, 8)))
  ```
- **Functional style** ([`functional`](https://github.com/felix-andreas/ry/blob/main/tools/bench/applications/functional.R), 1.78x): lapply, Map, Filter, Reduce, do.call and purrr over lists, function factories and memoization in an environment.
- **Object systems** ([`objects`](https://github.com/felix-andreas/ry/blob/main/tools/bench/applications/objects.R), 1.77x): S3 methods with UseMethod and NextMethod, S4 classes with validity, R6 classes with active bindings.
- **Order invoicing** ([`orders`](https://github.com/felix-andreas/ry/blob/main/tools/bench/applications/orders.R), 1.85x): Prices order lines with a loyalty discount and regional tax, then totals invoices: records and branches.

  Plain R:

  ```r
  price_line <- function(order, rates) {
    net <- order$quantity * order$unit
    if (order$loyal) net <- net * 0.95
    if (order$quantity > 5L) net <- net - 1
    tax <- net * rates[[order$region]]
    list(id = order$id, net = net, tax = tax, gross = net + tax)
  }
  price_orders <- function(n) {
    regions <- c("north", "south", "east", "west")
    rates <- c(north = 0.2, south = 0.1, east = 0.15, west = 0.05)
    total <- 0
    for (i in seq_len(n)) {
      order <- list(id = i, quantity = i %% 7L + 1L, unit = 10 + (i %% 13L) * 0.5, loyal = i %% 3L == 0L, region = regions[[i %% 4L + 1L]])
      invoice <- price_line(order, rates)
      total <- total + invoice$gross
    }
    total
  }
  measure("orders", function() price_orders(200000L))
  ```

  Typed:

  ```r
  alias Region = :North | :South | :East | :West
  alias Order = Record{id: I64, quantity: I64, unit: F64, loyal: Bool, region: Region}
  alias Invoice = Record{id: I64, net: F64, tax: F64, gross: F64}
  
  tax_rate <- function(region: Region) -> F64 {
    match (region) {
      :North => 0.2
      :South => 0.1
      :East => 0.15
      :West => 0.05
    }
  }
  price_line <- function(order: Order) -> Invoice {
    net <- Num.to_f64(order$quantity) * order$unit
    if (order$loyal) net <- net * 0.95
    if (order$quantity > 5) net <- net - 1
    tax <- net * tax_rate(order$region)
    (id = order$id, net = net, tax = tax, gross = net + tax)
  }
  region_of <- function(i: I64) -> Region {
    k <- i %% 4
    if (k == 0) :North else if (k == 1) :South else if (k == 2) :East else :West
  }
  price_orders <- function(n: I64) -> F64 {
    total <- 0
    for (i in List.range(1, n)) {
      order <- (id = i, quantity = i %% 7 + 1, unit = 10 + Num.to_f64(i %% 13) * 0.5, loyal = i %% 3 == 0, region = region_of(i))
      invoice <- price_line(order)
      total <- total + invoice$gross
    }
    total
  }
  measure("orders", function() price_orders(200000))
  ```
- **Records as lists** ([`records`](https://github.com/felix-andreas/ry/blob/main/tools/bench/applications/records.R), 2.24x): Named lists grown, updated with `$<-` and `[[<-`, merged with modifyList, grouped with split and bound with rbind.
- **Report formatting** ([`report`](https://github.com/felix-andreas/ry/blob/main/tools/bench/applications/report.R), 1.54x): Formats one line per account, then joins and measures the report: string building.

  Plain R:

  ```r
  report <- function(n) {
    lines <- character(n)
    for (i in seq_len(n)) {
      name <- paste0("ACC", formatC(i, width = 6L, flag = "0"))
      side <- if (i %% 2L == 0L) "long" else "short"
      lines[[i]] <- paste(name, sprintf("%.2f", i * 1.25), side, sep = ";")
    }
    text <- paste(lines, collapse = "\n")
    nchar(text) + sum(nchar(toupper(lines)) > 25L) + sum(startsWith(lines, "ACC00"))
  }
  measure("report", function() report(20000L))
  ```

  Typed:

  ```r
  report <- function(n: I64) -> F64 {
    padded <- function(digits: Str, width: I64) -> Str
      Str.concat(Str.repeat("0", width - Str.count_utf8_bytes(digits)), digits)
    lines <- List.map(List.range(1, n), function(i: I64) -> Str {
      cents <- i * 125
      amount <- f"{cents %/% 100}.{padded(Num.to_str(cents %% 100), 2)}"
      side <- if (i %% 2 == 0) "long" else "short"
      f"ACC{padded(Num.to_str(i), 6)};{amount};{side}"
    })
    text <- Str.join_with(lines, "\n")
    long <- List.count_if(lines, function(line) Str.count_utf8_bytes(Str.with_ascii_uppercased(line)) > 25)
    early <- List.count_if(lines, function(line) Str.starts_with(line, "ACC00"))
    Num.to_f64(Str.count_utf8_bytes(text) + long + early)
  }
  measure("report", function() report(20000))
  ```
- **Sales table pipeline** ([`sales`](https://github.com/felix-andreas/ry/blob/main/tools/bench/applications/sales.R), 2.04x): Filters a million sales rows, adds a revenue column and totals it per region: data frame indexing and tapply.

  Plain R:

  ```r
  pipeline <- function(sales) {
    big <- sales[sales$units > 3L, ]
    big$revenue <- big$units * big$price
    totals <- tapply(big$revenue, big$region, sum)
    sum(totals * seq_along(totals))
  }
  i <- seq_len(1000000L)
  sales <- data.frame(region = c("north", "south", "east", "west")[i %% 4L + 1L], units = i %% 17L, price = 1 + i %% 100L)
  measure("sales", function() pipeline(sales))
  ```

  Typed:

  ```r
  pipeline <- function(sales: DataFrame{region: character[], units: integer[], price: double[]}) -> double {
    totals <- sales |>
      DataFrame.filter(Expr.col("units") > 3L) |>
      DataFrame.with_columns(revenue = Expr.col("units") * Expr.col("price")) |>
      DataFrame.group_by("region") |>
      GroupBy.agg(total = Expr.sum(Expr.col("revenue"))) |>
      DataFrame.sort("region")
    sum(totals$total * seq_along(totals$total))
  }
  i <- seq_len(1000000L)
  sales <- DataFrame(region = c("north", "south", "east", "west")[i %% 4L + 1L], units = i %% 17L, price = 1 + i %% 100L)
  measure("sales", function() Num.from_r(pipeline(sales)))
  ```
- **String handling** ([`strings`](https://github.com/felix-andreas/ry/blob/main/tools/bench/applications/strings.R), 2.12x): paste, sprintf, gsub, regmatches, splitting and padding, and the same through stringr.
- **Tables** ([`tables`](https://github.com/felix-andreas/ry/blob/main/tools/bench/applications/tables.R), 1.27x): data.table grouping, `:=` updates, keyed joins and rbindlist, and the same steps on base data frames.
- **Row validation** ([`team_scores`](https://github.com/felix-andreas/ry/blob/main/tools/bench/applications/team_scores.R), 1.62x): Parses comma-separated rows, rejects invalid ones and totals scores per team: strsplit, conversion and failures as values.

  Plain R:

  ```r
  parse_row <- function(row) {
    parts <- strsplit(row, ",", fixed = TRUE)[[1L]]
    age <- suppressWarnings(as.integer(parts[[2L]]))
    if (is.na(age)) return(list(ok = FALSE, error = "bad_age"))
    if (age < 18L) return(list(ok = FALSE, error = "minor"))
    list(ok = TRUE, value = list(name = parts[[1L]], age = age, score = as.double(parts[[3L]]), team = parts[[4L]]))
  }
  team_scores <- function(rows) {
    parsed <- lapply(rows, parse_row)
    totals <- list()
    errors <- 0L
    for (p in parsed) {
      if (p$ok) {
        team <- p$value$team
        old <- totals[[team]]
        totals[[team]] <- (if (is.null(old)) 0 else old) + p$value$score
      } else {
        errors <- errors + 1L
      }
    }
    totals[["red"]] + 2 * totals[["blue"]] + 3 * totals[["green"]] + errors
  }
  i <- seq_len(60000L)
  rows <- paste0("user", i, ",", ifelse(i %% 17L == 0L, "x", as.character(i %% 90L)), ",", (i %% 50L) / 2, ",", c("red", "blue", "green")[i %% 3L + 1L])
  measure("team_scores", function() team_scores(rows))
  ```

  Typed:

  ```r
  alias Person = Record{name: Str, age: I64, score: F64, team: Str}
  
  parse_row <- function(row: Str) -> Result<Person, :BadAge | :BadScore | :Minor> {
    parts <- Str.split_on(row, ",")
    match (Str.to_i64(parts[[2]])) {
      :Err(_) => :Err(:BadAge)
      :Ok(age) => if (age < 18) :Err(:Minor) else match (Str.to_f64(parts[[3]])) {
        :Err(_) => :Err(:BadScore)
        :Ok(score) => :Ok((name = parts[[1]], age = age, score = score, team = parts[[4]]))
      }
    }
  }
  team_scores <- function(rows: List<Str>) -> F64 {
    parsed <- List.map(rows, parse_row)
    people <- List.keep_oks(parsed, function(p: Result<Person, :BadAge | :BadScore | :Minor>) -> Result<Person, :BadAge | :BadScore | :Minor> p)
    totals <- List.walk(people, Dict.empty(), function(seen: Dict<Str, F64>, person: Person) -> Dict<Str, F64> {
      Dict.update(seen, person$team, function(found: Result<F64, :Missing>) -> Result<F64, :Missing> :Ok(Result.with_default(found, 0) + person$score))
    })
    errors <- Num.to_f64(List.len(parsed) - List.len(people))
    score <- function(team: Str) -> F64 Result.with_default(Dict.get(totals, team), 0)
    score("red") + 2 * score("blue") + 3 * score("green") + errors
  }
  i <- seq_len(60000L)
  rows <- List.from_r(paste0("user", i, ",", ifelse(i %% 17L == 0L, "x", as.character(i %% 90L)), ",", (i %% 50L) / 2, ",", c("red", "blue", "green")[i %% 3L + 1L]))
  measure("team_scores", function() team_scores(rows))
  ```
- **testthat suite** ([`testthat`](https://github.com/felix-andreas/ry/blob/main/tools/bench/applications/testthat.R), 3.38x): Many small test_that() blocks with expect_equal, expect_error and expect_warning: testthat's condition machinery.
- **Time series and dates** ([`timeseries`](https://github.com/felix-andreas/ry/blob/main/tools/bench/applications/timeseries.R), 1.40x): xts and zoo series merged, lagged and rolled, lubridate parsing and month arithmetic, base Date sequences.
- **Argument validation** ([`validation`](https://github.com/felix-andreas/ry/blob/main/tools/bench/applications/validation.R), 2.81x): Small functions with defaults, `...` forwarding, match.arg, stopifnot, missing() and match.call().

## How it was measured

A run measures several rounds. Each round runs every program on every engine once, with the
programs in an order shuffled per round and the engines in an order shuffled per program, from a
recorded seed. Each ratio is formed inside one round, GNU R's time over the engine's, and the
reported speed-up is the median of those per-round ratios. Pairing matters because the measuring
machines are shared cloud VMs whose effective CPU speed drifts with other tenants' load: a slow
drift moves both engines of one round together, so it cancels in the ratio. Each speed-up and each
geometric mean carries a 95% bootstrap interval over the rounds.

What one timed run covers differs by suite:

| Suite | One process runs | The time includes |
| --- | --- | --- |
| Public R JIT benchmarks | the program once | byte compilation and JIT tier-up, not start-up |
| Package test suites | the package's tests | start-up and loading the package and its imports |
| Numeric kernels, application workloads | the program once cold, then five times warm | the median warm run only |

The runner, not this page, judges the data:

- **Checksums.** Every engine must print the same checksum as GNU R for a program, or the run fails
  and nothing is written. A numeric checksum may differ by a relative 1e-10, because a typed program
  may sum in a different order. A program that fails on any engine also fails the run.
- **Package tests.** The packages suite is the exception, because there the program is the
  package's own test suite. Test counts that differ between the engines are flagged as tests
  differ, and the package stays in the geometric mean. A test script that ends in an R error, as
  `R CMD check` reads a failing script, counts as one failed test. A test suite that crashes or
  does not run to the end on an engine is recorded as failing there, which leaves the package out
  of the geometric mean.
- **Too short.** A program GNU R finishes in under 0.1 s does too little work for a stable ratio, so
  it is flagged too short and left out of every geometric mean. A ry time under 0.1 s is flagged
  too, but it counts, because that speed is what the suite measures.
- **Wide intervals.** A ratio whose interval reaches more than 10% from its median gets extra
  rounds, and it is flagged wide interval if they do not narrow it.
- **Refused typed programs.** Where the typed core refuses part of a typed program, that part runs
  in the interpreter, and its flag gives the refusal.

| Suite | Reproduce | GNU R | Engines | Samples |
| --- | --- | --- | --- | --- |
| Public R JIT benchmarks | `just bench public --write` | 4.6.2, BLAS `libblas.so.3.12.0` | GNU R, ry | 10 paired rounds, 10 more for each wide interval, seed 1 |
| Package test suites | `just bench packages --write` | 4.6.2, BLAS `libblas.so.3.12.0` | GNU R, ry | 10 paired rounds, 10 more for each wide interval, seed 1 |
| Numeric kernels (ours) | `just bench kernels --write` | 4.6.2, BLAS `libblas.so.3.12.0` | GNU R, ry, ry typed | 10 paired rounds, 10 more for each wide interval, seed 1 |
| Application workloads (ours) | `just bench applications --write` | 4.6.2, BLAS `libblas.so.3.12.0` | GNU R, ry, ry typed | 10 paired rounds, 10 more for each wide interval, seed 1 |

To reproduce a suite, run its command from the table on a machine with GNU R; the packages suite
first needs `just bench-packages install`. With `--write`, the runner stores
`tools/bench/<suite>/results.json` with the commit that built ry, and `just readme` rewrites
the README summary from it. `--write` refuses a working tree with a change to a tracked file or an
untracked file under `crates/` or `tools/bench/`, so every stored commit reproduces its
measurement. The README summary warns when the suites were measured on different machines (CPU
model or kernel) or against a GNU R on a different BLAS, because their ratios are then not
comparable.
