# Development

ry is a Rust workspace whose crates live under `crates/`. [Structure](/development/structure.md) maps
every source file, and [Architecture](/development/architecture.md) explains how the analysis fits
together. The editor extensions live in `editors/code` (VS Code) and `editors/zed` (Zed).

## Build and test

The workspace builds on the dated nightly that `rust-toolchain.toml` pins, because std's allocator
API and other unstable features are worth having before they stabilize (the policy is in
`.agents/memory/decisions.md`, "Data-oriented design and explicit allocators"). With rustup, cargo
downloads that toolchain, with clippy and rustfmt, on the first build. The Zed extension is the
exception: `editors/zed/rust-toolchain.toml` pins a stable toolchain, because the nightly's
`wasm32-wasip2` std imports a WASI function that Zed's host does not provide yet.
The nix dev shell reads the same file. Nightly rustfmt also applies the import settings in
`rustfmt.toml`, so `cargo fmt` keeps a single `use { ... }` tree per file.

The `justfile` at the repository root names every day-to-day action, and `just` with no arguments
lists them all. The common ones:

```sh
just ry check                       # build ry and its runtime if needed, then run `ry check`
just build                          # every binary and ry's runtime (`just build --release`)
just try                            # build ry and walk through every command in a demo project
just try-editor                     # build ry and print how to point an editor at it
just gate                           # tests, clippy -D warnings, and fmt --check
just battery                        # all workspace tests, excluding zed_ry
just fixture <group__case>          # one fixture case
just bless -p semantics --test ...  # rewrite fixture expectations from current output
just docs-test [page]               # run the docs' marked examples through the real binary
just stats <path>                   # phase timings and memory for one workspace
just perf                           # corpus perf and memory budgets (needs corpus/)
just repl-e2e                       # drive `ry R` and `ry run` through a pty
just release-dry-run                # package and smoke-test this machine's release archive
just docs-deploy <branch>           # build the docs site and deploy it (main is production)
```

The simulators, fuzzers and long oracle runs have their own recipes, which
[Robustness](/development/robustness.md#running-the-long-harnesses) lists.

`just fixture` runs a case from the `test_typing_fixtures` target in `semantics` unless you pass a
different target and package as its second and third arguments. Review the diff after every bless:
bless records whatever the code produces, right or wrong.

Without `just`:

```sh
cargo build                              # the ry binary (the workspace default member)
cargo test                               # the ry crate's tests
cargo test --workspace --exclude zed_ry  # every crate
cargo test -p semantics                  # the analysis core, the usual loop while iterating
```

Most behavior is pinned by fixture tests, plain-text `.test` files whose rendered output is compared
to an expected block. Read [Testing](/development/testing.md) before you add or change one. Two
environment variables come up every day:

- `RY_BLESS=1` rewrites the expected `#++++` blocks in place.
- `FIXTURE_FILTER=group__case` runs a single case.

Some suites and every performance instrument read a corpus of real-world R code. Fetch it with
`cargo -Zscript scripts/fetch-corpus.rs`, which writes into the gitignored `corpus/`
directory. The resolved inventory is committed as `scripts/corpus-manifest.txt`.

### The R console needs no R to build

The `repl` crate loads R's shared library at runtime instead of linking against it, so the whole
workspace builds and its tests run on a machine with no R. Only running the console needs R. The
end-to-end tests skip cleanly where R is missing; run `just repl-e2e` before you touch the console.
[REPL design](/development/design/repl.md) records how the embedding works.

### ry's runtime is built separately

`ry run` and `ry repl --runtime ry` load ry's own runtime, `libR.so`, from the directory of the
`ry` executable. It is a separate crate (`libr`), so `cargo build -p ry-lang` does not rebuild it;
`just ry` and `just build` do. To keep a leftover library from running old code, `ry` compares a
hash of the runtime's sources with the one stored in `libR.so` right after loading it, and refuses
a mismatch with exit status 2:

```text
  x ry's runtime (target/debug/libR.so) is out of date: it has build 8fcdf6d9f4862983, but this ry is build
  | 2588183553632a45; rebuild it with `just build` or `cargo build -p libr` (same profile as ry)
```

Build `libr` with the same profile as `ry` and the error goes away. The hash covers the sources
of `libr` and the crates it links, so editing the formatter or the IDE layer never triggers it.

## Debugging

- `ry server --debug`, or `RY_DEBUG=1` in the server's environment, adds a "Debug" section to every
  hover with the Lowering, Naming, and Parsing views of the expression under the cursor. It is
  deliberately not a `ry.toml` key, because the configuration file is a user-facing contract.
- `ry debug ast <file>` prints a file's syntax tree.
- `ry debug erase <file>...` rewrites files in place as their erasure, the plain R that `ry R` runs.
- `ry debug analysis-stats [path]` (or `just stats`) runs the full pipeline over a workspace through
  the language server's queries, with `[check] typing` forced on, and prints:
  - the wall time and resident-memory growth of each phase, and the slowest files to type-check;
  - keystroke latency and rechecks per keystroke, from an incremental probe;
  - how often hover, goto-definition and signature help answer at sampled identifiers, by role
    (call head, variable, argument name, `$` field, `pkg::name`);
  - how often completion of a truncated identifier puts the original name in the top five, with
    latency percentiles;
  - whether find-references on sampled package bindings agrees with an independent lexical resolver
    (`crates/ry/src/scope_oracle.rs`), and whether renaming a binding and back restores the text.

  Build with `--release` when absolute numbers matter; a debug build gives honest ratios.

## The docs site

The sidebar (`docs/astro.config.ts`) has four sections: Introduction, Tutorial (one page per
adoption step), Reference and Development. A moved or deleted page gets an entry in the
`redirects` map there. `docs/src/components/Sidebar.astro` keeps the sidebar's scroll position
across page loads, because the theme scrolls `.sidebar-content` rather than the element
Starlight's own persister restores.

Typed examples are tab groups inside `<Tabs syncKey="annotation-style">`, so a reader's choice
follows them across pages: `Untyped` (the plain R a reader starts from), then `Comments` and
`Inline`, with the form the page teaches second: `Comments` on the tutorial's types page and
`Inline` elsewhere. An example drops `Untyped` when its plain R would not be R (records, tags,
`match`, Arrays, `DataFrame`, the standard library, nominal types) or when its subject is an
annotation itself. Pages with tabs are `.mdx`. Use one tab group per idea, not one per snippet.

Every block of `ry` output on a page is a test that `just docs-test` runs through the real binary;
[Docs examples](/development/testing.md#docs-examples) has the marker format and the rules for tabs.

Agents read the site as Markdown. Every page is also served at its URL plus `.md` (for example
`/tutorial/types.md`), with the frontmatter, imports and components removed and each tab
group turned into one subheading per tab. `/llms.txt` lists the pages with their descriptions in
sidebar order, and `/llms-full.txt` concatenates them all. Each page's head
links its Markdown version with `<link rel="alternate" type="text/markdown">`, and the breadcrumb row
has a "View as Markdown" link. The code is `docs/src/lib/markdown.ts`, the three endpoints in
`docs/src/pages/`, and `docs/src/routeData.ts`. A new MDX component must be handled in
`bodyToMarkdown`, or it leaks into the Markdown as raw tags. A `description` containing ` #` must be
quoted, because YAML reads the rest of the line as a comment.

Run `just docs` for a live preview, and `cd docs && bun run build` (or `pnpm build`) for a full
build, which needs Node 22.23.4 or newer. The live site changes only when someone deploys it with
`just docs-deploy` or the Docs workflow ([Releasing](/development/releasing.md#deploying-the-docs-site)).

[Formatting rules](/reference/formatting-rules.md) is generated, so do not edit it by hand. Edit
`crates/format/tests/formatter.template.md` and regenerate the page with `just format-docs`.

## The VS Code extension

1. Run `cd editors/code && bun install`.
2. Open the repository in VS Code and press Ctrl+Shift+B to compile the client in watch mode.
3. In the Run and Debug view (Ctrl+Shift+D), select `Launch Client` and press F5.
4. Open any `.R` file in the Extension Development Host window that appears.

## Why the formatter code is imperative

A comment in R can sit between any two tokens: inside an `if` header, between a call and its
argument list, after an operator. A declarative "format each field" style silently drops such a
comment, so the formatter walks the concrete children token by token and decides placement at the
token level. The parser attaches a trailing comment *inside* an expression node, so a decision about
where a closing token goes has to look at tokens, never at whole elements. See
`crates/format/src/format.rs` and the fixtures under `crates/format/tests/format/`.

## References

- R grammar and language: [gram.y](https://github.com/wch/r-source/blob/trunk/src/main/gram.y),
  [R Language Definition](https://cran.r-project.org/doc/manuals/r-release/R-lang.html)
- Formatting: [Wadler, A prettier printer](https://homepages.inf.ed.ac.uk/wadler/papers/prettier/prettier.pdf)
- VS Code extensions: [API reference](https://code.visualstudio.com/api/references/vscode-api),
  [language extensions](https://code.visualstudio.com/api/language-extensions),
  [lsp-sample](https://github.com/microsoft/vscode-extension-samples/tree/main/lsp-sample)
- Language servers in Rust: [gleam](https://github.com/gleam-lang/gleam/tree/main/compiler-core/src/language_server),
  [postgres-language-server](https://github.com/supabase-community/postgres-language-server),
  [nil](https://github.com/oxalica/nil) (async-lsp), [harper](https://github.com/Automattic/harper)
- Typing R and related languages: [enderpy](https://github.com/Glyphack/enderpy),
  [dgkf/R](https://github.com/dgkf/R), [typr](https://github.com/fabriceHategekimana/typr),
  [salsa's type-check example](https://github.com/salsa-rs/salsa/blob/master/examples/calc/type_check.rs)
- Out-of-order LSP messages: [tower-lsp#284](https://github.com/ebkalderon/tower-lsp/issues/284),
  [fe#1022](https://github.com/ethereum/fe/pull/1022),
  [tower-lsp-server#36](https://github.com/tower-lsp-community/tower-lsp-server/issues/36)