Skip to content

Optional pure-Rust allocator with the mimalloc algorithm #16

Description

@freedomlovesfrank

The value of integrating

blvm-node sets mimalloc as its #[global_allocator] in src/lib.rs, with mimalloc = { version = "=0.1", default-features = false } pinned exactly, and carries tikv-jemallocator = "0.5" alongside it — so the allocator is clearly something you have already thought about rather than left to the platform.

rusty_alloc is worth a look precisely because it is not a different design. It is the mimalloc v2.4.5 algorithm rebuilt in safe Rust, implementing about 150 of mimalloc's roughly 157 mi_* entry points, with the C original kept in-tree only as a differential oracle. It has 9,261 downloads on crates.io and no build script.

That last part matters more here than usual, because your storage story is already a C-dependency story: heed3 is the default auto backend and needs liblmdb, tidesdb needs the TidesDB C library, and mimalloc is a third. On the hot paths those backends serve — initial block download, address and value-range indexing — rayon, crossbeam and dashmap mean the thread that frees a decoded block or UTXO entry is usually not the thread that allocated it. On exactly that path the rusty_alloc README measures 0.83x mimalloc's instructions (xthread — every free performed by a non-owning thread). It also aborts on a double free rather than accepting it.

Not asking you to change the default — this would sit behind an off-by-default feature.

How to integrate

Following the pattern you already use for the storage backends:

[features]
rusty-alloc = ["dep:rusty_alloc-api"]   # opt in; mimalloc stays the default

[dependencies]
rusty_alloc-api = { version = "2.1", optional = true }
#[cfg(feature = "rusty-alloc")]
#[global_allocator]
static GLOBAL: rusty_alloc_api::RustyAlloc = rusty_alloc_api::RustyAlloc;

Nothing else changes: same consensus rules, same RBF and mempool policy, same backend selection. An IBD run is the obvious place to compare, since that is where the allocation volume is concentrated.

Remade With Rust & MATA

We are rebuilding the C libraries Rust projects still lean on, in safe Rust — allocators, codecs, compression, parsers, identity and storage — so a build can drop its C pieces without losing performance. The org is at https://github.com/Remade-With-Rust if anything else fits. If this is not for you, please close the issue.

  • x.com/farmer_timmm

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions