`RSTRING_LEN` on an unvalidated VALUE gates a fixed 64-byte read — SIGSEGV on an immediate, heap over-read on a type-confused object

[jeremy]

BCryptPbkdf::Engine.__bc_crypt_hash reads RSTRING_LEN off an unvalidated VALUE — wild dereference, and a 64-byte over-read

Repository: net-ssh/bcrypt_pbkdf-ruby
Affected: 1.1.2 (latest release) and master
Severity: latent — see Reachability below. Not filed privately because the trigger is the argument's type, and I could not find a path where an attacker chooses that.

1. Summary

bc_crypt_hash takes the length of both of its arguments before converting them to
Strings. RSTRING_LEN() compiles to an unchecked 8-byte load at VALUE + 16, so:

• Passing any immediate (nil, false, true, an Integer, a Float, a Symbol)
  dereferences a small non-pointer address and segfaults the interpreter. Measured:
  7/7 subjects, SIGSEGV, every time.
• Passing a heap object whose word at +16 happens to be 64 satisfies the length
  gate. StringValuePtr then calls to_str, and bcrypt_hash reads 64 bytes from
  whatever that returns — however short it actually is. This is a plain
  heap-buffer-overflow read; ASan reports it at bcrypt_pbkdf_ext.c:32.

This is latent: it is a type-confusion, not an input-length bug, and I could not find a
caller in the wild that passes a non-String. Filing it because the two lines are cheap
to fix and the failure mode (interpreter crash / OOB read into a crypto primitive) is
severe if a caller ever does.

2. Reproduction

Both parts, no dependencies beyond the gem.

2a. Wild dereference from an immediate

require "bcrypt_pbkdf_ext"
good = +("s" * 64)
BCryptPbkdf::Engine.__bc_crypt_hash(false, good)   # => SIGSEGV

Every immediate crashes, because RSTRING_LEN loads from VALUE + 16 and an immediate's
VALUE is a small tagged integer, so the address is in the first page:

argument	result
false, true, nil, 1, 4611686018427387903, 1.0, :sym	SIGSEGV, 7/7

With the fix below, all seven raise TypeError: no implicit conversion of … into String.

2b. 64-byte over-read past a short String

The non-obvious steps, each of which a simplification would remove:

• Why an Array grown then shrunk. RSTRING_LEN reads offset 16, which for
  struct RString is len. For a heap RArray the same offset is as.heap.len.
  Array.new(64) is embedded, so its +16 word is as.ary[0], not 64 — you must grow
  the array past the embedded capacity and shrink it back so it keeps a heap body.
• Why a shared substring. So the over-read is visible to ASan. A 16-byte substring
  taken from the tail of a 5000-byte parent shares the parent's buffer (Ruby shares
  when beg + len == parent.length and len >= 16), so RSTRING_PTR points 4984 bytes
  into a 5001-byte malloc block and reading 64 bytes runs 47 bytes past its end. A short
  literal would over-read into adjacent GC slots, which no sanitiser can see.

require "bcrypt_pbkdf_ext"

class GateArray < Array
  def initialize(payload)
    super(4096) { 0 }
    slice!(64..-1)          # heap body retained; as.heap.len (offset +16) == 64
    @payload = payload
  end
  def to_str = @payload     # runs only AFTER the length gate has passed
end

parent = +("P" * 5000)
sub    = parent[4984, 16]   # shared: RSTRING_PTR == parent_ptr + 4984
BCryptPbkdf::Engine.__bc_crypt_hash(GateArray.new(sub), +("s" * 64))

Built with -fsanitize=address, 3/3:

==1==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x522000014c89 ...
READ of size 1 at 0x522000014c89 thread T0
    #0 Blowfish_stream2word blowfish.c:412
    #1 Blowfish_expandstate blowfish.c:473
    #2 bcrypt_hash          bcrypt_pbkdf.c:69
    #3 bc_crypt_hash        bcrypt_pbkdf_ext.c:32

Control, same file, same run: with the one-line fix applied, the ASan build is clean
3/3 and the call returns nil (the length gate now sees the converted String's real
length, 16 ≠ 64).

Detector-free confirmation that the out-of-bounds bytes are actually consumed: hold
the pass String's 16 bytes identical across two calls and change only the allocation that
follows it. The digests differ, so 48 of the 64 bytes hashed were not the String's:

fill=A: pass bytes="PPPPPPPPPPPPPPPP" digest=216a5827071159a870b24ba344a4ffa7...
fill=B: pass bytes="PPPPPPPPPPPPPPPP" digest=4de073ab7567f69c4b2f630d3d32f01c...

Rates: deterministic, 3/3 for every case. Not probabilistic, so there is no natural-rate
figure to report.

3. Cause

ext/mri/bcrypt_pbkdf_ext.c:26-33 (1.1.2; unchanged on master):

static VALUE bc_crypt_hash(VALUE self, VALUE pass, VALUE salt) {
  u_int8_t hash[BCRYPT_HASHSIZE];
  if (RSTRING_LEN(pass) != 64U)      /* line 28: reads pass+16 with no type check */
    return Qnil;
  if (RSTRING_LEN(salt) != 64U)      /* line 30: same for salt */
    return Qnil;
  bcrypt_hash((const u_int8_t*)StringValuePtr(pass),   /* line 32: conversion happens
              (const u_int8_t*)StringValuePtr(salt),      here, after both gates, and
              hash);                                       nothing rechecks the length */
  return rb_str_new((const char*)hash, sizeof(hash));
}

RSTRING_LEN in ruby/internal/core/rstring.h is return RSTRING(str)->len;, and
RSTRING(obj) is a bare cast. Nothing validates the type. This is what the shipped
-O3 build actually does (gcc 14.2, aarch64):

1390:  ldr  x1, [x1, #16]      ; RSTRING_LEN(pass) — raw load, no type check
1394:  cmp  x1, #0x40
1398:  b.ne 13a8               ; return Qnil
139c:  ldr  x1, [x2, #16]      ; RSTRING_LEN(salt)
13a4:  b.eq 13b0
13b8:  bl   rb_string_value_ptr@plt   ; the type check happens only here
13c8:  bl   rb_string_value_ptr@plt
13d8:  bl   bcrypt_hash@plt

A related, currently-benign defect at lines 15-16

  int ret = bcrypt_pbkdf(
    StringValuePtr(pass), RSTRING_LEN(pass),
    (const u_int8_t*)StringValuePtr(salt), RSTRING_LEN(salt),

StringValuePtr(v) expands to rb_string_value_ptr(&(v)), which assigns to v.
RSTRING_LEN(pass) in the same argument list reads pass unsequenced with respect to
that assignment — undefined behaviour under C11 6.5p2. I could not make it misbehave: on
gcc 14.2 and clang 19.1.7, at -O0/-O1/-O2/-O3/-Os (10/10 configurations), every build
reloads pass from its stack slot after the call, so the length comes from the
converted String and the result is correct. Worth fixing anyway, since nothing in the
source guarantees that ordering. (Note neither compiler warns; -Wsequence-point is
silent on it.)

4. Affected versions

version	bc_crypt_hash	lines 15-16	notes
1.1.2 (latest release)	affected	UB, benign as built	tested
master @ today	affected — code identical	UB, benign as built	read from GitHub
1.2.0.beta1	affected — master adds an okeylen bound check to bc_crypt_pbkdf only	UB	read only

Not bisected: the defect is in the original shape of the function, not something that
regressed.

5. Suggested fix

Convert before measuring — the same StringValue-then-use order the rest of the
extension relies on:

 static VALUE bc_crypt_hash(VALUE self, VALUE pass, VALUE salt) {
   u_int8_t hash[BCRYPT_HASHSIZE];
+  StringValue(pass); StringValue(salt);
   if (RSTRING_LEN(pass) != 64U)
     return Qnil;

and, for lines 15-16, hoisting the same two calls above NUM2ULONG(keylen):

 static VALUE bc_crypt_pbkdf(VALUE self, VALUE pass, VALUE salt, VALUE keylen, VALUE rounds) {
-  size_t okeylen = NUM2ULONG(keylen);
+  size_t okeylen;
+  StringValue(pass); StringValue(salt);
+  okeylen = NUM2ULONG(keylen);

I ran this. With it applied: the seven immediates raise TypeError instead of
segfaulting (7/7), the ASan build is clean 3/3, and BCryptPbkdf.key continues to
produce byte-identical output for String arguments.

6. Reachability

I searched the only production bundle that ships this gem — 164 gems plus the
application — for every caller:

rg -n 'BCryptPbkdf|bcrypt_pbkdf|__bc_crypt' "$(ruby -e 'puts Gem.dir')/gems" app lib config

Results:

• __bc_crypt_hash has no caller at all outside the gem's own test/. The
  over-read is unreachable from any released library I can see.
• The gem's only entry point in use is BCryptPbkdf.key → __bc_crypt_pbkdf, reached
  from exactly one place: net-ssh/lib/net/ssh/authentication/ed25519.rb:77,
  BCryptPbkdf::key(password, salt, keylen + ivlen, rounds).
• Both String arguments there are Strings by construction: salt = kdfopts.read_string,
  and password comes from key_factory.rb:58, key_type.read(data, passphrase || 'invalid')
  — a nil passphrase is coerced to the String 'invalid', and the interactive branch
  returns a String from the prompter.

So the trigger is argument type, not attacker-supplied bytes, and no shipped caller
supplies the wrong type. Hence a public issue rather than a private report.

7. Environment

ruby 4.0.6 (2026-07-14 revision 03b6d3f889) +PRISM [aarch64-linux]
bcrypt_pbkdf 1.1.2, built from source in-tree (gem install --platform=ruby)
gcc (Debian 14.2.0-19) 14.2.0 / clang 19.1.7, aarch64
ASan: gcc's libasan, LD_PRELOAD'ed into a stock (non-instrumented) ruby,
      ASAN_OPTIONS=detect_leaks=0:halt_on_error=1:verify_asan_link_order=0

Everything above was run in a network-isolated container. RSTRING_LEN reading offset 16
depends on struct RString having len as a common field, which it does on every Ruby
from 3.4 onward; on older Rubies the offset differs but the missing type check does not.