Skip to content

Extralite support and some housekeeping - #1

Open
dbackeus wants to merge 6 commits into
fractaledmind:mainfrom
dbackeus:extralite
Open

Extralite support and some housekeeping#1
dbackeus wants to merge 6 commits into
fractaledmind:mainfrom
dbackeus:extralite

Conversation

@dbackeus

Copy link
Copy Markdown

The initial commits in this branch is house-keeping. You may want to just cherry pick some of those into your main branch to clean this PR up and let me know if there are any changes you'd prefer not to make.

I added a Benchmarking#long_running action to easily try out parallel execution in a threaded environment. I've been running rails server with a single worker / 5 threads for this purpose via:
WEB_CONCURRENCY=1 RUBY_YJIT_ENABLE=1 RAILS_LOG_LEVEL=debug SECRET_KEY_BASE=asdf RAILS_ENV=production DB_CLIENT=extralite rails s

Then benchmarking with concurrency of 5:
wrk -c 5 -t 1 -d 10s http://localhost:3000/benchmarking/long_running

With extralite I get ~4 rps, with sqlite3 <1rps, so the concurrency works as expected.

However, I've had a serious struggle with some weird initialization thing (bug?) where the very first requests do not appear to respect the busy_timeout = 5 configuration. This results in Extralite::BusyError occuring for just the initial queries, but never for subsequent ones. Super strange. This can be reproduced via RAILS_ENV=production SECRET_KEY_BASE=asdf DB_CLIENT=extralite rails c and running:

5.times do |i|
  Thread.new do
    @user = User.where("id >= ?", rand(User.minimum(:id)..User.maximum(:id))).limit(1).first
    @user.update!(last_seen_at: Time.now)
  end
end

The first time I run this I get busy errors, but subsequent executions do not. I have confirmed that the busy handler configuration method has run before I hit this issue so the busy_timeout is definitely set. I also cannot reproduce it when manually using using Extralite::Database:

user_ids = 1..100

10.times do |i|
  Thread.new do
    db = Extralite::Database.new("storage/production.sqlite3")
    db.busy_timeout = 5
    user = db.query("SELECT * FROM users WHERE id >= #{rand(user_ids)}").first
    db.execute <<~SQL
      UPDATE users
      SET last_seen_at = datetime('now')
      WHERE id = #{user.fetch(:id)}
    SQL
  end
end

So it appears something funky is going on inside the ActiveRecord code that causes this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant