A while back I created the gvl-tracing gem as an experiment to help understand what was going on inside Ruby when multiple threads are in use.

But Ruby is not standing still! Ruby 3.3 saw the introduction of M:N scheduling (off by default) which is a feature I’ve been quite looking forward to. What is M:N scheduling, you ask? It’s a new execution strategy for Ruby threads that is similar/borrows from to the one used by Java’s virtual threads, Go’s coroutines or Erlang’s processes. (Note that when I say it’s similar to other VMs, I’m comparing the execution strategy part of the implementation, not the APIs or features provided).

With M:N scheduling Ruby threads are no longer mapped 1:1 to operating system threads; instead any M Ruby threads you choose to create are mapped to a pool of N operating system threads that Ruby automatically manages for you. Up until now, Ruby used a 1:1 strategy; now M will usually not be the same as N, and this is where the name comes from.

Why is this cool? For two big reasons: it makes Ruby threads into a simpler programming model, and it provides improved performance. Starting with performance, operating system threads are costly to create and manage, which is where the typical guidance for not creating too many, and to use techniques such as pooling comes from. With the introduction of ractors, this became an even bigger bottleneck for Ruby: without M:N scheduling each ractor would require at least one (new) thread, which put a clear limit on not using too many ractors.

By decoupling Ruby threads from operating system threads, code using threads or ractors becomes a lot more scalable! This is actually quite similar to how when you use fibers your application can do a lot more with less.

The second benefit from M:N scheduling is how it simplifies the programming model: Once creating threads and ractors becomes extremely cheap, you no longer need to pool them or reuse them or treat them as very special expensive snowflakes. Instead, they become a lot closer to fibers, where in many cases you can create almost as many as you need and not think too much about it.

This brings us back to talking about the GVL for two reasons: 1. If you’re not using ractors, only threads, then even with M:N scheduling you’ll still only have one Ruby thread running at a time, regardless of how that gets mapped to operating system threads; and 2. How to visualize the impact of M:N scheduling on your app?

At RubyConf 2024 I explored this topic and introduced a novel feature for the gvl-tracing gem: the ability to show both what the Ruby threads were doing (the M part) as well as how the Ruby VM was mapping them into operating system threads (the N part).

And, while I thought this new feature may be quite interesting by itself, I was quite surprised to uncover two different Ruby VM bugs related to the M:N scheduler (fixes are work in progress as of this writing: 1, 2).

You’ll find the video and slides for my RubyConf 2024 talk below.

Slide deck:

And that’s not all! An additional element was added to this story with Ruby 3.4: it introduces RUBY_THREAD_TIMESLICE, a setting to control how long each Ruby thread gets to run before being switched. I’m hoping this will lead Ruby’s default of 100ms to be progressively lowered, hopefully to something close to 10ms (which is what Python uses 😉).

Although, in the medium/long term, and connected to the M:N scheduling changes, what I believe will unlock the next wave of great performance in the presence of multiple threads, ractors and fibers is for Ruby to introduce a full scheduler, similar to the one used inside the operating system. Such a scheduler would allow Ruby to better balance IO-heavy work with CPU-heavy work, and make sure that all parts of the application get to run timely and with a fair distribution of resources. This is especially relevant as the more Ruby does internally, the less the operating system’s scheduler can help out with.

In the future, I hope to explore more around this topic of a Ruby scheduler.