Limiting concurrency

Definition

Virtual threads are cheap, but the things they call (a database, a payment API) are not. Instead of limiting the threads, limit the use of the scarce resource, for example with a Semaphore.

Example code

Many virtual threads, at most 3 using the database at once

import java.util.concurrent.Executors;
import java.util.concurrent.Semaphore;
import java.util.concurrent.atomic.AtomicInteger;

class Demo {
    public static void main(String[] args) {
        Semaphore databaseSlots = new Semaphore(3);          // the database allows 3 connections
        AtomicInteger inUse = new AtomicInteger();
        AtomicInteger peak = new AtomicInteger();

        try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
            for (int i = 0; i < 100; i++) {
                executor.submit(() -> {
                    databaseSlots.acquire();                  // waits cheaply if all 3 slots are taken
                    try {
                        peak.accumulateAndGet(inUse.incrementAndGet(), Math::max);
                        Thread.sleep(20);                     // the "query"
                        inUse.decrementAndGet();
                    } finally {
                        databaseSlots.release();
                    }
                    return null;
                });
            }
        }
        System.out.println("100 tasks ran, peak database use: " + peak.get());
    }
}

// Output:
// 100 tasks ran, peak database use: 3