Low Level Design: Multi-threading and Concurrency

December 24, 2025

What are Threads?

internal impementation of the Thread class

You must call start() to actually begin the thread. If you call run() directly, it will just execute the code in the current thread rather than starting a new one.

When you create a thread, Java asks the operating system to create a real OS thread.

Method 1: Extending the Thread Class

class WorkerThread extends Thread {
    @Override
    public void run() {
        System.out.println("Thread is running!");
    }
}

class Main {
    public static void main(String[] args) {
       Thread thread = new WorkerThread();
       thread.start(); 
    }
}

This approach is simpler but does not allow you to extend other classes.

Method 2: Implementing the Runnable Interface

class MyTask implements Runnable {
    @Override
    public void run() {
        System.out.println("Thread is running!");
    }
}

class Main {
    public static void main(String[] args) {
       MyTask task = new MyTask();
       Thread thread = new Thread(task);
       thread.start(); 
    }
}

Implementing Runnable separates the task being performed from the execution mechanism (the Thread object).

Method 3: Using an ExecutorService

Managing raw threads can be dangerous. If you create too many, you risk running out of memory or causing high CPU contention due to context switching.

ExecutorService addresses these issues by:

  • Thread Pooling: It maintains a pool of worker threads. When a task is completed, the thread is not destroyed, it is reused to perform the next task.

  • Task Queueing: If all threads are busy, new tasks are placed in an internal queue until a thread becomes available.

  • Lifecycle Management: It provides methods to gracefully shut down the service, cancel tasks, or wait for tasks to finish.

To create an ExecutorService, you primarily use the factory methods provided by the java.util.concurrent.Executors class. Each method is designed for a specific workload pattern.

Fixed Thread Pool: Executors.newFixedThreadPool(int n)

This is the most common choice. It creates a pool with a set number of threads. If all threads are busy and new tasks arrive, they are placed in an unbounded queue until a thread becomes available.

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

public class Main {
    public static void main(String[] args) {
        // 1. Create a pool with a fixed number of threads
        ExecutorService executor = Executors.newFixedThreadPool(3);

        // 2. Submit tasks (using a Runnable)
        executor.submit(() -> System.out.println("Task 1 running"));
        executor.submit(() -> System.out.println("Task 2 running"));
        executor.submit(() -> System.out.println("Task 3 running"));
        executor.submit(() -> System.out.println("Task 4 running"));

        // 3. Always shut down the executor when finished
        executor.shutdown();
    }
}

Best for: Tasks that are somewhat predictable in volume or when you need to strictly limit resource usage (e.g., database connection limits).

Cached Thread Pool: Executors.newCachedThreadPool()

This creates a thread pool that grows as needed. It will reuse previously constructed threads when they are available. If no thread is available, a new one is created. Threads that have not been used for 60 seconds are terminated and removed from the cache.

Best for: Applications with many short-lived, asynchronous tasks.

Warning: It can create a massive number of threads if tasks arrive faster than they can be processed, potentially leading to OutOfMemoryError.

Single Thread Executor: Executors.newSingleThreadExecutor()

This creates an ExecutorService that uses exactly one worker thread. If the thread dies due to a failure during execution, a new one will take its place. This guarantees that all tasks are executed sequentially in the order they were submitted.

Best for: Scenarios requiring strict ordering or tasks that must not run concurrently (like writing to a single file).

Scheduled Thread Pool: Executors.newScheduledThreadPool(int n)

This is used when you need to execute tasks after a delay or periodically.

Best for: Recurring background tasks, such as clearing a cache every 10 minutes or running a heartbeat check every 5 seconds.

Work-Stealing Pool (Java 8+): Executors.newWorkStealingPool()

This creates a thread pool that uses the ForkJoinPool architecture. Unlike the others, it uses multiple queues to reduce contention. If one thread finishes its queue, it can "steal" work from the queue of another busy thread.

Best for: Highly parallel, recursive, or "divide-and-conquer" tasks.

How does Executor Service work?

ExecutorService is a Java framework component that manages and executes threads for you.

Inside each worker thread is a run() loop that looks roughly like this:

while (true) {
    // 1. Wait for a task to be placed in the queue
    Runnable task = taskQueue.take(); 
    
    // 2. Execute the task
    task.run(); 
    
    // 3. DO NOT terminate. Loop back to step 1.
}

What are Virtual Threads?

Volatile Keyword

The Problem: Visibility

In a multithreaded environment, each thread can cache variables in its local CPU cache for performance reasons. This means that if one thread updates the value of a variable, other threads might not immediately see that update, continuing to work with the stale value cached in their own local memory.

The Solution: volatile

When you declare a variable as volatile, you are telling the JVM and the compiler two things:

  1. Main Memory Access: The variable will never be cached in a thread's local CPU cache. Every read and write will happen directly from main memory.

  2. Instruction Reordering: The JVM is prevented from reordering instructions involving a volatile variable, which helps maintain predictable behavior in concurrent code.

Important Distinction: Visibility vs. Atomicity

It is a common misconception that volatile makes an operation "thread-safe." volatile only guarantees visibility. It ensures that threads see the most up-to-date value.

It does NOT guarantee atomicity. For example, an operation like count++ consists of three steps: (1) read the value, (2) increment it, and (3) write it back. Even with volatile, another thread could intervene between steps 1 and 3, leading to lost updates.

For operations that require atomicity (like counters or complex state updates), you should use synchronized blocks, locks, or the classes found in the java.util.concurrent.atomic package (such as AtomicInteger).

Atomic Variables

Atomic variables are classes from the java.util.concurrent.atomic package that provide a way to perform thread-safe operations on single variables without needing explicit synchronized blocks or heavy locks.

How they work: Compare-And-Swap (CAS)

Unlike standard synchronization, which blocks other threads, atomic variables use a CPU-level operation called Compare-And-Swap (CAS).

  1. Read: The thread reads the current value.

  2. Calculate: It calculates the new value.

  3. Compare-and-Swap: It asks the CPU to update the variable only if the value hasn't changed since it was read. If the value did change (because another thread got there first), the operation fails, and the loop retries.

This makes them lock-free and generally much faster than using the synchronized keyword for simple variables.

Concurrent Collections

ConcurrentHashMap

CopyOnWriteArrayList