Low Level Design: Multi-threading and Concurrency

May 24, 2026

What are Threads?

internal impementation of the Thread class

You must call start() to actually begin the thread. If you call run() directly, it will just execute the code in the current thread rather than starting a new one.

When you create a thread, Java asks the operating system to create a real OS thread.

Method 1: Extending the Thread Class

class WorkerThread extends Thread {
    @Override
    public void run() {
        System.out.println("Thread is running!");
    }
}

class Main {
    public static void main(String[] args) {
       Thread thread = new WorkerThread();
       thread.start(); 
    }
}

This approach is simpler but does not allow you to extend other classes.

Method 2: Implementing the Runnable Interface

class MyTask implements Runnable {
    @Override
    public void run() {
        System.out.println("Thread is running!");
    }
}

class Main {
    public static void main(String[] args) {
       MyTask task = new MyTask();
       Thread thread = new Thread(task);
       thread.start(); 
    }
}

Implementing Runnable separates the task being performed from the execution mechanism (the Thread object).

ExecutorService

Managing raw threads can be dangerous. If you create too many, you risk running out of memory or causing high CPU contention due to context switching.

ExecutorService addresses these issues by:

  • Thread Pooling: It maintains a pool of worker threads. When a task is completed, the thread is not destroyed, it is reused to perform the next task.

  • Task Queueing: If all threads are busy, new tasks are placed in an internal queue until a thread becomes available.

  • Lifecycle Management: It provides methods to gracefully shut down the service, cancel tasks, or wait for tasks to finish.

To create an ExecutorService, you primarily use the factory methods provided by the java.util.concurrent.Executors class. Each method is designed for a specific workload pattern.

Fixed Thread Pool: Executors.newFixedThreadPool(int n)

This is the most common choice. It creates a pool with a set number of threads. If all threads are busy and new tasks arrive, they are placed in an unbounded queue until a thread becomes available.

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

public class Main {
    public static void main(String[] args) {
        // 1. Create a pool with a fixed number of threads
        ExecutorService executor = Executors.newFixedThreadPool(3);

        // 2. Submit tasks (using a Runnable)
        executor.submit(() -> System.out.println("Task 1 running"));
        executor.submit(() -> System.out.println("Task 2 running"));
        executor.submit(() -> System.out.println("Task 3 running"));
        executor.submit(() -> System.out.println("Task 4 running"));

        // 3. Always shut down the executor when finished
        executor.shutdown();
    }
}

Best for: Tasks that are somewhat predictable in volume or when you need to strictly limit resource usage (e.g., database connection limits).

Cached Thread Pool: Executors.newCachedThreadPool()

This creates a thread pool that grows as needed. It will reuse previously constructed threads when they are available. If no thread is available, a new one is created. Threads that have not been used for 60 seconds are terminated and removed from the cache.

Best for: Applications with many short-lived, asynchronous tasks.

Warning: It can create a massive number of threads if tasks arrive faster than they can be processed, potentially leading to OutOfMemoryError.

Single Thread Executor: Executors.newSingleThreadExecutor()

This creates an ExecutorService that uses exactly one worker thread. If the thread dies due to a failure during execution, a new one will take its place. This guarantees that all tasks are executed sequentially in the order they were submitted.

Best for: Scenarios requiring strict ordering or tasks that must not run concurrently (like writing to a single file).

Scheduled Thread Pool: Executors.newScheduledThreadPool(int n)

This is used when you need to execute tasks after a delay or periodically.

Best for: Recurring background tasks, such as clearing a cache every 10 minutes or running a heartbeat check every 5 seconds.

Work-Stealing Pool (Java 8+): Executors.newWorkStealingPool()

This creates a thread pool that uses the ForkJoinPool architecture. Unlike the others, it uses multiple queues to reduce contention. If one thread finishes its queue, it can "steal" work from the queue of another busy thread.

Best for: Highly parallel, recursive, or "divide-and-conquer" tasks.

Limitations of the Runnable Interface

The Runnable interface is useful for executing tasks concurrently, but it has several limitations:

Cannot Return a Result

The run() method of Runnable interface has a void return type. So you cannot directly get the result of a computation.

public interface Runnable {
    void run();
}

Cannot Handle Exceptions

The run() method does not declare any checked exceptions. This means code like the following will not compile:

Runnable task = () -> {
    throw new IOException("File not found");
};

You must catch and handle checked exceptions inside the task itself. If a runtime exception occurs inside a Runnable, it is not automatically propagated to the caller.

Runnable task = () -> {
    throw new RuntimeException("Failure");
};

The thread may terminate, but the caller doesn't receive the exception.

Cannot Handle Cancellation

The run() method does not provide a way to cancel a task. This means you cannot cancel a task once it has started.

Introduction of the Callable and Future Interfaces

Most of the limitations of the Runnable interface were addressed by the Callable and Future interfaces introduced in Java 5 (as part of the java.util.concurrent package released in 2004).

The Callable Interface

A functional interface that can be used to define tasks that can be executed concurrently and return a result. It features a call() method that can return an object of any type V and throw checked exceptions.

public interface Callable<V> {
    V call() throws Exception;
}

Callable<String> task = () -> {
    return 100;
}

A Callable cannot be executed directly by a Thread because Thread accepts only a Runnable. Instead, a Callable must be submitted to an ExecutorService to be executed.

Callable<Integer> task = () -> {
    return 100;
}

ExecutorService executor = Executors.newSingleThreadExecutor();
Future<Integer> future = executor.submit(task);

The Future Interface

The Future interface represents the result of an asynchronous computation. It is returned when a task is submitted to an ExecutorService and provides methods such as:

  1. get(): to retrieve the result
  2. isDone(): to check completion status
  3. cancel(): to cancel execution
  4. isCancelled(): to verify cancellation
Callable<String> task = () -> {
    Thread.sleep(1000);
    return "hello world";
};

ExecutorService executor = Executors.newSingleThreadExecutor();
Future<String> future = executor.submit(task);

String result = future.get();
System.out.println(result);

executor.shutdown();

Limitations of the Future Interface

Future was a significant improvement over raw threads, but it had several limitations:

  1. Blocking Result Retrieval: future.get() blocks the calling thread until the task completes.

  2. No Callback Support: There is no way to register a callback that executes automatically when the task completes.

  3. No Task Chaining: You cannot express dependent asynchronous operations. For example, executing Task B after Task A completes requires manual coordination.

What is CompletableFuture?

The CompletableFuture class, introduced in Java 8, is an implementation of the Future interface that enables asynchronous, non-blocking, and composable programming. It addressed the limitations of Future interface by providing:

  1. Non-Blocking Callbacks: Execute actions automatically when a task completes, without blocking the calling thread.

  2. Task Chaining: Compose multiple asynchronous operations into a workflow where the output of one task becomes the input of another.

  3. Better Exception Handling: Handle and recover from exceptions declaratively within the asynchronous pipeline.

import java.util.concurrent.CompletableFuture;

public class Main {
    public static void main(String[] args) {
        
        // 1. Start an asynchronous task (e.g., fetching data from a database)
        CompletableFuture<String> future = CompletableFuture.supplyAsync(() -> {
            simulateDelay(2000); // Simulating 2 seconds of work
            return "User Data";
        })
        .exceptionally(ex -> {
            System.out.println("Exception: " + ex.getMessage());
            return "Exception Occurred.";
        });

        // 2. Define what happens when the task finishes (Non-blocking)
        future.thenAccept(data -> {
            System.out.println("Processing: " + data);
        });

        // 3. The main thread is free to do other things immediately
        System.out.println("Main thread is not blocked!");

        // 4. Keeping main alive long enough to see the output
        simulateDelay(3000);
    }

    private static void simulateDelay(int ms) {
        try { Thread.sleep(ms); } catch (InterruptedException e) { e.printStackTrace(); }
    }
}

What are Virtual Threads?

Volatile Keyword

The Problem: Visibility

In a multithreaded environment, each thread can cache variables in its local CPU cache for performance reasons. This means that if one thread updates the value of a variable, other threads might not immediately see that update, continuing to work with the stale value cached in their own local memory.

The Solution: volatile

When you declare a variable as volatile, you are telling the JVM and the compiler two things:

  1. Main Memory Access: The variable will never be cached in a thread's local CPU cache. Every read and write will happen directly from main memory.

  2. Instruction Reordering: The JVM is prevented from reordering instructions involving a volatile variable, which helps maintain predictable behavior in concurrent code.

Important Distinction: Visibility vs. Atomicity

It is a common misconception that volatile makes an operation "thread-safe." volatile only guarantees visibility. It ensures that threads see the most up-to-date value.

It does NOT guarantee atomicity. For example, an operation like count++ consists of three steps: (1) read the value, (2) increment it, and (3) write it back. Even with volatile, another thread could intervene between steps 1 and 3, leading to lost updates.

For operations that require atomicity (like counters or complex state updates), you should use synchronized blocks, locks, or the classes found in the java.util.concurrent.atomic package (such as AtomicInteger).

How to make Singleton class Thread-safe?

1. Synchronized Method

public class Singleton {

    private static Singleton instance;

    private Singleton() {}

    

}

What is an Immutable class?

In multi-threaded applications, bugs usually happen when two different threads try to modify the exact same piece of data at the same time, leading to "race conditions" or data corruption.

Because an immutable object cannot be changed after it is created, it can be shared across dozens of threads simultaneously without any synchronization, locks, or fear of corruption.

How to create an Immutable class?

Before Java 17:

  1. Make the class final so it cannot be inherited / overridden.
  2. Make all fields private and final.
  3. Initialize all fields via a constructor performing a deep copy.
  4. Provide only getter methods. Do not provide setters.
  5. Return copy for mutable fields.

// Rule 1: Make the class final so it cannot be inherited / overridden
class final ImmutableUser {
  
  // Rule 2: Make all fields private and final
  private final String name;
  private final int age;
  private final List<String> roles; // mutable object
  
  // Rule 3: Initialize all fields via a constructor making deep copy
  public ImmutableUser(String name, int age, List<String> roles) {
    this.name = name;
    this.age = age;
    // don't do this.roles = roles;
    // otherwise the caller can modify the list from outside
    this.roles = new ArrayList<>(roles);
  }
  
  // Rule 4: All provide getter methods
  public String getName() {
    return name;
  }
  
  public int getAge() {
    return age;
  }
  
  // Return a copy for mutable objects
  public List<String> getRoles() {
    // Prevents the caller from modifying the internal list
    return new ArrayList<>(roles);
  }
  
}

After Java 17:

  1. Introduced Record to create a type of class that is shallowly immutable by default.
  2. If your record takes a mutable collection like a List, use compact constructor to clone it so the outside world can't modify your record's internal state.

record ImmutableUser(String name, int age, List<String> roles) {
    public ImmutableUser {
        roles = List.copyOf(roles);
    }
}

NOTE: Think of Record as Java's version of a “Data Transfer Object” (DTO) or “Tuple”. They are perfect for scenarios where you need a simple carrier for immutable data without writing a mountain of boilerplate code.

Atomic Variables

Atomic variables are classes from the java.util.concurrent.atomic package that provide a way to perform thread-safe operations on single variables without needing explicit synchronized blocks or heavy locks.

How they work: Compare-And-Swap (CAS)

Unlike standard synchronization, which blocks other threads, atomic variables use a CPU-level operation called Compare-And-Swap (CAS).

  1. Read: The thread reads the current value.

  2. Calculate: It calculates the new value.

  3. Compare-and-Swap: It asks the CPU to update the variable only if the value hasn't changed since it was read. If the value did change (because another thread got there first), the operation fails, and the loop retries.

This makes them lock-free and generally much faster than using the synchronized keyword for simple variables.

Concurrent Collections

ConcurrentHashMap

CopyOnWriteArrayList