System Design (Bitly): URL Shortener

June 12, 2026

A URL shortening service allows us to create a short, easy-to-remember URL for a long URL.

url-shortener

Functional Requirements

  1. Users should able to submit a long URL along with additional details like custom alias, expiration date, etc., and get a shortened version.

  2. When a user visits the shortened URL, they are redirected to the original URL.

Non-Functional Requirements

  1. The system should scale to support 1 Billion shortened URLs and 100 Million visits per day.

  2. Each shortened URL should be unique.

  3. The redirection should occur with minimal latency (< 100ms).

Design Rationale

Q1. How to generate a unique shortened URL?

We need some entropy (randomness) to try to ensure that our codes are unique. Considering we are dealing with ?? write request per day, a random number generator does not provide enough entropy to ensure that our codes are unique.

Hash Function

We could use a hash function like SHA-256 to generate a fixed-size hash code. Hash functions take an input and return a deterministic, fixed-size string of characters. Hash functions also provide a high degree of entropy, meaning that the output appears random and is unlikely to collide for different inputs.

Note

Pure hash functions are deterministic: the same long URL always maps to the same short code without needing to query the database. This may be desirable (deduplication) or not (if you need multiple codes per URL or want to prevent guessability/adversarial preimages). For the latter cases, add a secret salt or nonce (HMAC).

We can then take the output and encode it using a base62 encoding scheme and take just the first N characters as our short code. N is determined based on the number of characters needed to minimize collisions (e.g., 8 characters gives 62^8 ā‰ˆ 218 trillion possible codes).

Note

Why base62?

It's a compact representation of numbers that uses 62 characters (a-z, A-Z, 0-9). The reason it's 62 and not the more common base64 is because we exclude + and /. The slash is a path separator in URLs and the plus sign can be interpreted as a space in query strings.

To reduce collision probability, we need higher entropy, which means generating longer short codes. However, longer codes negate the benefit of having a short URL. Detecting and resolving collisions also adds database lookups on insertion, introducing latency and complexity.

Unique Counter with Base 62 Encoding

One way to guarantee we don't have collisions is to simply increment a counter for each new url. We can then take the output of the counter and encode it using base62 encoding to ensure it's a compacted representation.

Note

Redis is particularly well-suited for managing this counter because it's single-threaded and supports atomic operations. Being single-threaded means Redis processes one command at a time, eliminating race conditions.

Its INCR command atomically increments the counter and returns the new value in a single operation. Because Redis is single-threaded, two simultaneous calls will always receive different values. If one gets 1000, the other gets 1001. This guarantee is what makes Redis ideal for distributed counter management.

Sequential counters produce predictable short codes, making URL enumeration possible. An attacker could iterate through codes to discover all URLs. If this is a concern, apply a reversible transformation (like XOR with a secret key) before base62 encoding, or accept the tradeoff since short URLs are often meant to be shared publicly anyway.

We also have to consider that the size of the short code continues to increase over time with this method. To determine whether we should be concerned about length, we can do a little math. If we have 1B urls, when base62 encoded, this would result in a 6-character string.

Q2. What is the expected behavior when multiple users request the same long URL?

The system should allow multiple short URLs to be generated for the same long URL since most users may want separate expiration dates or custom aliases.

Q3. How to deal with expired short URLs?

Do storage capacity estimation and suggest the answer to this.

Q4. How to support a sub-100ms latency?

When dealing with a large database of shortened URLs, finding the right match quickly becomes crucial for a smooth user experience. Without any optimization, our system would need to check every single pair of short and original URLs in the database to find the one we're looking for.

This process, known as a "full table scan," can be incredibly slow, especially as the number of URLs grows into the millions or billions.

Indexing

To avoid a full table scan, we can use a technique called indexing. An index creates a separate, sorted list of our short URLs, each with a pointer to where the full information is stored in the main table. This allows the database to use efficient search methods, dramatically reducing the time it takes to find a matching URL.

Relying solely on a disk-based database for redirects presents some challenges, although modern SSDs have significantly reduced the performance gap. While disk I/O is slower than memory access, it's not prohibitively slow. A typical SSD can handle around 100,000 IOPS (Input/Output Operations Per Second), which is quite fast for many applications.

However, the main challenge lies in the sheer volume of read operations required. With 100M DAU (Daily Active Users), assuming each user performs an average of 5 redirects per day, we're looking at:

100,000,000 users * 5 redirects = 500,000,000 redirects per day
500,000,000 / 86,400 seconds ā‰ˆ 5,787 redirects per second

This assumes redirects are evenly distributed throughout the day, which is unlikely. Most redirects will occur during peak hours, which means we need to design for high-traffic spikes. Multiplying by 100x to handle the spikes means we need to handle ~600k read operations per second.

Even with optimized queries and indexing, a single database instance may struggle to keep up with this volume of traffic. This high read load could lead to increased response times, potential timeouts, and might affect other database operations like URL shortening.

Caching

To improve redirect speed, we can introduce an in-memory cache like Redis or Memcached between the application server and the database.

The cache stores the frequently accessed mappings of short codes to long URLs. When a redirect request comes in, the server first checks the cache. If the short code is found in the cache (a cache hit), the server retrieves the long URL from the cache, significantly reducing latency. If not found (a cache miss), the server queries the database, retrieves the long URL, and then stores it in the cache for future requests.

Note

The key here is that instead of going to disk we access the mapping directly from memory. This difference in access speed is significant:

  • Memory access time: ~100 nanoseconds (0.0001 ms)
  • SSD access time: ~0.1 milliseconds
  • HDD access time: ~10 milliseconds

This means memory access is about 1,000 times faster than SSD and 100,000 times faster than HDD. In terms of operations per second:

  • Memory: Can support millions of reads per second
  • SSD: ~100,000 IOPS (Input/Output Operations Per Second)
  • HDD: ~100-200 IOPS

While implementing an in-memory cache offers significant performance improvements, it does come with its own set of challenges. Cache invalidation can be complex, especially when updates or deletions occur, though this issue is minimized since URLs are mostly read-heavy and rarely change.

The cache needs time to "warm up," meaning initial requests may still hit the database until the cache is populated. Memory limitations require careful decisions about cache size, eviction policies (e.g., LRU - Least Recently Used), and which entries to store.

Content Delivery Network (CDN)

The short URL domain can be served through a CDN with Points of Presence (PoPs) geographically distributed around the world. The CDN nodes cache the mappings of short codes to long URLs, allowing redirect requests to be handled close to the user's location.

Furthermore, by deploying the redirect logic to the edge using platforms like Cloudflare Workers or AWS Lambda@Edge, the redirection can happen directly at the CDN level without reaching the origin server.

The benefit here is that, at least for popular short codes, the redirection can happen at the CDN (close to the user) and it never even reaches our Primary Server, meaningfully reducing the latency.

However, this too presents some challenges. Ensuring cache invalidation and consistency across all CDN nodes can be complex. Setting up edge computing requires additional configuration and understanding of serverless functions at the edge. Cost considerations come into play, as CDNs and edge computing services may incur higher costs, especially with high traffic volumes. Edge functions may have limitations in execution time, memory, and available libraries, requiring careful optimization of the redirect logic. Lastly, debugging and monitoring in a distributed edge environment can be more challenging compared to centralized servers.

Note

You're trading cost and complexity for performance here. Whether or not this is worth it depends on factors like company price sensitivity, user experience requirements, and traffic patterns.

Sequence Flow

Use Case 1: Create Short URL

  1. User submits a long URL along with additional details like custom alias, expiration date, etc.

  2. The Client (browser or mobile app) sends a POST /shorten request (along with the long URL and additional details) to the Shortening Service.

  3. The Shortening Service validates the request (what validations?) and creates a unique shortened URL.

  4. Once the shortened URL is created, the Shortening Service inserts a record into the database (what should be the name of the table?) to store the mapping between the long URL and the shortened URL.

  5. Finally, the Shortening Service returns the shortened URL to the Client with HTTP status code 201 Created.

Use Case 2: Redirect to Original URL

  1. The Client (browser or mobile app) sends a GET /<shortened-url> request to the Redirection Service.

  2. The Redirection Service looks up the shortened URL in the database.

  3. If the shortened URL is found and is not expired, the Redirection Service sends an HTTP redirect response to the Client with the original URL in the Location header (Why in the Location header?, What HTTP status code to return?).

API Design

Shortening Service

API 1: POST /shorten
Sample Request
POST /shorten
Content-Type: application/json

{
  "longUrl": "https://www.google.com",
  "customAlias": "google",
  "expirationDate": "2026-06-12"
}

Algorithm:

input_url = "https://www.example.com/some/very/long/url"
# Canonicalize URL first (lowercase host, strip default ports, normalize trailing slash, etc.)
canonical_url = canonicalize(input_url)
hash_code = hash_function(canonical_url)
short_code_encoded = base62_encode(hash_code)
short_code = short_code_encoded[:8] # 8 characters

Response:

HTTP/1.1 201 Created
Location: https://bit.ly/google

Redirection Service

Note

Reads are much more frequent than writes, we can scale our Primary Server by separating the read and write operations. This introduces a microservice architecture where the Read Service handles redirects while the Write service handles the creation of new short urls.

This separation allows us to scale each service independently based on their specific demands.

Data-Model Design

Each row in our database consists of a short code (~8 bytes), long URL (~100 bytes), creationTime (~8 bytes), optional custom alias (~100 bytes), and expiration date (~8 bytes). This totals to ~200 bytes per row. We can round up to 500 bytes to account for any additional metadata like the creator id, analytics id, etc.

If we store 1B mappings, we're looking at 500 bytes * 1B rows = 500GB of data. The reality is, this is well within the capabilities of modern SSDs. Given the number of urls on the internet is our maximum bound, we can expect it to grow but only modestly.

If we were to hit a hardware limit, we could always shard our data across multiple servers but a single Postgres instance, for example, should do for now.

Final Design

Follow-up Questions

The shortened URL exists at a domain that we own.

We can use ...

The basic structure of a shortened URL is: ?.

If the user provides custom alias, the system should check if it already taken or not. How domain works for custom alias?.