Database Connection Pools Explained: Timeouts, Saturation, and PgBouncer at Scale
How Connection Pools Actually Work, Why a Healthy Database Can Still Be Unreachable, and Where PgBouncer Fits at Scale
There is a failure that confuses engineers every time they meet it.
The database is healthy. It is up, it is fast, it has spare capacity, and its own metrics look fine. And yet the application cannot serve requests, because it cannot get to the database.
Nothing is wrong with either the application or the database, and yet the system is failing.
The culprit is almost always the layer between them, which is the connection pool.
Connection pools are one of the most important and least understood pieces of how an application talks to a database, and they are the source of a surprising share of production incidents.
When a pool saturates, the application grinds to a halt while the database sits idle, and the mismatch between where the problem appears and where it actually is makes it hard to diagnose.
Understanding connection pools is therefore valuable out of proportion to how much attention they usually get. Knowing why they exist, how they saturate, how timeouts govern their behavior, and how a tool like PgBouncer manages them at scale is the difference between a system that handles database access gracefully under load and one that mysteriously stalls.
This article explains connection pools from the ground up. It covers why they exist, how saturation and timeouts work, why the pool fails when the database is healthy, what PgBouncer adds at scale, and how to size and tune pools well. It is written for engineers who want to understand this common bottleneck rather than rediscover it during an outage.
Why Connection Pools Exist
To understand pools, start with the problem they solve, which is the cost of a database connection.
Opening a connection to a database is expensive. It involves establishing a network connection, authenticating, and setting up the session on the database side, all of which take time and consume resources. This cost is small for a single connection but becomes significant when an application handles many requests, because opening a fresh connection for every request would spend a large fraction of its time and resources just on connection setup.
Databases also limit how many connections they can handle at once. Each open connection consumes memory and resources on the database, and there is a maximum number the database can support. Opening a new connection per request would quickly exceed this limit under load, since a busy application would try to open far more connections than the database allows.
A connection pool solves both problems.
Instead of opening a new connection for each request, the application maintains a fixed set of open connections, the pool, and reuses them.
When a request needs the database, it borrows a connection from the pool, uses it, and returns it for the next request to use.
The connections stay open and are shared across many requests, so the expensive setup happens once rather than repeatedly, and the number of connections to the database is capped at the pool size.
This is why nearly every application uses connection pooling. It makes database access fast by reusing connections and safe by bounding how many connections exist.
The pool is the well-behaved intermediary that lets many requests share a limited, expensive resource. But that same bounding is what causes the failures examined next, because the pool is a finite resource that can run out.
How a Pool Saturates
The pool has a fixed number of connections, and this fixed size is both its virtue and its vulnerability. Understanding what happens when demand for connections exceeds the pool size is the heart of the topic.
Under normal conditions, requests borrow connections, use them briefly, and return them quickly, so the pool has connections available whenever a request needs one.
The key is that connections are held only briefly, so they cycle through many requests, and the pool serves far more requests than its size because each connection is reused rapidly.
Saturation happens when all the connections in the pool are in use at the same time and a new request arrives needing one.
With no connection available, the request cannot immediately proceed.
Instead, it waits for a connection to be returned to the pool.
As long as connections are returned quickly, this wait is brief and the system keeps functioning. But if requests keep arriving faster than connections are freed, more and more of them queue up waiting, and the pool is saturated.
The trigger for saturation is usually that connections are being held longer than normal.
Anything that makes a request hold its connection longer reduces how quickly connections cycle back to the pool, which means fewer are available and more requests must wait.
A slightly slow query, a delay in a downstream dependency that happens while a connection is held, a spike in concurrency, or a lock causing a query to wait can all lengthen how long connections are held.