PostgreSQL Architecture: Processes, Memory and WAL

PostgreSQL architecture is a multi-process design. One supervisor process, the postmaster, listens for connections and forks a dedicated operating-system process for every client session. All of those processes share one big memory region for cached data pages and write-ahead log records. A set of background processes handles flushing, checkpoints, cleanup and archiving. Each backend also has private memory for sorts and hashes.

Knowing which process does what explains a lot of everyday behavior: why connections are expensive, why work_mem can exhaust RAM, why a crash in one session disconnects everyone, and why a commit is durable before the data file has been touched. This guide walks through the pieces in the order a query meets them, using PostgreSQL 16.

The big picture of PostgreSQL architecture

Here is the whole system in one view:

A request flows top to bottom: the postmaster forks a backend, the backend authenticates the client and runs its SQL against shared memory, and background processes move changes to disk later.

The postmaster and fork per connection

When you start PostgreSQL, the first process is the postmaster (in ps output it appears as the plain postgres binary with its -D data directory). It does very little query work itself. Its jobs are to:

• listen on the configured port (5432 by default) and Unix socket,
• fork() a new child process for each incoming connection,
• start and supervise the background processes,
• react when a child process dies.

That last point matters in production. If any backend crashes abnormally, for example with a segfault in an extension, the postmaster can't trust shared memory anymore. It terminates every other session, reinitializes shared memory, and runs crash recovery from the WAL, so one bad session can disconnect all users for a few seconds. The logger is the exception: it keeps running through the reset, so the messages explaining the crash aren't lost.

You can see the process tree on Linux:

ps -ef --forest | grep [p]ostgres

Trimmed output from an Ubuntu server:

postgres   812     1  /usr/lib/postgresql/16/bin/postgres -D /var/lib/postgresql/16/main -c config_file=/etc/postgresql/16/main/postgresql.conf
postgres   813   812   \_ postgres: 16/main: checkpointer
postgres   814   812   \_ postgres: 16/main: background writer
postgres   816   812   \_ postgres: 16/main: walwriter
postgres   817   812   \_ postgres: 16/main: autovacuum launcher
postgres   818   812   \_ postgres: 16/main: logical replication launcher
postgres  2231   812   \_ postgres: 16/main: app appdb 10.0.0.12(53418) idle

Every child has the postmaster (PID 812) as its parent. The last line is a client backend: user app, database appdb, the client address, and its state. Inside the database, pg_stat_activity lists the same processes with a backend_type column.

Why connections are expensive

One process per connection gives strong isolation, but each process costs memory and startup time. Thousands of mostly idle connections waste RAM and add scheduling overhead. max_connections defaults to 100 for this reason. Applications with many instances normally put a connection pooler such as PgBouncer in front of the database, or use a pool inside the application, rather than raising max_connections into the thousands.

Authentication with pg_hba.conf

A common misconception is that the postmaster authenticates the client. In fact, the postmaster forks the child first, and the new backend process performs authentication. That keeps the supervisor simple and means a slow or hostile client can't block the listener.

The rules live in pg_hba.conf (host-based authentication). PostgreSQL reads them top to bottom and uses the first line that matches the connection type, database, user and address:

# TYPE  DATABASE  USER      ADDRESS         METHOD
local   all       postgres                  peer
host    appdb     app       10.0.0.0/24     scram-sha-256
host    all       all       0.0.0.0/0       reject

The common methods are:

• peer: for local socket connections, trust the operating system user name.
• scram-sha-256: password authentication without sending the password in clear text. Use this for network connections.
• md5: the older password method. It is deprecated in newer PostgreSQL releases, so migrate to SCRAM.
• trust: no check at all. Use it only on a throwaway development machine.

After editing the file, reload the configuration with SELECT pg_reload_conf();. You don't need a restart.

Backend processes: where queries actually run

Once authenticated, the backend serves that one client for the life of the connection. For every statement it:

1. parses the SQL text,
2. rewrites it (views, rules),
3. plans it, choosing scans, join order and join methods,
4. executes the plan, reading and modifying pages in shared buffers,
5. sends result rows back to the client.

For a SELECT, the backend looks for the needed pages in shared buffers and reads them from disk only on a miss. For an UPDATE, it modifies the page in shared buffers, marks it dirty, and writes a WAL record describing the change. It does not write the data file at that moment.

Local memory: work_mem, temp_buffers and maintenance_work_mem

Each backend has private memory that no other process can see.

work_mem (default 4MB) is the budget for a single sort or hash operation: ORDER BY, DISTINCT, hash joins, hash aggregates, and merge joins that need sorted input. When the data doesn't fit, PostgreSQL spills to temporary files on disk, which you can see in a plan:

SET work_mem = '4MB';
EXPLAIN (ANALYZE, BUFFERS)
SELECT customer_id, sum(amount)
FROM orders
GROUP BY customer_id
ORDER BY sum(amount) DESC;
-- look for: Sort Method: external merge  Disk: 51200kB

The trap is that work_mem is per operation, per backend, not per server. A complex query with four sort and hash nodes can use four times the value, and hash operations may use up to hash_mem_multiplier (default 2.0) times more. Multiply that by the number of active connections before raising it globally. A safer pattern is to raise it only for the session or role that runs heavy reports:

ALTER ROLE reporting SET work_mem = '256MB';

For more on reading plans like the one above, see how to read query plans and fix slow SQL.

temp_buffers (default 8MB) caches pages of temporary tables created with CREATE TEMP TABLE. Temp tables belong to one session, so their pages stay out of shared buffers.

maintenance_work_mem (default 64MB) is used by VACUUM, CREATE INDEX, REINDEX and ALTER TABLE ADD FOREIGN KEY. Raising it for the session running a large index build often speeds it up considerably.

Shared memory: shared buffers and WAL buffers

Shared memory is allocated once at startup and used by every process.

Shared buffers

shared_buffers is PostgreSQL's page cache. It holds 8KB pages of tables and indexes. The default of 128MB is deliberately small; on a dedicated database server a common starting point is around 25% of RAM. PostgreSQL also relies on the operating system's file cache, so it doesn't pay to give it most of the memory.

When a backend modifies a page, the page becomes dirty: the version in memory is newer than the version on disk. Dirty pages are written back later by the background writer, the checkpointer, or, if neither has kept up, the backend itself when it needs a free buffer. How pages and rows are laid out inside those 8KB blocks is covered in PostgreSQL storage internals.

WAL buffers

Every change also produces a write-ahead log (WAL) record. Records go into wal_buffers first; the default of -1 sizes it automatically at 1/32 of shared buffers, capped at 16MB. The core rule is simple: a change's WAL record must reach disk before the changed data page does, and a transaction counts as committed once its WAL is flushed. After a crash, PostgreSQL replays the WAL to redo any changes that never made it into data files.

In short, shared buffers hold the data, and WAL buffers hold the record of how the data changed.

Check the current values with SHOW shared_buffers; and SHOW wal_buffers;.

Background processes and what each one does

A few details worth knowing:

• Checkpointer. A checkpoint is a save point. After it, crash recovery only needs to replay WAL from that point, not from the beginning. Checkpoints that happen too often waste I/O, partly because the first change to each page after a checkpoint writes a full page image into WAL. Checkpoints that are too far apart make recovery take longer. If the logs show checkpoints "occurring too frequently", raise max_wal_size.
• WAL writer. It reduces the flushing work backends must do, but with the default synchronous_commit = on a committing backend still waits until its own WAL records are flushed. The flush is the price of durability.
• Autovacuum. It exists because of MVCC: an UPDATE or DELETE leaves the old row version behind, and something has to clean it up. ACID and MVCC explained covers why those versions exist.
• Archiver and logger only run when enabled: archive_mode = on and logging_collector = on respectively. On Debian and Ubuntu packages, logging usually goes to /var/log/postgresql without the collector, which is why the ps output above has no logger.
• No stats collector. Older diagrams show a "stats collector" process. Since PostgreSQL 15, cumulative statistics live in shared memory, and that process no longer exists.

PostgreSQL 16 adds pg_stat_io, which breaks down I/O by backend type. If client backends show many writes, they are flushing dirty pages themselves, and the background writer or checkpointer needs tuning:

SELECT backend_type, object, context, reads, writes, fsyncs
FROM pg_stat_io
WHERE writes > 0
ORDER BY writes DESC;

Files on disk

• Data files: table and index pages under base/, split into 1GB segments.
• WAL files: 16MB segments in pg_wal/, used for crash recovery and replication. Never delete them by hand.
• Archive files: completed WAL segments copied elsewhere for point-in-time recovery.
• Log files: errors, connections, slow statements, checkpoint and autovacuum messages.

Don't confuse the two kinds of "log": the logger writes diagnostics for humans, while WAL is the binary change log PostgreSQL needs to recover. Backups, replicas and monitoring are covered in running databases in production.

FAQ

Is PostgreSQL multi-threaded or multi-process?

Multi-process. Each connection gets its own backend process, forked from the postmaster. Parallel query also uses extra worker processes, not threads.

How much memory should I give shared_buffers?

On a dedicated server, around 25% of RAM is a common starting point. Measure the cache hit ratio and I/O afterwards instead of going much higher by default, since the OS file cache also holds PostgreSQL data.

Why does raising work_mem cause out-of-memory errors?

Because it applies per sort or hash node in every backend. Many connections running queries with several such nodes can together allocate far more than the value suggests. Raise it per role or per session for heavy queries.

What is the difference between the WAL writer and the checkpointer?

The WAL writer flushes WAL records, the change log. The checkpointer flushes dirty data pages and marks a point from which crash recovery can start.

Putting PostgreSQL architecture to work

A few habits follow directly from this design:

• Pool connections instead of raising max_connections into the thousands.
• Size shared_buffers deliberately, and raise work_mem per role rather than globally.
• Watch checkpoint frequency and pg_stat_io to see whether backends are doing their own writes.
• Keep autovacuum on and let it keep up with your write rate.
• Turn on WAL archiving before you need point-in-time recovery, not after.

If you want a second pair of eyes on a PostgreSQL setup, Vectorkub's engineering team works on database-backed systems like these.