postgresql shared memory in docker swarm: when free disk is not enough

An admin query that had worked for months suddenly started returning HTTP 500.

PostgreSQL reported this:

could not resize shared memory segment "/PostgreSQL.3291219976"
to 16777216 bytes: No space left on device
SQLSTATE 53100

The VM had free disk space. The database container was healthy. The query was not writing a large export.

No space left on device was accurate, but it was talking about the wrong device. The full filesystem was /dev/shm inside the container.

Which memory ran out?

The request loaded an administrative data view and several aggregate statistics. While executing it, PostgreSQL needed another 16 MB of dynamic shared memory. The allocation failed because the container’s /dev/shm was too small.

PostgreSQL can use dynamic shared memory for parallel query execution and other runtime work. Docker containers commonly start with a small POSIX shared memory filesystem unless it is configured explicitly.

That limit is independent from both the database cache and the persistent disk:

resource	purpose
shared_buffers	PostgreSQL’s main buffer cache
/dev/shm	runtime shared memory, including dynamic query allocations
/var/lib/postgresql	persistent database files

A large shared_buffers value does not make /dev/shm larger. Free space under /var/lib/postgresql does not help either.

The configuration that looked right

My first attempt added a service-level tmpfs entry:

services:
  postgres:
    tmpfs:
      - /dev/shm:size=1g

The stack deployed without an obvious error. The running container still had the old shared memory size.

The useful check was the live service specification:

docker service inspect app_postgres \
  --format '{{json .Spec.TaskTemplate.ContainerSpec.Mounts}}' | jq

It contained the persistent database volume, but no /dev/shm mount.

The YAML had described what I wanted. The running task proved that Swarm had not applied it.

Mounting tmpfs in a Swarm service

For this stack, the reliable form was a long-form entry under volumes:

services:
  postgres:
    image: postgres:18
    volumes:
      - postgres_data:/var/lib/postgresql
      - type: tmpfs
        target: /dev/shm
        tmpfs:
          size: 1073741824

The size is in bytes, so 1073741824 is 1 GiB.

The two mounts have separate jobs. postgres_data keeps database files across task replacements. /dev/shm is RAM-backed workspace that disappears with the container.

Verifying the running container

After deploying the corrected stack, I checked the mount from inside the new task:

PG=$(docker ps -q --filter name=app_postgres | head -n1)
docker exec "$PG" df -h /dev/shm

The result was:

Filesystem      Size  Used Avail Use% Mounted on
tmpfs           1.0G  1.1M  1023M   1% /dev/shm

That confirmed three things:

• Swarm had created a new task
• the tmpfs mount was active at the expected size
• the persistent database volume was untouched

The query worked again without the shared memory error.

What does a 1 GiB tmpfs cost?

The 1g value is a limit, not an immediate reservation. The mount used almost no memory at startup and grew only when processes wrote to it.

It is still real memory from the container and host. After raising the limit, PostgreSQL memory usage needs monitoring, especially if shared_buffers and work_mem are already aggressive.

Applying the mount also replaces the PostgreSQL task. That causes a short database interruption, but it does not remove persistent data.

Trust the live mount

When PostgreSQL says No space left on device, these checks answer different questions:

df -h
docker exec "$PG" df -h /dev/shm
docker service inspect app_postgres \
  --format '{{json .Spec.TaskTemplate.ContainerSpec.Mounts}}' | jq

The first checks host filesystems. The second checks shared memory inside the container. The third checks whether Swarm actually attached the requested mount.

A healthy disk, a 1/1 service, and a successful deploy can all coexist with the wrong /dev/shm. The live container is the source of truth.