serverhabit.com

Practical server tutorials and cost-conscious self-hosting for indie hackers

How to Set Up Docker on Linux (The Right Way)

by David Park

Most Linux distros ship a version of Docker in their default repos that's anywhere from 6 months to 2 years behind the current release. On Ubuntu 22.04, apt install docker.io gets you Docker 20.10 — while Docker CE 26.x has been out since early 2024. That gap matters for security patches and newer features like BuildKit improvements.

So if you've ever followed a random tutorial, installed Docker, and then hit weird bugs or missing features, there's a good chance you installed the wrong package from the wrong source. This post fixes that.

I'll walk you through how to set up Docker on Linux the correct way — using Docker's official repo, configuring it properly for a single-user VPS, and running a real container to confirm everything works. No fluff, no enterprise nonsense.

Why the Distro Package Is Usually the Wrong Choice

The docker.io or docker-engine packages you get from apt or dnf are maintained by the distro team, not Docker Inc. They lag behind. They also sometimes have different default configurations — logging drivers, cgroup versions, storage backends — that can bite you later.

Docker's own install method adds their official APT or RPM repo and keeps you on the current stable release. It takes maybe 3 extra minutes. Just do it.

One exception: if you're on a locked-down corporate server where you can't add external repos, use what you've got. But for a personal VPS or a dev machine? Use the official repo every time.

Prerequisites Before You Start

You need:

  • A Linux server or VM. I'm using Ubuntu 22.04 LTS and Debian 12 for these examples, but the steps are nearly identical for both.
  • A non-root user with sudo access. Running everything as root works but is sloppy.
  • Basic terminal comfort. If you can ssh into a box and run commands, you're fine.

If you're spinning up a fresh VPS, Hetzner's CAX11 (ARM64, 2 vCPU, 4GB RAM) runs Docker perfectly and costs €3.79/month as of mid-2024. That's the box I use for most of my side projects.

Step 1: Remove Old Docker Packages

Before installing anything, clean house. If you've ever run apt install docker or docker.io before, remove it:

sudo apt-get remove docker docker-engine docker.io containerd runc

Don't worry if apt says those packages aren't installed. That's fine — the command won't error out, it just won't find anything to remove.

On Fedora/RHEL-based systems:

sudo dnf remove docker docker-client docker-client-latest docker-common docker-latest docker-latest-logrotate docker-logrotate docker-engine

Step 2: Install Docker Engine from the Official Repo

This is the main event. Docker's official docs have the authoritative steps, but here's the condensed version for Ubuntu/Debian.

Add Docker's GPG key and repo:

# Install dependencies
sudo apt-get update
sudo apt-get install -y ca-certificates curl

# Create the keyring directory
sudo install -m 0755 -d /etc/apt/keyrings

# Download Docker's GPG key
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

# Add the Docker repo
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

# Update and install
sudo apt-get update
sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

For Debian, replace ubuntu with debian in the repo URL. That's literally the only difference.

For Fedora:

sudo dnf -y install dnf-plugins-core
sudo dnf config-manager --add-repo https://download.docker.com/linux/fedora/docker-ce.repo
sudo dnf install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
sudo systemctl start docker
sudo systemctl enable docker

On Ubuntu/Debian, Docker starts automatically after install. On Fedora you need those last two systemctl commands.

Verify the install:

docker --version
# Docker version 26.1.4, build 5650f9b

If you see a version number, you're good.

Step 3: Run Docker Without Sudo

By default, Docker requires root. Every command starts with sudo docker .... That gets old fast, and it's also a mild footgun for scripting.

Add your user to the docker group:

sudo usermod -aG docker $USER

Then apply the group change. You can either log out and back in, or run:

newgrp docker

The newgrp approach works for your current session. Log out/in makes it permanent across all sessions.

Verify it works:

docker run hello-world

You should see Docker pull the hello-world image and print a confirmation message. No sudo needed.

A note on security: Adding a user to the docker group is effectively giving them root-equivalent access to the host. On a shared server, think twice. On your own VPS that only you access, it's fine.

Step 4: Configure Docker for a VPS (Don't Skip This)

Default Docker settings are fine for a laptop. On a VPS, you want a few tweaks.

Set up log rotation. Docker's default logging driver (json-file) will happily fill your disk if containers are chatty. Create /etc/docker/daemon.json:

sudo nano /etc/docker/daemon.json

Add this:

{
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "10m",
    "max-file": "3"
  }
}

This caps each container's logs at 10MB × 3 files = 30MB max. For a small VPS with a 40GB disk, that's sane.

Enable BuildKit by default. BuildKit is Docker's newer, faster build backend. It's been the default in recent Docker versions, but making it explicit doesn't hurt:

{
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "10m",
    "max-file": "3"
  },
  "features": {
    "buildkit": true
  }
}

Restart Docker to apply:

sudo systemctl restart docker

Step 5: Confirm Everything Works with a Real Example

hello-world is fine for a smoke test, but let's run something actually useful — Nginx serving a static page.

docker run -d -p 8080:80 --name test-nginx nginx:alpine

Breaking that down:

  • -d — run detached (in the background)
  • -p 8080:80 — map port 8080 on your host to port 80 in the container
  • --name test-nginx — give it a name so you can reference it later
  • nginx:alpine — use the Alpine-based Nginx image (smaller, ~23MB vs ~187MB for the full image)

Now curl it:

curl http://localhost:8080

You should get the default Nginx HTML back. If you're on a VPS, you can also hit http://your-server-ip:8080 from a browser (assuming your firewall allows port 8080).

Clean up when you're done:

docker stop test-nginx
docker rm test-nginx

Docker Compose: You'll Need It

If you installed Docker using the steps above, you already have the Compose plugin — it was part of the docker-compose-plugin package. Run it as:

docker compose version
# Docker Compose version v2.27.1

Note: it's docker compose (space), not docker-compose (hyphen). The hyphenated version is the old standalone V1 binary, which Docker deprecated in July 2023. If a tutorial tells you to use docker-compose, it's outdated.

Here's a minimal docker-compose.yml to confirm it works — a simple Nginx setup:

services:
  web:
    image: nginx:alpine
    ports:
      - "8080:80"

Save that, then:

docker compose up -d
docker compose ps
docker compose down

That's the basic workflow. Most of my self-hosted apps — Plausible, Gitea, Miniflux — run as Compose stacks on a single Hetzner box. It's dead simple to manage.

Quick Comparison: Install Methods

Method Package Source Docker Version Recommended?
apt install docker.io Ubuntu repos ~20.10 (stale) No
snap install docker Snap Store Usually current Rarely
Official Docker repo (this guide) docker.com Current stable (26.x) Yes
Docker Desktop for Linux Docker Inc Current Dev machines only

Snap Docker works but introduces weird filesystem permission quirks with bind mounts. I've hit those bugs twice and wasted hours. Stick with the official repo.

Troubleshooting Common Issues

"Permission denied" when running docker commands: You probably haven't applied the group change. Log out and back in, or run newgrp docker.

"Cannot connect to the Docker daemon": Docker isn't running. Check with sudo systemctl status docker. If it's failed, sudo journalctl -u docker -n 50 will show you why.

Port conflicts: If -p 8080:80 fails with "port is already allocated", something else is using 8080. Use ss -tlnp | grep 8080 to find it.

Disk filling up: Run docker system prune -a to remove stopped containers, unused images, and dangling build cache. I run this manually every month or two. Be careful — it removes all unused images, so if you've got images you want to keep that aren't attached to a running container, they'll go.

How to Set Up Docker on Linux: The Short Version

If you're setting up Docker on Linux today, here's what to do:

  1. Remove any existing docker packages
  2. Add Docker's official repo
  3. Install docker-ce, docker-ce-cli, containerd.io, docker-buildx-plugin, and docker-compose-plugin
  4. Add your user to the docker group
  5. Configure /etc/docker/daemon.json with log rotation
  6. Test with docker run -d -p 8080:80 nginx:alpine

That's it. The whole process takes about 10 minutes on a fresh box. Once Docker's running, you can start deploying real applications — check out this guide on how I run multiple apps on a single cheap VPS if you want to see what the full stack looks like.

If you're setting up a new server from scratch and want the full baseline config I use — firewall, fail2ban, Docker, Caddy as a reverse proxy — that's covered in /blog/vps-initial-server-setup.

Tomorrow's action: if you've got a VPS sitting around with docker.io installed from the distro repos, spend 15 minutes switching to the official Docker repo. You'll get a current version, better defaults, and one less thing to debug later.