basforge

Search titles, text and the “If it doesn’t work” cases.

    RU

    Traefik from scratch: the first service by name, with HTTPS

    Bring up Traefik as its own compose stack, open a test service and memos through it by name with a Let’s Encrypt certificate, and see which changes apply live and which only with a restart. You need a domain, A records pointing at the server, and free ports 80 and 443.

    What you end up with

    /opt/traefik/                       the proxy stack
      docker-compose.yaml
      data/traefik.yml                  static config: entry points, providers, certificates
      data/acme.json                    certificates and the account key (mode 600, not in git)
      data/dynamic/                     dynamic config: shared middleware, TLS options
    /opt/whoami/docker-compose.yaml     a test service: shows what arrived in the request
    /opt/memos/docker-compose.yaml      the first real service
    
    the proxy network is shared: Traefik and every service it exposes live on it
    only 80 and 443 are open to the outside; service ports aren't published

    Setup

    1. DNS: the names point at the server

      Create A records for whoami.example.org and memos.example.org pointing at the server. While setting up, keep them without Cloudflare’s proxy (grey cloud): that way Let’s Encrypt checks the server directly. Living behind the Cloudflare proxy — CrowdSec with Traefik behind Cloudflare and Traefik and Let’s Encrypt.

      dig +short whoami.example.org
      # 203.0.113.10
    2. Ports 80 and 443 are free

      ss -tlnp | grep -E ':(80|443) '
      # empty — the ports are free

      If nginx or apache is there, stop it and remove it from boot, or the Traefik container won’t start: Bind for 0.0.0.0:80 failed: port is already allocated.

      You still need port 80 after setup: Let’s Encrypt comes there for the HTTP-01 challenge.

    3. The shared network

      docker network create proxy

      The network is created separately from the stacks, so compose files mark it external: true. Recreate the Traefik stack and the network stays, with the services still attached.

    4. Directories and the certificate file

      mkdir -p /opt/traefik/data/dynamic /opt/traefik/data/logs
      touch /opt/traefik/data/acme.json
      chmod 600 /opt/traefik/data/acme.json

      touch is required: if the file is missing, Docker creates an empty directory in its place when mounting (A missing volume path), and Traefik can’t save certificates. Traefik checks the 600 mode itself.

    5. The static config

      /opt/traefik/data/traefik.yml
      global:
        checkNewVersion: false
        sendAnonymousUsage: false

      Turn off the new-version check: the image tag sets the version.

      log:
        level: INFO
      
      accesslog:
        format: json
        filePath: /logs/access.log

      Traefik’s own log goes to stdout, readable with docker logs traefik. The request log goes to a JSON file that CrowdSec and Loki read later (Traefik logs).

      entryPoints:
        http:
          address: ":80"
          http:
            aliasHeadersStrategy: delete
            redirections:
              entryPoint:
                to: https
                scheme: https
      Key part

      Every request on 80 is sent to HTTPS right at the entry point. Separate http routers and redirect middleware in service labels aren’t needed after that.

        https:
          address: ":443"
          http:
            aliasHeadersStrategy: delete
            tls:
              certResolver: letsencrypt

      A resolver on the entry point: every router on it gets a certificate, so you can leave tls=true out of the labels. aliasHeadersStrategy: delete goes on both entry points: headers with _ or . in the name are dropped. PHP and nginx read X_Auth_User as X-Auth-User, so a visitor could forge a header that Traefik sets. Without the line, Traefik 3.7 logs a warning for each entry point.

      providers:
        docker:
          exposedByDefault: false
          network: proxy
        file:
          directory: /dynamic/
          watch: true

      Only containers with traefik.enable=true get exposed. Traefik reaches them through the proxy network, even if they’re on networks of their own too.

      certificatesResolvers:
        letsencrypt:
          acme:
            email: [email protected]
            storage: acme.json
            httpChallenge:
              entryPoint: http

      Write the address literally: ${…} isn’t substituted in this file (traefik.yml doesn’t substitute variables). HTTP-01 validates through port 80.

      What each line means and what else is possible — traefik.yml block by block.

    6. The proxy’s compose file

      /opt/traefik/docker-compose.yaml
      services:
        traefik:
          image: traefik:v3.7
          container_name: traefik
          restart: unless-stopped
          security_opt:
            - no-new-privileges:true

      The v3.7 tag pins the minor version: fixes arrive with docker compose pull, and syntax changes between minor versions only when you change the tag yourself.

          ports:
            - 80:80
            - 443:443
      Key part

      Only Traefik publishes ports: 80 takes the redirect and the Let’s Encrypt check, 443 serves the sites. Services have no ports:.

          volumes:
            - /var/run/docker.sock:/var/run/docker.sock:ro
            - ./data/traefik.yml:/traefik.yml:ro
            - ./data/acme.json:/acme.json
            - ./data/dynamic/:/dynamic/:ro
            - ./data/logs/:/logs/

      docker.sock is needed to read labels; :ro doesn’t make it safe (docker.sock). Mount dynamic/ as a directory: a single-file mount doesn’t see changes after git pull (Single-file mounts).

          networks:
            - proxy
      
      networks:
        proxy:
          external: true

      The proxy network was created in step 3, so here it’s external: true.

      Start it and check the log for errors and warnings:

      cd /opt/traefik && docker compose up -d
      docker logs traefik 2>&1 | grep -E 'ERR|WRN'
      # … WRN Traefik can reject some encoded characters in the request path. …

      One warning stays, and it’s information, not an error. Traefik lets encoded characters through in the path (%2F, %00, %3B and four more), and you can forbid them under the entry point’s encodedCharacters. For a typical site this changes nothing, so leave the config as is. Any other ERR or WRN line — sort it out before the next step.

    7. The test service

      traefik/whoami answers with text: which headers arrived and from which address. Use it to check the route, the certificate and X-Forwarded-For before a real service is connected.

      /opt/whoami/docker-compose.yaml
      services:
        whoami:
          image: traefik/whoami
          container_name: whoami
          restart: unless-stopped
          labels:
            - "traefik.enable=true"
            - "traefik.http.routers.whoami.rule=Host(`whoami.example.org`)"
            - "traefik.http.routers.whoami.entrypoints=https"
      Key part

      No port is given: the image exposes one port, 80, and Traefik takes it on its own. No service is declared either — Traefik creates one and binds it to the router.

          networks:
            - proxy
      
      networks:
        proxy:
          external: true

      There’s no ports:: from the outside whoami is reachable only through Traefik.

      cd /opt/whoami && docker compose up -d
      curl -s https://whoami.example.org | grep -E '^(Host|X-Forwarded-Proto|X-Real-Ip):'
      # Host: whoami.example.org
      # X-Forwarded-Proto: https
      # X-Real-Ip: <the address you ran curl from>

      The first request to a new name can take 10–30 s while Traefik obtains the certificate. Until it has one, the browser sees TRAEFIK DEFAULT CERT.

    8. The first real service

      /opt/memos/docker-compose.yaml
      services:
        memos:
          image: neosmemo/memos:stable
          container_name: memos
          restart: unless-stopped
          volumes:
            - ./data:/var/opt/memos

      Data goes to ./data next to the compose file, that is /opt/memos/data.

          labels:
            - "traefik.enable=true"
            - "traefik.http.routers.memos.rule=Host(`memos.example.org`)"
            - "traefik.http.routers.memos.entrypoints=https"
            - "traefik.http.services.memos.loadbalancer.server.port=5230"
      Key part

      Here the port is explicit: memos listens on 5230. If an image exposes several ports, Traefik takes the lowest — not always the one you need.

          networks:
            - proxy
      
      networks:
        proxy:
          external: true

      Spell the router name (memos) the same in all its labels. A typo raises no error: Traefik creates a second router, and the settings drift apart (Middleware on a missing router).

    9. A shared middleware from a file

      What many services need lives in data/dynamic/ and is attached by name with the @file suffix.

      # /opt/traefik/data/dynamic/middlewares.yml
      http:
        middlewares:
          global-ratelimit:
            rateLimit:
              average: 100
              burst: 50
              period: 1s

      The limit is counted per address: 100 requests per second on average plus a burst of up to 50 more. Whoever goes over gets 429. If Cloudflare sits in front of Traefik, you also need sourceCriterion.ipStrategy.depth: 1, or all visitors coming through one Cloudflare node share the limit (traefik.yml block by block).

      The file is picked up on save. Attach it to memos with one more label and recreate the container:

            - "traefik.http.routers.memos.middlewares=global-ratelimit@file"
      cd /opt/memos && docker compose up -d

    How to make changes

    What you change Where How to apply What gets interrupted
    shared middleware, TLS options data/dynamic/*.yml save the file nothing
    a service’s name, middleware, port labels in the service’s compose file docker compose up -d in the service directory: the container is recreated, and Traefik sees the event only that service, while it’s recreated
    entry points, providers, certificates, logs, plugins data/traefik.yml docker compose restart traefik every site, for under a second: connections are refused first, then 404 while Traefik reads the labels again
    Traefik’s ports, volumes, image the proxy’s compose file docker compose up -d in /opt/traefik every site while the container is recreated
    certificates acme.json don’t edit by hand —

    An error in a file from dynamic/ — broken YAML syntax or an unknown field — goes to the log, and Traefik keeps running on the previous config. Only until a restart, though: started with a broken file, Traefik takes nothing from it, and routers using a middleware from that file return 404. An error in traefik.yml is worse: Traefik won’t start, and every site goes down at once.

    If it doesn’t work

    The log says permissions 644 for /acme.json are too open, please use 600, and there are no certificates

    Cause
    The file was created with default permissions.
    Fix
    chmod 600 /opt/traefik/data/acme.json and docker compose restart traefik.

    The log says is a directory about acme.json

    Cause
    The file didn’t exist on first start, so Docker created a directory in its place (A missing volume path).
    Fix
    docker compose down, rmdir data/acme.json, the touch and chmod 600 from step 4, docker compose up -d.

    No certificate is issued; the log has an acme error with connection or timeout

    Cause
    Let’s Encrypt couldn’t reach port 80: the provider’s firewall or a rule on the server blocks it.
    Fix
    Test from a phone on mobile data: http://whoami.example.org should answer with a redirect. If port 80 can’t be opened, use the DNS challenge (Traefik and Let’s Encrypt).

    The log says too many failed authorizations recently

    Cause
    Five failed validations per name per hour — after that Let’s Encrypt refuses that account for that name until the hour passes.
    Fix
    While experimenting, switch to the staging server: caServer: https://acme-staging-v02.api.letsencrypt.org/directory. Then remove the line, clear acme.json and restart.

    404 page not found on whoami or memos

    Cause
    No traefik.enable=true, a typo in the Host name, or the container isn’t running.
    Fix
    Check docker ps --filter name=memos and compare the labels.

    Gateway Timeout after about 30 seconds

    Cause
    The container isn’t on the proxy network: its compose file has no networks block.
    Fix
    Add the network; see who’s on it with docker network inspect proxy --format '{{range .Containers}}{{.Name}} {{end}}'.

    Traefik sees no containers at all; the log says client version 1.24 is too old

    Cause
    Docker 29 with Traefik older than 3.6.1.
    Fix
    Update the image: traefik:v3.7.

    How to check

    First the HTTP redirect. For GET Traefik answers 301, for other methods 308, so the method and body aren’t lost. curl -I sends HEAD, hence 308 in the check:

    # HTTP redirects to HTTPS
    curl -sI http://whoami.example.org | head -3
    # HTTP/1.1 308 Permanent Redirect
    # Location: https://whoami.example.org/
    # the certificate is from Let's Encrypt, not TRAEFIK DEFAULT CERT
    echo | openssl s_client -connect whoami.example.org:443 -servername whoami.example.org 2>/dev/null \
      | openssl x509 -noout -issuer -enddate
    # issuer=C = US, O = Let's Encrypt, CN = …
    # notAfter=…

    The expiry should be about 90 days from today.

    # the memos port is closed from outside: the service opens only by name
    curl -m 5 -s http://203.0.113.10:5230 || echo "closed"
    # closed