What broke?
Symptoms from the “If it doesn’t work” sections of every entry. Each link goes straight to the cause and the fix.
SSHLinux
Permission denied (publickey)SSH: how logging in to a server works and what else it’s forConnection refusedSSH: how logging in to a server works and what else it’s forConnection timed out— no refusal, no promptSSH: how logging in to a server works and what else it’s forWARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!SSH: how logging in to a server works and what else it’s forReceived disconnect … Too many authentication failuresSSH: how logging in to a server works and what else it’s for- The session freezes after a few idle minutesSSH: how logging in to a server works and what else it’s for
Permission denied (publickey)even though the key was addedSSH keys: from ssh-keygen to passwordless loginWARNING: UNPROTECTED PRIVATE KEY FILE!and the key isn’t usedSSH keys: from ssh-keygen to passwordless loginLoad key "…": invalid formatSSH keys: from ssh-keygen to passwordless login- No
ssh-copy-id:ssh-copy-id: command not foundSSH keys: from ssh-keygen to passwordless login - The server still asks for the user’s passwordSSH keys: from ssh-keygen to passwordless login
- Forgot the key’s passphraseSSH keys: from ssh-keygen to passwordless login
- The file says
PasswordAuthentication no, but the server still asks for a passwordsshd_config: key-only login, no root, no passwords - After restarting
sshd, new sessions can’t get in:Connection refusedsshd_config: key-only login, no root, no passwords sudo sshd -tsaysMissing privilege separation directory: /run/sshdsshd_config: key-only login, no root, no passwords- Changed
Port, butsshdstill listens on 22sshd_config: key-only login, no root, no passwords - Root can still log in with a keysshd_config: key-only login, no root, no passwords
- Added
AllowUsers— and nobody can log insshd_config: key-only login, no root, no passwords - A setting in a host block has no effect~/.ssh/config: short names, a key per server, jumping through a server
Bad owner or permissions on /home/me/.ssh/config~/.ssh/config: short names, a key per server, jumping through a server- On Windows the connection fails with
ControlMaster~/.ssh/config: short names, a key per server, jumping through a server git clone git@github-infra:…—Could not resolve hostname github-infra~/.ssh/config: short names, a key per server, jumping through a server- Through
ProxyJump—channel 0: open failed: administratively prohibited~/.ssh/config: short names, a key per server, jumping through a server Could not open a connection to your authentication agent.ssh-agent: type the passphrase once per session, and why not ForwardAgent- On Windows:
Error connecting to agent: No such file or directoryssh-agent: type the passphrase once per session, and why not ForwardAgent - The key is in the agent, but
git pushstill asks for the passphrasessh-agent: type the passphrase once per session, and why not ForwardAgent sign_and_send_pubkey: signing failed for ED25519 … agent refused operationssh-agent: type the passphrase once per session, and why not ForwardAgent- Many keys in the agent, and the server drops the connection:
Too many authentication failuresssh-agent: type the passphrase once per session, and why not ForwardAgent bind [127.0.0.1]:8080: Address already in use, the tunnel didn’t openSSH tunnels: a closed admin panel through localhost- The page doesn’t open, and the ssh window shows
channel 2: open failed: connect failed: Connection refusedSSH tunnels: a closed admin panel through localhost channel 2: open failed: administratively prohibited: open failedSSH tunnels: a closed admin panel through localhost- A background tunnel stopped answering after an hourSSH tunnels: a closed admin panel through localhost
- A container with no published port; the tunnel to
127.0.0.1:8080doesn’t reach itSSH tunnels: a closed admin panel through localhost WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!after reinstalling the serverCheck the server’s fingerprint before you answer yes- The same warning, and nobody touched the serverCheck the server’s fingerprint before you answer yes
- After the port change SSH is still on 22, and the new one refusesMoving off port 22 cuts log noise but doesn’t protect you
- After the port change —
Connection timed outMoving off port 22 cuts log noise but doesn’t protect you
Ubuntu serverLinux
- After
ufw enableSSH won’t let you in, new sessions hangA new Ubuntu server: what to set up in the first hour and why Could not get lock /var/lib/dpkg/lock-frontendA new Ubuntu server: what to set up in the first hour and why- Login shows
*** System restart required ***A new Ubuntu server: what to set up in the first hour and why - Processes on a small VPS vanish, and
dmesgshowsOut of memory: Killed processA new Ubuntu server: what to set up in the first hour and why - The disk filled up within a few months, though there’s little dataA new Ubuntu server: what to set up in the first hour and why
- The server clock lags, TLS certificates are “not yet valid”A new Ubuntu server: what to set up in the first hour and why
Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 1234 (unattended-upgr)First login to a server: updates, hostname, a sudo user, key login- During the upgrade — a dialog
A new version of configuration file /etc/ssh/sshd_config is availableFirst login to a server: updates, hostname, a sudo user, key login sudo: unable to resolve host web1: Name or service not knownFirst login to a server: updates, hostname, a sudo user, key login- After a reboot the hostname is
vm-4821againFirst login to a server: updates, hostname, a sudo user, key login example-user is not in the sudoers fileFirst login to a server: updates, hostname, a sudo user, key login- The new user logs in but gets
Could not chdir to home directory /home/example-userFirst login to a server: updates, hostname, a sudo user, key login - Updates don’t install,
/var/log/unattended-upgrades/is emptyunattended-upgrades: security updates on their own and a reboot at 4:00 - The server rebooted at 4:00 UTC, not local timeunattended-upgrades: security updates on their own and a reboot at 4:00
- All containers restarted overnightunattended-upgrades: security updates on their own and a reboot at 4:00
aptsaysThe following upgrades have been deferred due to phasingunattended-upgrades: security updates on their own and a reboot at 4:00/bootfilled up, the kernel update fails withNo space left on deviceunattended-upgrades: security updates on their own and a reboot at 4:00swapon: /swapfile: swapon failed: Invalid argumentorskipping - it appears to have holesSwap, journal and time: three settings for a small VPSswapon: /swapfile: insecure permissions 0644, 0600 suggestedSwap, journal and time: three settings for a small VPS- The journal still takes several gigabytesSwap, journal and time: three settings for a small VPS
timedatectl:System clock synchronized: noSwap, journal and time: three settings for a small VPS- 26.04:
chronyc sourcesshows every source as^?withReach 0, though UDP 123 is openSwap, journal and time: three settings for a small VPS Failed to set ntp: NTP not supported, orchronydwon’t startSwap, journal and time: three settings for a small VPS- The server was created, but nothing from the file was appliedcloud-init: a server configured the moment it’s created
- Logging in as
example-userworks, butsudoasks for a password that doesn’t existcloud-init: a server configured the moment it’s created - One
runcmdcommand failed, the rest workedcloud-init: a server configured the moment it’s created - A minute after creation SSH lets nobody incloud-init: a server configured the moment it’s created
Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 1234 (unattended-upgr)New server: the first hour- During the upgrade a dialog says
A new version of configuration file /etc/ssh/sshd_config is availableNew server: the first hour - After a reboot the hostname is
vm-4821againNew server: the first hour example-user is not in the sudoers fileNew server: the first hourssh example-user@…asks for the user’s password, not the key’sNew server: the first hour- After
restart sshnew sessions fail withConnection refusedNew server: the first hour Permission denied (publickey)forexample-user, though the key worked before stage 3New server: the first hour- After
ufw enablenew SSH sessions hang and time outNew server: the first hour --dry-runinstalls nothing,/var/log/unattended-upgrades/is emptyNew server: the first hour- The server rebooted at 4:00 UTC, not local timeNew server: the first hour
swapon: /swapfile: swapon failed: Invalid argumentorskipping - it appears to have holesNew server: the first hourtimedatectl show -p NTPSynchronized --valuesaysnoNew server: the first hourdocker-ce : Conflicts: docker.ioduring installNew server: the first hourpermission denied while trying to connect to the Docker daemon socketNew server: the first hourzramswap.servicefailed; the journal saysError: inserting the zram kernel modulezram-tools: compressed swap in memory- Changed
ALGOorPERCENT, butzramctlshows the old valueszram-tools: compressed swap in memory zramswap.servicewon’t start; the log showsecho: write error: Device or resource busyandError: setting compression algo to zstdzram-tools: compressed swap in memory- There’s swap, but processes still get killed by
Out of memoryzram-tools: compressed swap in memory - The server got sluggish;
topshows highsyandkswapd0zram-tools: compressed swap in memory
FirewallLinux
- After
systemctl restart nftablescontainers can’t reach the internet: even names don’t resolve,wget: bad addressnftables: the Linux kernel firewall that replaces iptables - The port is allowed in your table, yet from the outside it still times outnftables: the Linux kernel firewall that replaces iptables
- After enabling your firewall the server stopped answering over IPv6, while IPv4 works finenftables: the Linux kernel firewall that replaces iptables
- A container can’t connect to a service on the host itself — a database or
host.docker.internalnftables: the Linux kernel firewall that replaces iptables - A
tcp dport 8080 droprule inforwarddoesn’t close the published port8080:80nftables: the Linux kernel firewall that replaces iptables iptables -Lshows one thing,nft list rulesetanother, andnftprintsWarning: iptables-legacy tables presentnftables: the Linux kernel firewall that replaces iptables- An
nft add … { … ; }command in the terminal fails with a syntax errornftables: the Linux kernel firewall that replaces iptables - The rules are gone after a rebootnftables: the Linux kernel firewall that replaces iptables
- After
ufw enable, new SSH sessions hang and time outThe ufw firewall: everything closed except SSH and web ufw deny 8080, but the container’s port is open from outsideThe ufw firewall: everything closed except SSH and web- Blocked an address, but it still gets inThe ufw firewall: everything closed except SSH and web
- The port is closed over IPv4 but open over IPv6The ufw firewall: everything closed except SSH and web
- With
ufw limit OpenSSH, a deploy script orrsyncin a loop can no longer connectThe ufw firewall: everything closed except SSH and web - The address is banned, but the site in a container still opens for itDocker ports bypass the input chain
ufw deny 8080, but the container port is still open from outsideDocker ports bypass the input chain- A container with
network_mode: hostgets blocked, the others don’tDocker ports bypass the input chain - The firewall doesn’t ban visitors of a site behind CloudflareDocker ports bypass the input chain
- The ban works over IPv4 but not over IPv6Docker ports bypass the input chain
systemdLinux
- Some sites didn’t come back after a rebootsystemd: delayed, one-by-one start of compose stacks
- After a reboot, a service came up that should stay offsystemd: delayed, one-by-one start of compose stacks
- Edited the unit — nothing changedsystemd: delayed, one-by-one start of compose stacks
- The script started nothing, and
systemctl statusshowsactive (exited)systemd: delayed, one-by-one start of compose stacks - The server moved to a fast disksystemd: delayed, one-by-one start of compose stacks
- The server changed roles and has hardly any stacks leftsystemd: delayed, one-by-one start of compose stacks
Docker ComposeDocker
ERROR 2002 … Can't connect to local server through socketright afterup -dMariaDB in a container: compose, explainedERROR 1045 Access deniedafter a password changeMariaDB in a container: compose, explainedcrash recoveryin the log after a restartMariaDB in a container: compose, explaineddeprecatedin the logMariaDB in a container: compose, explained- Changed the app user’s password — the site can’t connect to the databaseMariaDB in a container: compose, explained
commandstill holds settings for different hardwareMariaDB in a container: compose, explained- The root password ended up in shell history or a chatMariaDB in a container: compose, explained
- Added a line to
.env— a container that doesn’t need it got recreated.env feeds compose substitution, env_file goes into the container - Moved a Basic Auth hash into
.env— the password stopped workingThe $ sign: $$ in compose, single quotes in .env - After
git pullandup -d, Traefik didn’t come up and every site is downThe secret file comes before up -d - Moved the stack directory — the app started with an empty databaseThe stack directory is the compose project name and the volume prefix
- Started the stack from another directory — the app came up with an empty databaseUse ./ in volume paths, not $PWD
Docker EngineDocker
- After
docker compose up -dthe data is gone: the database is empty, uploaded files have vanishedDocker: what it is, where it came from and what it costs docker stopanddocker compose downwait exactly 10 seconds every timeDocker: what it is, where it came from and what it costs- A container port is open to the outside even though ufw doesn’t allow itDocker: what it is, where it came from and what it costs
toomanyrequests: You have reached your pull rate limitDocker: what it is, where it came from and what it costsexec format errorright when the container startsDocker: what it is, where it came from and what it costs- You updated an image, and the service stopped starting, though compose wasn’t touchedDocker: what it is, where it came from and what it costs
- The server stopped responding, and
dmesgsaysOut of memory: Killed processDocker: what it is, where it came from and what it costs - A container can’t find its neighbour by name:
could not translate host name "db"Docker: what it is, where it came from and what it costs - The disk is full, though there isn’t much data in the volumesDocker: what it is, where it came from and what it costs
permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sockDocker on Ubuntu: from Docker’s repository, with a log limitdocker-ce : Conflicts: docker.ioduring installDocker on Ubuntu: from Docker’s repository, with a log limit- Docker from snap:
docker composecan’t see files in/optor/srvDocker on Ubuntu: from Docker’s repository, with a log limit /var/lib/docker/containers/*/*-json.logfiles filled the diskDocker on Ubuntu: from Docker’s repository, with a log limitdocker: 'compose' is not a docker commandDocker on Ubuntu: from Docker’s repository, with a log limit- A container port is open from outside though ufw doesn’t allow itDocker on Ubuntu: from Docker’s repository, with a log limit
- Removed a stack, and its directory came back — empty and owned by rootA missing volume path makes Docker create an empty directory
- Updated
nginx.confwithgit pull, rannginx -s reload— nothing changedA single-file mount doesn’t see your edits - A script updates an address list,
reloadsucceeds, but nginx has the old listA single-file mount doesn’t see your edits - The file nginx reads hasn’t changed in months, while another one grows next to itA single-file mount doesn’t see your edits
- Edited a mounted file in
vimright on the server — the container has the old versionA single-file mount doesn’t see your edits - WordPress asks for FTP credentials when installing a pluginWho owns WordPress files in Docker: www-data and uid 33
Unable to create directory wp-content/uploads/2026/10. Is its parent directory writable by the server?Who owns WordPress files in Docker: www-data and uid 33- Ran
chown www-data:www-dataon the host, and WordPress still can’t writeWho owns WordPress files in Docker: www-data and uid 33 - Pages open, but images and CSS return
403Who owns WordPress files in Docker: www-data and uid 33 - After restoring from backup the files have the wrong owner againWho owns WordPress files in Docker: www-data and uid 33
ls -lon the host shows the owner as100033Who owns WordPress files in Docker: www-data and uid 33
Images and DockerfileDocker
npm ciruns again after every text editYour own image: a multi-stage build of a static site- The build starts with
transferring contextof hundreds of megabytesYour own image: a multi-stage build of a static site npm cican only install packages when your package.json and package-lock.json … are in syncYour own image: a multi-stage build of a static site- The image works on the laptop, but on the server —
exec format errorYour own image: a multi-stage build of a static site - The build fails on the last step:
nginx: [emerg] … in /etc/nginx/conf.d/default.conf:5Your own image: a multi-stage build of a static site - The build in the container fails:
You installed esbuild for another platform than the one you're currently using.dockerignore — before the first COPY . . .envwas added to.dockerignore, but it’s still in the old image.dockerignore — before the first COPY . .npm ciin the build fails on a private package:404 Not Foundor401 UnauthorizedA build token goes through --secret, not ARG- The token was already built in through
ARG, and the image is in the registryA build token goes through --secret, not ARG
Docker networksDocker
- A container can’t find its neighbor:
could not translate host name "db" to addressorgetaddrinfo ENOTFOUND dbDocker networks: neighbors by name, a shared network with Traefik and a database with no way out - Containers started with
docker run; the neighbor answers by IP, but by name —bad addressDocker networks: neighbors by name, a shared network with Traefik and a database with no way out network proxy declared as external, but could not be foundDocker networks: neighbors by name, a shared network with Traefik and a database with no way out- The service is on two networks, and Traefik returns
Gateway Timeout— always or after a recreateDocker networks: neighbors by name, a shared network with Traefik and a database with no way out - A host service doesn’t open from the container at
127.0.0.1Docker networks: neighbors by name, a shared network with Traefik and a database with no way out - A new stack came up, and the server vanished for machines on the home or office networkDocker networks: neighbors by name, a shared network with Traefik and a database with no way out
- The service got the
proxynetwork and stopped finding the database:could not translate host name "db" to addressA service’s networks list replaces the default network - Another stack came up, and SSH to the server from the office or a neighboring home network hangsDocker’s subnet clashed with the home network — set default-address-pools
- A device on the home network doesn’t open from a container, but opens from the hostDocker’s subnet clashed with the home network — set default-address-pools
could not find an available, non-overlapping IPv4 address pool among the defaults to assign to the networkDocker’s subnet clashed with the home network — set default-address-pools
TraefikHomelab
- A blank
404 page not foundinstead of the siteTraefik: what it is and when to pick it - The browser complains about the certificate, which says
TRAEFIK DEFAULT CERTTraefik: what it is and when to pick it Gateway Timeoutafter about 30 secondsTraefik: what it is and when to pick itBad Gatewayright awayTraefik: what it is and when to pick it- After a Docker upgrade every site returns 404, and Traefik’s log says
client version 1.24 is too oldTraefik: what it is and when to pick it - A change in
traefik.ymlhas no effectTraefik: what it is and when to pick it - A large upload breaks off at about one minuteTraefik: what it is and when to pick it
- The log says
permissions 644 for /acme.json are too open, please use 600, and there are no certificatesTraefik from scratch: the first service by name, with HTTPS - The log says
is a directoryaboutacme.jsonTraefik from scratch: the first service by name, with HTTPS - No certificate is issued; the log has an
acmeerror withconnectionortimeoutTraefik from scratch: the first service by name, with HTTPS - The log says
too many failed authorizations recentlyTraefik from scratch: the first service by name, with HTTPS 404 page not foundon whoami or memosTraefik from scratch: the first service by name, with HTTPSGateway Timeoutafter about 30 secondsTraefik from scratch: the first service by name, with HTTPS- Traefik sees no containers at all; the log says
client version 1.24 is too oldTraefik from scratch: the first service by name, with HTTPS - After editing
traefik.ymlthe container restarts in a looptraefik.yml block by block: what each setting means docker logs traefikis emptytraefik.yml block by block: what each setting means- The request log and CrowdSec show only Cloudflare addressestraefik.yml block by block: what each setting means
- The rate limit hits everyone at oncetraefik.yml block by block: what each setting means
- A new service is exposed though it has no
traefik.enabletraefik.yml block by block: what each setting means - The wildcard isn’t issued; the log mentions TXT record propagationtraefik.yml block by block: what each setting means
- Sites suddenly show Traefik’s self-signed certificateTraefik and Let’s Encrypt: wildcard, ACME, a certificate for neighbours
- The config says
email: ${EMAIL_FOR_SSL}Traefik and Let’s Encrypt: wildcard, ACME, a certificate for neighbours - nginx serves an expired certificate while Traefik’s is freshTraefik and Let’s Encrypt: wildcard, ACME, a certificate for neighbours
acme.jsonhas permissions wider than 600Traefik and Let’s Encrypt: wildcard, ACME, a certificate for neighbours- Experiments hit Let’s Encrypt rate limitsTraefik and Let’s Encrypt: wildcard, ACME, a certificate for neighbours
- The certificate doesn’t renewTraefik and Let’s Encrypt: wildcard, ACME, a certificate for neighbours
dig +shortprints nothingA service over HTTPS: Traefik and the first servicessshows nginx or apache on:80or:443A service over HTTPS: Traefik and the first service- The log says
is a directoryaboutacme.jsonA service over HTTPS: Traefik and the first service - The log says
permissions 644 for /acme.json are too open, please use 600A service over HTTPS: Traefik and the first service curlto a new name prints nothing, andcurl -vcomplains about the certificate:TRAEFIK DEFAULT CERTA service over HTTPS: Traefik and the first service404 page not foundon whoami or memosA service over HTTPS: Traefik and the first serviceGateway Timeoutafter about 30 secondsA service over HTTPS: Traefik and the first service- No certificate gets issued, and the log shows an
acmeerror withconnectionortimeoutA service over HTTPS: Traefik and the first service - The log says
too many failed authorizations recentlyA service over HTTPS: Traefik and the first service - Port 5230 answers from the outsideA service over HTTPS: Traefik and the first service
- Added CrowdSec to a router — the rate limit disappearedA router’s middlewares go in one line: a second label overwrites the first
- Attached protection to a router — the site works, but there’s no protectionMiddleware on a router that doesn’t exist raises no error
CrowdSecHomelab
cscli metrics show acquisitionshows noauth.loglineProtection: brute-force bans and closed ports- The
auth.logline is there, but the numbers are zerosProtection: brute-force bans and closed ports Bind for 127.0.0.1:8080 failed: port is already allocatedProtection: brute-force bans and closed portscsclisaysfirewall-bounceralready exists, andapi_keyin the config is emptyProtection: brute-force bans and closed ports- The bouncer won’t start;
journalctl -u crowdsec-firewall-bouncershows a key error or403Protection: brute-force bans and closed ports nft list table ip crowdsec | grep hookshows onlyinputProtection: brute-force bans and closed ports- The ban decision is there, but Chrome on the phone still says
ERR_CONNECTION_REFUSEDright awayProtection: brute-force bans and closed ports - You banned your own address: SSH from your computer says
Connection timed outProtection: brute-force bans and closed ports nmapshows a port open besides 22, 80 and 443Protection: brute-force bans and closed portssslists more addresses than:22Protection: brute-force bans and closed portscscli metricsshows no lines fromauth.logCrowdSec against SSH brute force: a ban across the whole server- The bouncer won’t start;
journalctl -u crowdsec-firewall-bouncershows a key error or403CrowdSec against SSH brute force: a ban across the whole server - Banned your own address; SSH gives
Connection timed outCrowdSec against SSH brute force: a ban across the whole server - Ban decisions exist, but the address still connectsCrowdSec against SSH brute force: a ban across the whole server
- The CrowdSec container eats hundreds of megabytes and keeps growingCrowdSec against SSH brute force: a ban across the whole server
- Your own address is banned, but the site behind Cloudflare opensCrowdSec with Traefik behind Cloudflare: two bouncers and X-Forwarded-For
- The ban is there, but the site on a direct domain opensCrowdSec with Traefik behind Cloudflare: two bouncers and X-Forwarded-For
- After editing
crowdsec.yml, every site with the plugin went downCrowdSec with Traefik behind Cloudflare: two bouncers and X-Forwarded-For - CrowdSec finds nothingCrowdSec with Traefik behind Cloudflare: two bouncers and X-Forwarded-For
cscli bouncers listshows lots of[email protected]entriesCrowdSec with Traefik behind Cloudflare: two bouncers and X-Forwarded-For- Changed the ban duration, but old bans keep the old oneCrowdSec with Traefik behind Cloudflare: two bouncers and X-Forwarded-For
- You banned yourselfCrowdSec with Traefik behind Cloudflare: two bouncers and X-Forwarded-For
SecretsHomelab
- A secret got into git — even a private repositorySecrets in Docker stacks: where to keep them and how not to leak them
- A password ended up in a chat, a ticket or a screenshotSecrets in Docker stacks: where to keep them and how not to leak them
.envwith mode 644Secrets in Docker stacks: where to keep them and how not to leak them- The same secret variable sits in the
.envof several stacks, but only one reads itSecrets in Docker stacks: where to keep them and how not to leak them - A container needs one variable, but it has
env_file: .envSecrets in Docker stacks: where to keep them and how not to leak them - A script made backup copies of
.envin/tmpSecrets in Docker stacks: where to keep them and how not to leak them - A key from a file is rejected, though it’s correctSecrets in Docker stacks: where to keep them and how not to leak them
- Changed a secret’s path in compose — the container won’t be createdSecrets in Docker stacks: where to keep them and how not to leak them
Server changesHomelab
deploy.shsayscat: /srv/secrets/deploy/hc-url: No such file or directoryDeploy without SSH- The
web1-deploycheck in Healthchecks.io staysnewor goesdownwhile the server is aliveDeploy without SSH - An alert, and the check’s log says
В /opt правка без коммита(an uncommitted change in /opt)Deploy without SSH - An alert, and the check’s log says
На сервере коммиты, которых нет на GitHub(commits on the server that GitHub doesn’t have)Deploy without SSH git pushwent through, and 20 minutes later nothing changed on the serverDeploy without SSH- The alert keeps repeating, and the log says
<commit> не поднялся, ждём следующего коммита(didn’t come up, waiting for the next commit)Deploy without SSH - The Dependency Dashboard doesn’t list the images from the compose filesDeploy without SSH
- Monday has passed, and there are no pull requests with new digestsDeploy without SSH
git add --dry-runlists.env,acme.jsonor a database fileMoving to a new server- A new stack doesn’t show up in
git add --dry-runMoving to a new server ssh -T github-infraanswersPermission denied (publickey)Moving to a new servergit pushsaysERROR: The key you are authenticating with has been marked as read onlyMoving to a new servergit clonesaysdestination path '/opt' already exists and is not an empty directoryMoving to a new serverbootstrap.shstopped withНет /opt/host/authorized_keysMoving to a new server- After the script,
ssh [email protected]givesPermission denied (publickey)Moving to a new server curl … install.crowdsec.netorapt-get updatefails and the script stopsMoving to a new serverrestore.shsays there are no snapshots for/opt/…Moving to a new server- Traefik on the new server serves the
TRAEFIK DEFAULT CERTcertificateMoving to a new server sudo docker exec crowdsec cscli bouncers listshowsfirewall-bouncerwith noLast API pullMoving to a new server- After the DNS change, some requests still reach the old serverMoving to a new server
fatal: remote error: … is not a valid repository nameServer configs in git: /opt as the repositorygit pullrefuses: the file already existsServer configs in git: /opt as the repository- After removing a stack,
git statusshows deleted filesServer configs in git: /opt as the repository git pullconflicts over a generated fileServer configs in git: /opt as the repository- On Windows a symlink turns into a text file with a pathServer configs in git: /opt as the repository
- A nested git clone inside
/optServer configs in git: /opt as the repository - A migration script left copies of
.envin/tmpServer configs in git: /opt as the repository - Old stacks kept “for reference” in
/opt(_old,_backup) with databases and keysServer configs in git: /opt as the repository - After
git pullthe container doesn’t see the changesServer configs in git: /opt as the repository - A secret is already in git historyServer configs in git: /opt as the repository
LogsHomelab
- After a vector update,
split/*.loghas duplicates and CrowdSec bans for old eventsTraefik logs: CrowdSec, vector, Loki, rotation - The splitter “works”, but the files are emptyTraefik logs: CrowdSec, vector, Loki, rotation
- Some
traefik.loglines go missing, files get mixed upTraefik logs: CrowdSec, vector, Loki, rotation access.loggrows foreverTraefik logs: CrowdSec, vector, Loki, rotation- A
*.logglob in a CrowdSec source also picks uptraefik.logTraefik logs: CrowdSec, vector, Loki, rotation - CrowdSec stopped seeing events after rotationTraefik logs: CrowdSec, vector, Loki, rotation
- An attack disguised as a monitoring agent went unnoticedTraefik logs: CrowdSec, vector, Loki, rotation
MonitoringHomelab
curlwent through, but the phone stays silentMetrics, uptime checks and alerts- Beszel rejects the password from
.envMetrics, uptime checks and alerts web1is red in the Beszel tableMetrics, uptime checks and alerts- “Test URL” in Beszel shows an errorMetrics, uptime checks and alerts
gatuskeeps restarting,sudo docker logs gatusshows a config parsing errorMetrics, uptime checks and alertsmemosis red on the Gatus page, but no alert arrivesMetrics, uptime checks and alerts- The
web1check in Healthchecks.io staysnewMetrics, uptime checks and alerts - A
downalert arrives at night, around 4:00Metrics, uptime checks and alerts - memos has been down for 5 minutes, and no alertMetrics, uptime checks and alerts
BackupsHomelab
apt updatesaysNO_PUBKEYoris not signedaboutpackages.kopia.ioA backup that outlives the server- Option A:
kopia repository create sftpsaysunable to authenticateA backup that outlives the server - Option A: the error
knownhosts: key is unknownA backup that outlives the server - Option B:
kopia repository create rclonesaysexecutable file not foundA backup that outlives the server - Option B: backups are slow, the log shows
rateLimitExceededA backup that outlives the server sqlite3saysunable to open databaseA backup that outlives the serversudo kopia snapshot createworks by hand, but from the timer it fails saying the repository isn’t connectedA backup that outlives the server- The
web1-backupcheck in Healthchecks.io staysnewA backup that outlives the server - After the restore memos says
database disk image is malformedor shows the wrong memosA backup that outlives the server - On a new server
kopia snapshot restore /optsays there are no snapshotsA backup that outlives the server
Web serversHomelab
- After
apt install nginxit won’t start;journalctl -u nginxsaysbind() to 0.0.0.0:80 failed (98: Address already in use)Web servers since 2008: Apache, nginx, Caddy and what to pick today 502 Bad Gatewayon PHP pages;/var/log/nginx/error.logsaysconnect() to unix:/run/php/php8.3-fpm.sock failed (2: No such file or directory)Web servers since 2008: Apache, nginx, Caddy and what to pick today- Files over a megabyte won’t upload:
413 Request Entity Too LargeWeb servers since 2008: Apache, nginx, Caddy and what to pick today - The server’s IP or someone else’s domain opens one of your sitesWeb servers since 2008: Apache, nginx, Caddy and what to pick today
- After moving to Apache 2.4 the whole site returns
403 Forbidden; the log saysAH01630: client denied by server configurationWeb servers since 2008: Apache, nginx, Caddy and what to pick today - WordPress on Apache: the home page opens, every other page is
404Web servers since 2008: Apache, nginx, Caddy and what to pick today - The app behind nginx gets the wrong path:
/api/usersarrives as/usersor//usersWeb servers since 2008: Apache, nginx, Caddy and what to pick today - A WebSocket through nginx won’t connect, or drops after exactly a minuteWeb servers since 2008: Apache, nginx, Caddy and what to pick today
- Caddy is running, but the site shows a certificate error or doesn’t open at allWeb servers since 2008: Apache, nginx, Caddy and what to pick today
MikroTikHomelab
- Connected it as the guide says, but there’s no internet, and the
ether1light is onMikroTik with no networking experience: what works right away and what you’ll have to learn - It won’t accept
adminwith no password, as old guides sayMikroTik with no networking experience: what works right away and what you’ll have to learn - Changed something, everything broke, and nobody remembers how it wasMikroTik with no networking experience: what works right away and what you’ll have to learn
- Bought a hAP ax lite, and there’s no 5 GHz networkMikroTik hardware for home: product lines, internals and which model to buy in 2026
- The ISP gives 2 Gbit/s, but speed tops out at 940 Mbit/sMikroTik hardware for home: product lines, internals and which model to buy in 2026
- Powered a camera from a hAP PoE port, and it won’t turn onMikroTik hardware for home: product lines, internals and which model to buy in 2026
- A 10G copper SFP+ module (
S+RJ10) in the RB5009 overheats and drops outMikroTik hardware for home: product lines, internals and which model to buy in 2026 /container addanswersnot allowed by device-modeRouterOS 7: how the MikroTik router system works- Put the
containerpackage in Files, rebooted — no packageRouterOS 7: how the MikroTik router system works - A command from an article fails:
bad command name wirelessorexpected end of commandRouterOS 7: how the MikroTik router system works - A scheduled script doesn’t run, and the log is silentRouterOS 7: how the MikroTik router system works
- Winbox doesn’t see the router in the Neighbors tabRouterOS 7: how the MikroTik router system works
- Copying files from the NAS to a wired computer is slow, and the router’s CPU is busy meanwhileWired networking on MikroTik: bridge, switch chip, fasttrack and VLANs
- Turned on
vlan-filtering=yes— and the router vanishedWired networking on MikroTik: bridge, switch chip, fasttrack and VLANs - A 2.5G port comes up at 1 Gbit/s, or even 100 Mbit/sWired networking on MikroTik: bridge, switch chip, fasttrack and VLANs
- The main ISP went down, and the backup route didn’t kick inWired networking on MikroTik: bridge, switch chip, fasttrack and VLANs
- The 5 GHz network disappears for a minute, or even 10, and comes back on another channelWi-Fi on MikroTik: the wifi package, CAPsMAN and roaming — where it’s good and where it falls short
- Next to the router, 5 GHz gives 200 Mbit/s instead of 600–900Wi-Fi on MikroTik: the wifi package, CAPsMAN and roaming — where it’s good and where it falls short
- An old laptop or a smart plug can’t see the network after the switch to WPA3Wi-Fi on MikroTik: the wifi package, CAPsMAN and roaming — where it’s good and where it falls short
- After moving to CAPsMAN, the access point won’t connect to the routerWi-Fi on MikroTik: the wifi package, CAPsMAN and roaming — where it’s good and where it falls short
- The phone clings to a far access point though another is right thereWi-Fi on MikroTik: the wifi package, CAPsMAN and roaming — where it’s good and where it falls short
- PPPoE is up, the router itself pings
1.1.1.1, but the computers have no internetMikroTik at home: what it is, why people like it, and who it suits - A 1 Gbit/s plan, but through MikroTik only 300–400 Mbit/s with the CPU at 100%MikroTik at home: what it is, why people like it, and who it suits
- The 5 GHz network shows up a minute, or even 10 minutes, after a rebootMikroTik at home: what it is, why people like it, and who it suits
- The router is sluggish, the CPU busy, and
/tool torchonether1shows incoming UDP to port 53MikroTik at home: what it is, why people like it, and who it suits - Changed one bridge or firewall setting — and lost access to the routerMikroTik at home: what it is, why people like it, and who it suits
- Updated RouterOS, but
/system routerboard printshows different versionsMikroTik at home: what it is, why people like it, and who it suits http://192.168.88.1won’t openFirst MikroTik setup: internet, your own user, updates, Wi-Fi and a backup- PPPoE won’t come up; the log says
authentication failedFirst MikroTik setup: internet, your own user, updates, Wi-Fi and a backup - PPPoE is up, the router pings
1.1.1.1, but the computers have no internetFirst MikroTik setup: internet, your own user, updates, Wi-Fi and a backup - After
installthe router hasn’t answered for several minutesFirst MikroTik setup: internet, your own user, updates, Wi-Fi and a backup - After
/user disable adminnobody can log inFirst MikroTik setup: internet, your own user, updates, Wi-Fi and a backup - Added a port forward rule — the port is closed from outsideThe MikroTik default firewall line by line: what each rule does
- Added an
acceptfor a port on the router, and it’s still closedThe MikroTik default firewall line by line: what each rule does - The router is visible from outside: Winbox, DNS or the web UI answersThe MikroTik default firewall line by line: what each rule does
- The “drop invalid” counter grows by thousands per minuteThe MikroTik default firewall line by line: what each rule does
- The phone joins the guest network and hangs at “Obtaining IP address”A guest Wi-Fi network on MikroTik: a separate bridge, its own DHCP and three firewall rules
- An address is assigned, sites don’t open by name, but
1.1.1.1pingsA guest Wi-Fi network on MikroTik: a separate bridge, its own DHCP and three firewall rules - The NAS and the router’s web UI open from the guest networkA guest Wi-Fi network on MikroTik: a separate bridge, its own DHCP and three firewall rules
- Limited guests’ speed with a Simple Queue, and they download at full speedA guest Wi-Fi network on MikroTik: a separate bridge, its own DHCP and three firewall rules
- The guest network exists only on 2.4 GHz or only on 5A guest Wi-Fi network on MikroTik: a separate bridge, its own DHCP and three firewall rules
- The internet got slow, the router CPU is at 100%, and
torchonether1shows thousands of requests to UDP 53allow-remote-requests — the router’s DNS is for your home, not the internet - Restored a
.backuponto a new router — an address conflict on the network, the second router vanishedBack up MikroTik with a text /export; .backup is only for the same router /importon another model stops halfwayBack up MikroTik with a text /export; .backup is only for the same router- The export has no Wi-Fi or PPPoE passwordsBack up MikroTik with a text /export; .backup is only for the same router
- Turned on
vlan-filtering, the connection dropped and didn’t come back — with Safe Mode onEdit the MikroTik firewall and bridge in Safe Mode <SAFE>vanished from the prompt though you never left the modeEdit the MikroTik firewall and bridge in Safe Mode- A guest queue exists, its counter barely grows, guests download at full speedFasttrack bypasses queues: rate limits only work without it
- Turned fasttrack off entirely — internet speed dropped, the CPU is at 100%Fasttrack bypasses queues: rate limits only work without it
- Clicked Apply in Quick Set, and the guest network or a port forward disappearedQuick Set or manual MikroTik setup — pick one
- The log shows
login failure for user admin from 203.0.113.50 via sshevery minuteRouter Winbox, WebFig and SSH — from the home network only - An unfamiliar user, script or scheduler entry appeared in the configRouter Winbox, WebFig and SSH — from the home network only
- Ran
/system routerboard upgrade, butcurrent-firmwarehasn’t changedUpdated RouterOS? Update the RouterBOOT bootloader too - After an update the router logs that the bootloader is older than the systemUpdated RouterOS? Update the RouterBOOT bootloader too
ProxmoxHomelab
- A guide from the internet doesn’t work: no
vzctl, no/etc/vz, commands about OpenVZWhere Proxmox came from: virtualization before it and the road from 0.9 to 9.2 - The guide has a
sources.listwithbookworm, but we have.sourcesfiles andtrixieWhere Proxmox came from: virtualization before it and the road from 0.9 to 9.2 - There’s no ESXi import wizard in the Storage › Add menuWhere Proxmox came from: virtualization before it and the road from 0.9 to 9.2
- A VM doesn’t start:
KVM virtualisation configured, but not availableHardware for Proxmox: CPU, memory, disks and network cards - Rocky Linux 10 or RHEL 10 in a VM crashes right after the boot loader:
Fatal glibc error: CPU does not support x86-64-v3Hardware for Proxmox: CPU, memory, disks and network cards - The host network drops for a minute, and
journalctl -kshowse1000e … Detected Hardware Unit HangHardware for Proxmox: CPU, memory, disks and network cards - VMs are slow on writes,
iowaitis high, though the SSD is newHardware for Proxmox: CPU, memory, disks and network cards - The SSD’s Wearout in Disks grows by percents a monthHardware for Proxmox: CPU, memory, disks and network cards
- Docker in an LXC container:
open sysctl net.ipv4.ip_unprivileged_port_start file: reopen fd 8: permission deniedVirtual machine or LXC container in Proxmox: which service goes where - In a host directory mounted into a container, files belong to
nobody, and writes fail withPermission deniedVirtual machine or LXC container in Proxmox: which service goes where - Migrating a container to another node stops itVirtual machine or LXC container in Proxmox: which service goes where
- A Debian container doesn’t start after a host update:
systemdin the guest crashes or hangsVirtual machine or LXC container in Proxmox: which service goes where - VMs froze all at once or report I/O errors, and
lvsshowsData%100.00 ondataProxmox storage: local, local-lvm, ZFS — what lives where and what it can do - Snapshot rollback on ZFS:
can't rollback, 'before-upgrade' is not most recent snapshotProxmox storage: local, local-lvm, ZFS — what lives where and what it can do - Can’t upload an ISO to
local-lvm— there’s no ISO Images tabProxmox storage: local, local-lvm, ZFS — what lives where and what it can do - The UI throws errors, and
df -h /shows 100 %Proxmox storage: local, local-lvm, ZFS — what lives where and what it can do - A ZFS host has almost all memory in use, though guests were given less than halfProxmox storage: local, local-lvm, ZFS — what lives where and what it can do
- Changed the network in the UI — nothing changedNetworking in Proxmox: the vmbr0 bridge, VLANs, a private network behind NAT and the firewall
- After a kernel update the host dropped off the networkNetworking in Proxmox: the vmbr0 bridge, VLANs, a private network behind NAT and the firewall
- A laptop host on Wi-Fi works, but guests on the bridge have no networkNetworking in Proxmox: the vmbr0 bridge, VLANs, a private network behind NAT and the firewall
- Rented server: a guest was added to the bridge with a public address — the provider shut the port or sent a complaintNetworking in Proxmox: the vmbr0 bridge, VLANs, a private network behind NAT and the firewall
- A VM with a VLAN tag doesn’t get an addressNetworking in Proxmox: the vmbr0 bridge, VLANs, a private network behind NAT and the firewall
- Turned the firewall on at Datacenter — lost access to the UINetworking in Proxmox: the vmbr0 bridge, VLANs, a private network behind NAT and the firewall
apt updatecomplains:401 Unauthorizedfromenterprise.proxmox.comProxmox VE: what it is, who makes it and who needs it at home- A
No valid subscriptionpop-up at every loginProxmox VE: what it is, who makes it and who needs it at home - A VM doesn’t start:
KVM virtualisation configured, but not availableProxmox VE: what it is, who makes it and who needs it at home http://address:8006doesn’t openProxmox VE: what it is, who makes it and who needs it at home- Almost all host memory is in use, though VMs were given less than halfProxmox VE: what it is, who makes it and who needs it at home
- After a kernel update the host dropped off the networkProxmox VE: what it is, who makes it and who needs it at home
- A black screen after picking the installer entryInstalling Proxmox VE 9: USB stick, installer, no-subscription repository and the first update
- The USB stick doesn’t boot, or the installer can’t find the installation mediumInstalling Proxmox VE 9: USB stick, installer, no-subscription repository and the first update
- The installer warns that there’s no KVM hardware supportInstalling Proxmox VE 9: USB stick, installer, no-subscription repository and the first update
- The install finished, but
https://10.0.0.2:8006doesn’t openInstalling Proxmox VE 9: USB stick, installer, no-subscription repository and the first update apt update:401 Unauthorizedfromenterprise.proxmox.comInstalling Proxmox VE 9: USB stick, installer, no-subscription repository and the first update- Emails about backups and updates don’t arriveInstalling Proxmox VE 9: USB stick, installer, no-subscription repository and the first update
qm createfails:storage 'local-lvm' does not existAn Ubuntu template with cloud-init in Proxmox: a VM with a key and an address in a minute- The VM console in noVNC is empty or says
starting serial terminalAn Ubuntu template with cloud-init in Proxmox: a VM with a key and an address in a minute ssh [email protected]—Permission denied (publickey)An Ubuntu template with cloud-init in Proxmox: a VM with a key and an address in a minute- Two clones get the same address from the routerAn Ubuntu template with cloud-init in Proxmox: a VM with a key and an address in a minute
- The disk was enlarged, but
df -h /shows the old sizeAn Ubuntu template with cloud-init in Proxmox: a VM with a key and an address in a minute - The VM pings
1.1.1.1, but names don’t resolveAn Ubuntu template with cloud-init in Proxmox: a VM with a key and an address in a minute pveam download—400 Parameter verification failed, or the template isn’t thereThe first LXC container in Proxmox: Debian from a template, network, SSH by key and a directory shared with the hostssh [email protected]with a password —Permission deniedThe first LXC container in Proxmox: Debian from a template, network, SSH by key and a directory shared with the host- After login:
perl: warning: Setting locale failedorsetlocale: LC_ALL: cannot change localeThe first LXC container in Proxmox: Debian from a template, network, SSH by key and a directory shared with the host - In the mounted directory files belong to
nobody, and writes fail withPermission deniedThe first LXC container in Proxmox: Debian from a template, network, SSH by key and a directory shared with the host - The container backup doesn’t include the files from
/srv/shareThe first LXC container in Proxmox: Debian from a template, network, SSH by key and a directory shared with the host apt updatein the container:Temporary failure resolving 'deb.debian.org'The first LXC container in Proxmox: Debian from a template, network, SSH by key and a directory shared with the host- The job fails:
storage 'nas-backup' is not onlineBackups in Proxmox: a nightly vzdump job, retention and a restore check - The container log says
mode failure - some volumes do not support snapshots, thentrying 'suspend' mode insteadBackups in Proxmox: a nightly vzdump job, retention and a restore check - During the backup the VM is sluggish, and the database in it takes seconds to answerBackups in Proxmox: a nightly vzdump job, retention and a restore check
- Old backups aren’t deleted, and the NAS runs out of spaceBackups in Proxmox: a nightly vzdump job, retention and a restore check
- The VM can’t be started or changed:
VM 101 is locked (backup)Backups in Proxmox: a nightly vzdump job, retention and a restore check - After a restore the VM causes an address conflict with the originalBackups in Proxmox: a nightly vzdump job, retention and a restore check
- Changed the memory or CPU of a running VM — inside everything is as beforeA Proxmox virtual machine config line by line: /etc/pve/qemu-server/111.conf
- A manual edit of the file disappeared, or the VM reports
parse errorA Proxmox virtual machine config line by line: /etc/pve/qemu-server/111.conf - The VM can’t be started:
VM is locked (snapshot)A Proxmox virtual machine config line by line: /etc/pve/qemu-server/111.conf - Linux in the VM says
Fatal glibc error: CPU does not support x86-64-v3A Proxmox virtual machine config line by line: /etc/pve/qemu-server/111.conf - Discard is on,
fstrim -avran, butData%inlvsdidn’t go downDiscard on a VM disk — otherwise Proxmox’s thin pool only grows - Docker went onto the host — VMs and containers lost their networkDocker on Proxmox goes in a VM, not on the host or in LXC
- After an address change the UI doesn’t open, and
systemctl status pve-clustersays failedPick the Proxmox node name and address during install: they can be changed later, but in several places - After a name change the UI shows no VMs, and a node with the old name hangs around greyed outPick the Proxmox node name and address during install: they can be changed later, but in several places
- After editing
/etc/default/pveproxythe UI doesn’t open even through the tunnelProxmox host port 8006 and SSH — from your own network only qm shutdown 101fails:QEMU guest agent is not runningqemu-guest-agent in every Proxmox VM, plus the Guest Agent option on the VM- The backup log says:
skipping guest-agent 'fs-freeze', agent configured but not running?qemu-guest-agent in every Proxmox VM, plus the Guest Agent option on the VM apt update:The repository … ceph-squid trixie InRelease is not signedor401fromenterprise.proxmox.com/debian/ceph-squidWithout a subscription: the pve-no-subscription repository, both enterprise ones off- After a file edit
apt updatecomplains:Malformed entry … in sources fileWithout a subscription: the pve-no-subscription repository, both enterprise ones off
Language modelsAI
- RAG answers “the documents don’t mention it”, though they doYour own data in a model: context, RAG or fine-tuning
- The answer is assembled from chunks of different versions of a documentYour own data in a model: context, RAG or fine-tuning
- After fine-tuning, the model still mixes up your factsYour own data in a model: context, RAG or fine-tuning
- The whole archive in the context — and every question costs as much as a bookYour own data in a model: context, RAG or fine-tuning
- The model confidently named a flag or option that doesn’t existLanguage models: how they work, what kinds exist and how they differ
- The answer breaks off mid-wordLanguage models: how they work, what kinds exist and how they differ
- Toward the end of a long chat the model forgets what was agreedLanguage models: how they work, what kinds exist and how they differ
- It suggests syntax from an old version of a programLanguage models: how they work, what kinds exist and how they differ
- It agrees with any edit you make, even a wrong oneLanguage models: how they work, what kinds exist and how they differ
- The same question gets different answersLanguage models: how they work, what kinds exist and how they differ
- The model cited an article, issue or document that doesn’t existWhere models fail: the miss, the cause, the fix
- The code calls a library function that the library doesn’t haveWhere models fail: the miss, the cause, the fix
- The agent reported “tests pass”, but they never ranWhere models fail: the miss, the cause, the fix
PromptsAI
- The model brought back an option you dropped an hour agoLong conversations: how not to hit the context limit and not lose what you agreed on
- It edits an old version of a file, not the one on disk nowLong conversations: how not to hit the context limit and not lose what you agreed on
- Mid-task the agent “forgot” a rule from the start of the sessionLong conversations: how not to hit the context limit and not lose what you agreed on
- Answers got slower, and the subscription limit runs out in an hourLong conversations: how not to hit the context limit and not lose what you agreed on
- The answer covers everything at once, and what you need is one paragraph in tenA prompt, block by block: context, materials, task, format, examples
- The model mixes up your question and the pasted textA prompt, block by block: context, materials, task, format, examples
- After “NEVER change ports” the model refuses to fix even a typo in a portA prompt, block by block: context, materials, task, format, examples
Tokens and costAI
- The monthly bill is several times what you “roughly estimated”Tokens: measuring text size and the price of a request
- The
tiktokencount doesn’t match Claude’susageTokens: measuring text size and the price of a request - Switched to a new model — more tokens on the same promptsTokens: measuring text size and the price of a request
- The model can’t count the letters in a word or mixes up digits in a long numberTokens: measuring text size and the price of a request
Model APIsAI
401andauthentication_error—invalid x-api-keyFirst API request: a key that stays out of history, curl, the response and the bill400with a message about a low credit balanceFirst API request: a key that stays out of history, curl, the response and the bill429andrate_limit_errorFirst API request: a key that stays out of history, curl, the response and the bill529andoverloaded_errorFirst API request: a key that stays out of history, curl, the response and the billjqprintsnullinstead of the answer textFirst API request: a key that stays out of history, curl, the response and the bill400mentioningtemperatureafter switching to a new modelA model request, block by block: what goes out every time400on a request that ends with anassistantmessageA model request, block by block: what goes out every time400withthinking: {type: "disabled"}on Opus 5.5A model request, block by block: what goes out every time- The model “doesn’t see” the previous answerA model request, block by block: what goes out every time
Local modelsAI
- It answers at one word per secondA model on your own hardware: Ollama, an API on port 11434 and a web UI
- A long document — the model “forgot” the beginningA model on your own hardware: Ollama, an API on port 11434 and a web UI
- The container doesn’t see the GPU, everything runs on the CPUA model on your own hardware: Ollama, an API on port 11434 and a web UI
- Port 11434 is closed in ufw but answers from outsideA model on your own hardware: Ollama, an API on port 11434 and a web UI
AI agentsAI
- The agent did something other than asked, with edits scattered across a dozen filesAgents and tools: how a model reads files and runs commands
- After connecting several MCP servers, answers got worse and pricierAgents and tools: how a model reads files and runs commands
- The agent ran a command from someone else’s READMEAgents and tools: how a model reads files and runs commands
- The agent says it called a tool, but nothing happenedAgents and tools: how a model reads files and runs commands
- The agent took a comment in a config for a factRunning a server with an AI agent: roles, checks, secrets
- The agent drew a conclusion from a file nameRunning a server with an AI agent: roles, checks, secrets
- The agent “fixed” something that wasn’t brokenRunning a server with an AI agent: roles, checks, secrets
- The agent took a closed chat for a refusalRunning a server with an AI agent: roles, checks, secrets
- A typo in a command gave a cryptic errorRunning a server with an AI agent: roles, checks, secrets
- The person pasted a command with a password into the chatRunning a server with an AI agent: roles, checks, secrets
- “Which of these have I already done?”Running a server with an AI agent: roles, checks, secrets
- A command ran on the wrong serverRunning a server with an AI agent: roles, checks, secrets
GitDevelopment
fatal: not a git repository (or any of the parent directories): .gitGit: what it is and how it stores historyAuthor identity unknown *** Please tell me who you are.Git: what it is and how it stores history! [rejected] main -> main (fetch first)Git: what it is and how it stores historyfatal: Need to specify how to reconcile divergent branches.Git: what it is and how it stores historyYou are in 'detached HEAD' stateGit: what it is and how it stores history[email protected]: Permission denied (publickey).Git: what it is and how it stores historyerror: src refspec main does not match anyGit from scratch: the first repository and a push to GitHub over SSHfatal: refusing to merge unrelated historiesGit from scratch: the first repository and a push to GitHub over SSHremote: Support for password authentication was removed on August 13, 2021.Git from scratch: the first repository and a push to GitHub over SSHerror: remote origin already exists.Git from scratch: the first repository and a push to GitHub over SSHnothing added to commit but untracked files presentGit from scratch: the first repository and a push to GitHub over SSH- A setting in
~/.gitconfighas no effect in one repository~/.gitconfig block by block: what to set once - On Windows
git config --globalwrites somewhere other than the file you have open~/.gitconfig block by block: what to set once includeIfdoesn’t pick up the work settings~/.gitconfig block by block: what to set onceerror: cannot rebase: You have unstaged changes. error: Please commit or stash them.Branches in git: merge, rebase and conflicts step by stepCONFLICT (content): Merge conflict in app.ymlBranches in git: merge, rebase and conflicts step by step- During a rebase
--ourstook the wrong versionBranches in git: merge, rebase and conflicts step by step error: the branch 'feature' is not fully mergedBranches in git: merge, rebase and conflicts step by step- After a rebase
git pushis rejected:! [rejected] … (non-fast-forward)Branches in git: merge, rebase and conflicts step by step git bisectheads the wrong way, and the commit it finds is innocentFind where it broke: log, blame and bisectbisect runmarked everythingbad, though old commits definitely workFind where it broke: log, blame and bisect- A commit doesn’t build at all — neither good nor badFind where it broke: log, blame and bisect
git blamepoints to a commit that only reformatted the fileFind where it broke: log, blame and bisect- After
bisectthe project is in an odd state, andgit statussaysHEAD detachedFind where it broke: log, blame and bisect ERROR: The key you are authenticating with has been marked as read only.Git and SSH keys: deploy keys, two accounts, signed commits- GitHub won’t accept a key:
Key is already in useGit and SSH keys: deploy keys, two accounts, signed commits - The push goes as the wrong account:
Permission to work-org/app.git denied to example-userGit and SSH keys: deploy keys, two accounts, signed commits - Commits on GitHub show someone else’s name or no avatarGit and SSH keys: deploy keys, two accounts, signed commits
error: gpg failed to sign the dataGit and SSH keys: deploy keys, two accounts, signed commits- Signed commits show “Unverified” on GitHubGit and SSH keys: deploy keys, two accounts, signed commits
/bin/bash^M: bad interpreter: No such file or directorySet line endings in .gitattributes, not core.autocrlf$'\r': command not foundon every line of a scriptSet line endings in .gitattributes, not core.autocrlfgit statusshows every file as modified, though nobody touched themSet line endings in .gitattributes, not core.autocrlf
ObsidianDevelopment
- You renamed a note in File Explorer, and links to it now lead to a new empty noteObsidian: what it is and how it stores notes
- You renamed a note in Obsidian, but links in other notes kept the old nameObsidian: what it is and how it stores notes
- Files like
Pasted image 20261024103512.pngpile up in the vault rootObsidian: what it is and how it stores notes - On GitHub,
[[…]]links show up as plain textObsidian: what it is and how it stores notes - The community plugin catalog won’t open, a plugin won’t installObsidian: what it is and how it stores notes
- You overwrote part of a note or deleted it entirelyObsidian: what it is and how it stores notes
PythonDevelopment
- You type
python, and the Microsoft Store opensPython: where it came from, why it became the language of AI, and where to start Command 'python' not found, did you mean: command 'python3'Python: where it came from, why it became the language of AI, and where to starterror: externally-managed-environmentonpip installPython: where it came from, why it became the language of AI, and where to startModuleNotFoundError: No module named '…'Python: where it came from, why it became the language of AI, and where to startSyntaxError: invalid syntaxin response topython frame.pyRunning Python: the interactive mode and your own .py filecan't open file '…\frame.py': [Errno 2] No such file or directoryRunning Python: the interactive mode and your own .py fileSyntaxError: Missing parentheses in call to 'print'. Did you mean print(...)?Running Python: the interactive mode and your own .py file- You type
python, and the Microsoft Store opensRunning Python: the interactive mode and your own .py file NameError: name 'speed' is not definedVariables and types: a name, a value, and what you can do with itTypeError: can only concatenate str (not "int") to strVariables and types: a name, a value, and what you can do with itTypeError: unsupported operand type(s) for /: 'str' and 'int'Variables and types: a name, a value, and what you can do with itValueError: could not convert string to float: '4,7'Variables and types: a name, a value, and what you can do with itTypeError: 'int' object is not callableVariables and types: a name, a value, and what you can do with it0.1 + 0.2 == 0.3givesFalseNumbers in Python: integers without a limit, fractions with an errorround(2.5)gives2, andround(3.5)gives4Numbers in Python: integers without a limit, fractions with an errorZeroDivisionError: division by zeroNumbers in Python: integers without a limit, fractions with an errorOverflowError: (34, 'Result too large')Numbers in Python: integers without a limit, fractions with an errorValueError: Exceeds the limit (4300 digits) for integer string conversionNumbers in Python: integers without a limit, fractions with an errorIndexError: string index out of rangeStrings in Python: slices, methods and f-stringsTypeError: 'str' object does not support item assignmentStrings in Python: slices, methods and f-strings- Called
name.lower(), and the name didn’t changeStrings in Python: slices, methods and f-strings - The path
"D:\new\table"prints with a line break and a tabStrings in Python: slices, methods and f-strings - It prints
{x}instead of the valueStrings in Python: slices, methods and f-strings SyntaxError: invalid syntax. Maybe you meant '==' or ':=' instead of '='?Conditions in Python: if, elif and what counts as falseSyntaxError: expected ':'Conditions in Python: if, elif and what counts as falseIndentationError: expected an indented block after 'if' statement on line 2Conditions in Python: if, elif and what counts as false- The condition fires no matter what’s in
xConditions in Python: if, elif and what counts as false - Typed
5, andif answer == 5:doesn’t fireConditions in Python: if, elif and what counts as false - The script never ends, and the terminal doesn’t respondLoops in Python: for over anything, while until a condition, break
range(1, 10)didn’t reach 10Loops in Python: for over anything, while until a condition, break- The total prints on every step instead of onceLoops in Python: for over anything, while until a condition, break
TypeError: 'int' object is not iterableLoops in Python: for over anything, while until a condition, breakIndentationError: expected an indented block after 'while' statement on line 2Loops in Python: for over anything, while until a condition, breakNameError: name 'human_size' is not definedFunctions in Python: def, arguments and returnTypeError: human_size() missing 1 required positional argument: 'n'Functions in Python: def, arguments and returnTypeError: human_size() takes 1 positional argument but 2 were givenFunctions in Python: def, arguments and returnTypeError: unsupported operand type(s) for +: 'NoneType' and 'int'Functions in Python: def, arguments and returnUnboundLocalError: cannot access local variable 'count' where it is not associated with a valueFunctions in Python: def, arguments and returnpython: can't open file '…\password.py': [Errno 2] No such file or directoryA 120-bit password from your own script: the first .py file and the secrets moduleNameError: name 'secrets' is not defined. Did you forget to import 'secrets'?A 120-bit password from your own script: the first .py file and the secrets moduleIndentationError: expected an indented block after function definitionA 120-bit password from your own script: the first .py file and the secrets moduleAttributeError: module 'secrets' has no attribute 'token_urlsafe' (consider renaming '…\secrets.py' …)A 120-bit password from your own script: the first .py file and the secrets moduleModuleNotFoundError: No module named 'qrcode'A QR code for guest Wi-Fi: your first pip package in a virtual environmentModuleNotFoundError: No module named 'PIL'on themake_imagelineA QR code for guest Wi-Fi: your first pip package in a virtual environmentActivate.ps1 cannot be loaded because running scripts is disabled on this systemA QR code for guest Wi-Fi: your first pip package in a virtual environmenterror: externally-managed-environmentonpip installon UbuntuA QR code for guest Wi-Fi: your first pip package in a virtual environment- The camera sees the code but doesn’t join the networkA QR code for guest Wi-Fi: your first pip package in a virtual environment
403 Forbiddenfrom a site that opens in the browserAre the sites up: a list check and errors handled with try/exceptno connection: [SSL: CERTIFICATE_VERIFY_FAILED]Are the sites up: a list check and errors handled with try/except- The script hangs on one URLAre the sites up: a list check and errors handled with try/except
- Two photos that look the same didn’t show up as duplicatesDuplicate photos by content: a hash and a dictionary as memory
AttributeError: module 'hashlib' has no attribute 'file_digest'Duplicate photos by content: a hash and a dictionary as memory- The script is silent for a long time on a video folderDuplicate photos by content: a hash and a dictionary as memory
- You type
python, and the Microsoft Store opensPython from scratch: install it and send files to your phone over Wi-Fi Command 'python' not found, did you mean: command 'python3'Python from scratch: install it and send files to your phone over Wi-Fi- The phone loads for a long time and says the page can’t be openedPython from scratch: install it and send files to your phone over Wi-Fi
- The address is typed in, but a foreign page or nothing opensPython from scratch: install it and send files to your phone over Wi-Fi
- Started it in another folder, but the phone shows the old onePython from scratch: install it and send files to your phone over Wi-Fi
- Tapped a
.mdor.txt, and instead of a download you get text with garbled charactersPython from scratch: install it and send files to your phone over Wi-Fi FileNotFoundError: [WinError 3] The system cannot find the path specified: 'C:\Users\…\Downloads'Sort Downloads into folders: a script with a dry run- A
.JPGfile stayed put, though.jpgis in the dictionarySort Downloads into folders: a script with a dry run PermissionError: [WinError 32] The process cannot access the file because it is being used by another processSort Downloads into folders: a script with a dry run- The dry run prints “keeping” for almost everythingSort Downloads into folders: a script with a dry run
- The script thinks for several minutesWhat’s taking up disk space: the biggest files in one script
- The “total” is less than the folder’s Properties or the disk’s used spaceWhat’s taking up disk space: the biggest files in one script
Presenting informationDesign
- The answer is on the page, but people still askPresenting information: how people read a screen and what follows from it
- People scroll to the end of the page and leavePresenting information: how people read a screen and what follows from it
- Lots is highlighted, and the main thing is invisiblePresenting information: how people read a screen and what follows from it
- The text was broken into bullets, and it didn’t get clearerPresenting information: how people read a screen and what follows from it
- Neat on a computer, a mess on a phonePresenting information: how people read a screen and what follows from it
- Nobody notices the warningPresenting information: how people read a screen and what follows from it
- There are headings, but people still read straight through to find thingsScannable text: reworking a wall of paragraphs step by step
- An eight-item list reads worse than the paragraph didScannable text: reworking a wall of paragraphs step by step
- After the rework the text got twice as longScannable text: reworking a wall of paragraphs step by step
- A 15-item list gets scrolled past unreadChunking text: where to cut and how much goes in one chunk
- The list items are three sentences each, and it’s heavier than paragraphsChunking text: where to cut and how much goes in one chunk
- On a phone a paragraph fills the whole screenChunking text: where to cut and how much goes in one chunk
- A section heading seems to hang between two paragraphsWhite space in CSS: column width, line height and spacing line by line
- Set
max-width: 65ch, and the line came out longer than expectedWhite space in CSS: column width, line height and spacing line by line - The last line of a paragraph is a single short wordWhite space in CSS: column width, line height and spacing line by line
- At 200% zoom, text spills out of its boxesWhite space in CSS: column width, line height and spacing line by line
- The eye jumps around the page and stops on nothingVisual anchors: what the eye catches on, and when it stops
- Links and important words share one colorVisual anchors: what the eye catches on, and when it stops
- Items have icons, but people still confuse themVisual anchors: what the eye catches on, and when it stops
- The page looks “cheap”, and it’s unclear what exactly is wrongWhat spoils a layout: the sign, why it hurts, what to use instead
- Everything follows the rules, and the page is boringWhat spoils a layout: the sign, why it hurts, what to use instead
- The console’s heading list is empty, though the page shows headingsChecking presentation before publishing: six checks in the browser
- On a phone the page slides sidewaysChecking presentation before publishing: six checks in the browser
- In dark mode some images are white rectanglesChecking presentation before publishing: six checks in the browser
TypographyDesign
- On load the text appears in one font, then jumps to another, and paragraphs shiftTypography: typeface, size and the line on screen
- Basic Latin letters are in your font, and accented or non-Latin ones in another, Arial-like fontTypography: typeface, size and the line on screen
- Bold looks smeared, and italic looks like upright letters slantedTypography: typeface, size and the line on screen
- On an iPhone the page zooms in when you tap an input fieldTypography: typeface, size and the line on screen
- Figures in a table column don’t line upTypography: typeface, size and the line on screen
- A dash ended up at the start of a line, and “px” on a line without its numberTypography: typeface, size and the line on screen
- The font is set up, but the Network tab in DevTools shows no fileYour own font on a site: @font-face, subsets and loading without a jump
- The font file downloads twice, and the console warns about preloadYour own font on a site: @font-face, subsets and loading without a jump
- A font from a CDN won’t load, and the console shows a CORS errorYour own font on a site: @font-face, subsets and loading without a jump
- The fallback is tuned, but in Safari lines still shift vertically on the swapYour own font on a site: @font-face, subsets and loading without a jump
ColorDesign
- In dark mode, input fields and scrollbars stayed whiteColor: a palette by role, contrast and dark mode
- On load the page flashes white, then goes darkColor: a palette by role, contrast and dark mode
- People confuse the red and green lines on a chartColor: a palette by role, contrast and dark mode
- Palette shades with the same lightness in HSL look differently brightColor: a palette by role, contrast and dark mode
- After
filter: invert(1)photos became negatives and the orange accent turned cyanColor: a palette by role, contrast and dark mode - The theme switched, but some elements stayed lightA palette in CSS: role-based tokens for the light and dark themes
- Chose the light theme, and with a dark system the site is still darkA palette in CSS: role-based tokens for the light and dark themes
- Dark mode flashes light on every navigationA palette in CSS: role-based tokens for the light and dark themes
light-dark()always returns the light valueA palette in CSS: role-based tokens for the light and dark themes- Captions and dates can’t be seen on a phone outdoorsText contrast: the WCAG thresholds and the colors that fail them
- White text on a green button is hard to readText contrast: the WCAG thresholds and the colors that fail them
- In dark mode a gray that passed in the light theme can’t be readText contrast: the WCAG thresholds and the colors that fail them
No entry describes this symptom yet.