basforge

Search titles, text and the “If it doesn’t work” cases.

    RU
    OverviewLinuxSSH

    SSH: how logging in to a server works and what else it’s for

    SSH is an encrypted channel to a server on port 22: you use it for a shell, file copies, admin panels that are closed to the outside, and git pushes. Server and user prove who they are with key pairs — and almost every SSH failure comes down to which key sits where.

    What problem it solves

    The server sits in someone else’s data center with no monitor. Everything you do with it goes over the network: commands, files, git, access to panels. SSH is one protocol for all of it. It encrypts the channel, checks that the other end is the right server, and lets in only those who prove they belong.

    What doesn’t

    • websites: the web server serves them on 80 and 443
    • the provider’s panel and console — a separate path that works even when SSH is broken (Provider console)
    • Samba and NFS shares — separate protocols

    On the server, sshd from the openssh-server package accepts connections; on your computer, ssh from openssh-client connects. Windows 10 and 11 ship an OpenSSH client (C:\Windows\System32\OpenSSH\ssh.exe), and Git for Windows brings its own.

    How a login goes

    computer · ssh [email protected]

    1. TCP connection:22Port 22 must be open in the server’s firewall and at the provider. No answer — Connection timed out; an active refusal — Connection refused.
    2. Key exchangemlkem768x25519-sha256The two sides agree on a cipher and derive a shared session key. Since OpenSSH 10.0 the default algorithm is a hybrid that resists quantum computers.
    3. Server key~/.ssh/known_hostsThe server shows its key. The first time, the client asks yes/no; after that it compares with the recorded key and drops the connection on a mismatch.
    4. User login~/.ssh/authorized_keysThe client proves it holds the private key by signing session data. The server looks for the public key in the user’s authorized_keys. If it’s not there, it tries a password, if passwords are allowed.
    5. Channelshell · sftp · tunnelOne connection carries several channels: a shell, a copy, a port forward.

    server · shell of user example-user

    The key exchange happens before login, so a password, if allowed, also travels over the encrypted channel. The problem with passwords is different: they can be guessed. Bots try root with dictionary passwords, and an hour after a fresh server goes up, /var/log/auth.log has hundreds of Failed password lines.

    Two key pairs

    Half the confusion around SSH comes from there being two key pairs that do different jobs.

    Server key

    /etc/ssh/ssh_host_*_key
    • Generated when openssh-server is installed; every server has its own.
    • Proves to the client: “I am that server”.
    • The client records it in ~/.ssh/known_hosts. Reinstall the server and the key is new, so the client warns REMOTE HOST IDENTIFICATION HAS CHANGED (Server fingerprint).

    User key

    ~/.ssh/id_ed25519
    • You create it with ssh-keygen on your own computer. The private half never leaves it.
    • Proves to the server: “I am example-user”.
    • The public half, id_ed25519.pub, is a line in ~/.ssh/authorized_keys on the server. One line is one key, and a user can have any number of them.

    The public key isn’t a secret: show it, paste it in a chat, put it on GitHub. The private key is the secret, and a passphrase protects it. Without a passphrase the key is a file that opens the server to anyone who copies it.

    Where it’s used

    The main case

    Server shell

    With a key, no password, as your own user with sudo. Don’t let root in over SSH (sshd_config).

    Copies and backups

    rsync -a over SSH transfers only what changed. On the receiving side, the backup key gets one command: command="rrsync -ro /srv/backup" in authorized_keys. On Ubuntu 24.04 rrsync comes with the rsync package in /usr/bin, which is in PATH.

    Services closed to the outside

    A database admin panel, the Traefik dashboard, a monitoring panel listen only on the server’s 127.0.0.1. A tunnel, ssh -L 8080:127.0.0.1:8080, opens them on your computer (SSH tunnels).

    Automation

    CI deploys with a key; a script on one server pulls files from another. Such a key gets its own line with restrictions: from="…", command="…", no-pty.

    Other ways to solve it

    Ways to get into a server — from documentation at the time of writing.

    How you log in What the server needs How it differs
    OpenSSH with keys private key on your computer, public key in authorized_keys openssh-server, part of Ubuntu Server guessing doesn’t work; lose the key and you need a second one or the provider console
    OpenSSH with a password the user’s password the same, PasswordAuthentication yes nothing to set up; bots guess passwords around the clock
    OpenSSH certificates a key signed by your own CA, with an expiry TrustedUserCAKeys in sshd_config no editing authorized_keys on servers, access expires by itself; you need your own CA — ssh-keygen -s or step-ca
    Mosh login over SSH, then its own UDP protocol mosh, UDP ports 60000–61000 the session survives network changes and laptop sleep, typing has no lag; no tunnels or copying
    Provider console a web page in the hosting panel nothing works with no network on the server and with a broken sshd; slow, no copying — a fallback
    Teleport login through its proxy with short-lived certificates and SSO the Teleport agent session log and recording, role-based access; one more service to keep running
    If… Use
    your own servers, one or two people log in OpenSSH with keys
    dozens of servers, more than five people, access is granted and revoked OpenSSH certificates
    you work from a laptop on trains and in cafés, and the link drops Mosh on top of the same SSH
    sshd is broken or blocked by the firewall the provider console — then fix SSH
    an audit requires a log of who did what Teleport or similar

    What you get and what it costs

    What you get

    • Guessing is useless. An ed25519 key is 256 bits; no dictionary finds it. With PasswordAuthentication no, bots are refused before any password check, and the log only shows Connection closed … [preauth].
    • One protocol for everything: shell, files, git, tunnels. One port is open to the outside.
    • A swapped server shows up. The host key in known_hosts is checked on every login.
    • Access splits by key. Laptop, phone, CI — each has its own key and its own line in authorized_keys. Lose the phone and you delete one line (Each key and token grants one access).

    What it costs

    • A key is a file. Copy an id_ed25519 with no passphrase and you have access. Hence a passphrase on the key and an agent so you don’t type it every time (ssh-agent).
    • Lose the key, lose the login. If passwords are off and there was one key, only the provider console is left. Add a second key in advance.
    • A mistake in sshd_config doesn’t show right away. You apply it and new sessions can’t get in. The running session stays up, so keep it open until a login from a second one works (Keep the old session open).
    • Everyone knows port 22. Bots knock from the first minute. Moving the port cuts log noise but isn’t protection (Port 22).
    • Ubuntu changed how it starts. Since 22.10 (for LTS, since 24.04) sshd starts through ssh.socket: changing the port needs systemctl daemon-reload and a socket restart, not just restart ssh (sshd_config).

    Where to start

    If it doesn’t work

    Permission denied (publickey)

    Cause
    The server didn’t find the offered key in ~/.ssh/authorized_keys of the user you log in as: the key isn’t there, it’s the wrong user, or ~/.ssh has permissions that are too open — then sshd silently skips the file.
    Fix
    See which keys the client offers — ssh -v [email protected] 2>&1 | grep -E 'Offering|Authenticated' — and on the server, journalctl -u ssh -n 20. Permissions and order — SSH keys.

    Connection refused

    Cause
    The packets reach the server, but nothing listens on that port: sshd isn’t running or listens on another port.
    Fix
    From the provider console (Provider console): ss -tlnp | grep ssh and systemctl status ssh ssh.socket.

    Connection timed out — no refusal, no prompt

    Cause
    Packets are dropped on the way: the server’s or the provider’s firewall, and often your own address banned by CrowdSec after a few failed logins.
    Fix
    Try from your phone on mobile data — it has a different address. If that works, look for the ban: docker exec crowdsec cscli decisions list (cscli and bouncers).

    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!

    Cause
    The server’s key doesn’t match the one recorded in ~/.ssh/known_hosts. Usually the server was reinstalled or the address went to a new server; less often someone is in the middle.
    Fix
    Compare the new fingerprint through the provider console, then ssh-keygen -R 203.0.113.10 (Server fingerprint).

    Received disconnect … Too many authentication failures

    Cause
    The agent offers all its keys one by one, and after MaxAuthTries (6 by default) the server drops the connection before the right key’s turn.
    Fix
    In ~/.ssh/config for the host: IdentityFile and IdentitiesOnly yes (~/.ssh/config).

    The session freezes after a few idle minutes

    Cause
    A router or NAT on the way forgot the quiet TCP connection.
    Fix
    ServerAliveInterval 30 in ~/.ssh/config: the client sends an empty packet every 30 s, so the connection never looks idle.