basforge

Search titles, text and the “If it doesn’t work” cases.

    RU

    Are the sites up: a list check and errors handled with try/except

    The script walks through a list of URLs and prints each one’s status code and response time, or the reason there’s no answer: 404 Not Found, no connection, a timeout. One unreachable page doesn’t stop the checks of the rest — try/except catches the errors. Standard library only, no packages.

    New hereHTTP requests: urllibErrors: try and except

    What you end up with

    https://example.org                  200 OK, 5.60 s
    https://example.org/no-such-page     404 Not Found
    https://example.invalid              no connection: [Errno 11001] getaddrinfo failed

    The site is up — 200 and the response time. The server answered but the page doesn’t exist — 404. No answer at all — the reason in the system’s words: here the domain doesn’t exist.

    Step by step

    1. Write the script

      The file check_sites.py:

      import time
      import urllib.error
      import urllib.request
      
      SITES = [
          "https://example.org",
          "https://example.org/no-such-page",
          "https://example.invalid",
      ]
      TIMEOUT = 10  # seconds per site
      
      
      def check(url):
          """The status code and time — or what went wrong."""
          request = urllib.request.Request(url, headers={"User-Agent": "site-check/1.0"})
          start = time.monotonic()
          try:
              with urllib.request.urlopen(request, timeout=TIMEOUT) as response:
                  seconds = time.monotonic() - start
                  return f"{response.status} OK, {seconds:.2f} s"
          except urllib.error.HTTPError as error:
              return f"{error.code} {error.reason}"
          except urllib.error.URLError as error:
              return f"no connection: {error.reason}"
          except TimeoutError:
              return f"no answer within {TIMEOUT} s"
      
      
      for url in SITES:
          print(f"{url:36} {check(url)}")

      Put your own URLs in SITES, each in quotes with a comma at the end of the line.

    2. Run it

      python check_sites.py
    3. On a schedule

      Hourly or every morning — Windows Task Scheduler or cron on Linux: the action is python with the full path to the script. The script prints to the terminal; to keep the results, write the output to a file: python check_sites.py >> sites.log.

    What’s new here

    • A network request: urllib. urllib.request.urlopen sends a request and returns the response: response.status is the code, and 200 means all is well. Request builds a request with headers — here with our own User-Agent.
    • Errors: try and except. Code in try runs as usual; if an error happens in it, Python looks for a matching except and runs that instead of crashing the whole script. Order matters: HTTPError (the server answered with an error, 404, 500) is a special case of URLError (no answer at all), so it comes first. as error gives access to the details: error.code, error.reason.

    time.monotonic() is a stopwatch: the difference between two calls is how much time passed, and it isn’t thrown off if the computer’s clock gets adjusted.

    A timeout is two different cases

    If the server doesn’t answer the connection, urlopen raises URLError with the reason timed out. If it connects but stays silent on the response, it’s TimeoutError. The script catches both: the first as “no connection: timed out”, the second as “no answer within 10 s”.

    If it doesn’t work

    403 Forbidden from a site that opens in the browser

    Cause
    The site looks at the User-Agent header. By default urllib identifies itself as Python-urllib/3.15, and bot protection may reject such a request.
    Fix
    The script already sends its own User-Agent. If that doesn’t help, the site wants a browser; for an “is it up” check, the fact that it answered is enough: 403 means the server works.

    no connection: [SSL: CERTIFICATE_VERIFY_FAILED]

    Cause
    The site’s certificate has expired or was issued for another domain, and Python refuses to talk — like a browser’s red warning screen.
    Fix
    That is the finding: the certificate needs renewing on the server. Don’t turn the check off.

    The script hangs on one URL

    Cause
    The server accepted the connection and stays silent; without a timeout you could wait forever.
    Fix
    TIMEOUT in the script is 10 s per site. For a slow site, raise it, but don’t remove it.

    How to check

    Add a page of your own site that’s known not to exist to SITES — it gets 404, the others 200, and the script reaches the end of the list. Turn off Wi-Fi and run it again: “no connection” for every URL, with no crash.