Skip to content

About

A lightweight HTTP/1.1 web server built from raw sockets in Python.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

6 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

PyHTTP — an HTTP/1.1 server from scratch in Python

📘 Case study: raqueeb.dev/projects/pyhttp-server — architecture, stack and implementation notes.

A working web server built on raw sockets. No Flask, no framework, no third-party packages. Parses HTTP requests from bytes, routes them, serves static files and JSON API responses.

python main.py          # localhost:8000
python main.py 9090     # custom port
$ curl localhost:8000/health
{"status": "ok"}

$ curl localhost:8000/greet/raqueeb
Hello, raqueeb!

$ curl localhost:8000/style.css
body { font-family: sans-serif; }

Endpoints

Method Path What it does
GET /health liveness check
GET / POST /echo returns the request back as JSON
GET /greet/{name} greeting with a path parameter
GET /status/{code} returns whatever status code you ask for
GET /* static files from static/

HEAD is accepted automatically wherever GET is registered.

Layout

File Responsibility
src/request.py parse raw HTTP bytes into a Request
src/response.py build raw HTTP bytes from a Response
src/router.py pattern matching, path parameters, 404 vs 405
src/static.py serve files with MIME detection and traversal protection
src/server.py TCP listener, select() loop, connection management
src/app.py route table, wiring, dispatch order
main.py entry point
tests/ 72 tests

The dependency direction is: server.py calls app.py, which calls router.py + static.py, which call request.py + response.py. Nothing below the app knows what a socket is; nothing above the request parser knows what bytes are.

Tests

python -m pytest tests/ -v

72 tests, all passing:

  • test_request.py (22) — parses valid requests, normalises paths, rejects every class of malformed input (bad method, missing parts, negative Content-Length, body too large, incomplete headers).
  • test_router_response_static.py (26) — routing with parameters, 405 with Allow, HEAD implicit acceptance, response serialisation (including the HEAD bug fix), static file serving, directory traversal, MIME detection.
  • test_integration.py (24) — real TCP connections against a running server: all endpoints, path parameters, query strings, percent-encoding, HEAD, static files, malformed requests, oversized headers, 10 simultaneous connections, a slow client not blocking others, and a clean SIGINT shutdown.

Bugs the tests caught

HEAD returned Content-Length: 0. The response serialiser computed Content-Length from the suppressed body instead of the body that a GET would have returned. RFC 9110 §9.3.2 requires HEAD to report the same Content-Length as GET. A unit test caught it; the fix is one line: use len(self.body) rather than len(body) after suppression.

Malformed request crashed the server. After sending a 400, the code called conn.sock.fileno() on the just-closed socket, which returns -1, causing a KeyError that propagated up and killed the event loop. A real bug that existed because the "save fileno before closing" pattern was missed in one of three identical code paths. The fix was a _reject() helper that does it once, in one place, so the same bug cannot reappear.

HEAD returned 405 on GET routes. The router matched methods strictly, so HEAD on a GET-only route was a 405. The fix: when the method is HEAD and the route has no explicit HEAD handler but does have GET, dispatch to GET and let the server suppress the body.

Design decisions, and how to defend them

Why no framework. The HTTP spec, socket programming, and status-code semantics are what an interviewer asks about, and they are invisible when Flask does them. The whole server is about 600 lines; each concept is in its own file and can be pointed at.

Why select() and not threads. The server is single-threaded and uses select() to multiplex all connections. A thread-per-connection version would be shorter, but one slow client holding a thread hostage at ~8 MB of stack per thread quickly exhausts memory. With select(), a slow client simply does not appear in the ready set, and everyone else is served in the meantime. There is an integration test that opens a connection, sends nothing, and confirms a second client is served immediately.

Why Request and Response are objects, not $_GET and echo. This is the same decision as in the PHP project: a handler takes a Request and returns a Response. The entire application is a pure function of its input and can be tested without a running server — 48 of the 72 tests run this way in 50ms. Code that reads $_SERVER and calls header() inline cannot be tested that way.

Why the parser is strict. A get in lowercase is rejected, not silently uppercased. A non-integer Content-Length is an error, not a 0. Unknown methods are 501, not 400. Strictness catches bugs in the client early, and the error messages explain what is wrong so a developer can fix it. Leniency hides them.

Why 404 and 405 are distinct. DELETE /health is not "not found" — the path exists, the method is wrong. The router collects every method that would have matched, and if any did, it returns 405 with an Allow header listing them, as RFC 9110 requires. HEAD is always included when GET is present. This is the same pattern as the PHP and Go projects.

Why directory traversal uses resolve() and not string filtering. Filtering .. from the path string is fragile: ....//, URL-encoded dots, and symlinks all bypass it. Path.resolve() follows every symlink, collapses every .., and returns the canonical path. The check is then a single startswith(root) comparison against the resolved root. There are two tests: /../../../etc/passwd and /sub/../../etc/passwd.

Why the server reports the actual port, not the requested one. When port 0 is requested the kernel picks one. The integration tests rely on this: each test class starts its own server on port 0 and reads the real port from stdout. The first version printed the requested port, which was 0 and useless.

Why idle connections are reaped. Without a timeout, a client that opens a TCP connection and never sends anything holds a slot in the select() set forever. Any client can do this deliberately; 256 such connections fills the limit and the server refuses real traffic. IDLE_TIMEOUT (30s) closes them.

Why the server still serves after a bad request. The _reject() helper saves the file descriptor number before closing the socket, then removes it from the connection map. The original code closed first and then asked the closed socket for its descriptor, which returned -1 and crashed the event loop. This is the kind of bug that only appears when a real attacker sends garbage.

Trade-offs I would raise. HTTP/1.1 keep-alive is not supported; every response sets Connection: close, so each request is one TCP connection. Supporting it means parsing the next request from the same buffer after the current response is sent, which is the right next step. There is no TLS, no chunked transfer encoding, no If-None-Match / 304, no multipart form parsing, and no WebSocket upgrade. The static file server reads the entire file into memory, which is wrong for large files — sendfile() or chunked reading would fix it. And select() has a FD_SETSIZE limit on some platforms, the same reason the C project used poll().

About

A lightweight HTTP/1.1 web server built from raw sockets in Python.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages