Skip to content

Support HTTP.jl 2.0 alongside 1.x#96

Open
krynju wants to merge 9 commits into
mainfrom
http2-support
Open

Support HTTP.jl 2.0 alongside 1.x#96
krynju wants to merge 9 commits into
mainfrom
http2-support

Conversation

@krynju

@krynju krynju commented Jun 22, 2026

Copy link
Copy Markdown
Member

Summary

Adds HTTP.jl 2.0 support while keeping 1.x working. The package now resolves and tests cleanly against both HTTP majors.

HTTP.jl 2.0 is a major rewrite with a reworked public API. A load-time feature flag (_HTTP_V2 = !isdefined(HTTP, :Messages)) plus small shims in the HTTP.jl client backend bridge the differences:

  • statustextHTTP.Messages removed → fall back to Response.reason
  • header lookupHTTP.Header / HTTP.header(::Vector) removed → plain case-insensitive lookup over the request headers
  • content_type — returns a String on 2.0 (was a Pair on 1.x)
  • error fieldsTimeoutError.readtimeout.timeout_ns; ConnectError.error.cause
  • timeout ms — forced to Integer so the reason string still matches the \d+ milliseconds regex in is_longpoll_timeout (real bug, not just cosmetic)
  • streamingread(io, n) removed → readbytes!; drop the verbose kwarg unsupported by 2.0 HTTP.open; map readtimeoutread_idle_timeout on 2.0
  • serverHTTP.Forms.MultipartHTTP.Multipart (same type on 1.x)

Test fixtures updated for both versions; the timeout server fixture uses HTTP.listen! on 2.0 vs serve!(...; stream=true) on 1.x.

CI

Adds an http: ['1', '2'] matrix dimension. A step pins the HTTP compat per cell (resolver and Pkg.test's sandbox both honor it). Julia 1.6 + HTTP 2 is excluded, since HTTP.jl 2.0 requires Julia ≥ 1.10.

Verification

HTTP 1.11 HTTP 2.4
load
client: JSON / urlencoded form / 404 status text
Utils testset 16/16 16/16
buffered timeout
real longpoll-timeout (server + client) 1/1 1/1 + 14/14 timeout_tests

Forms/multipart-upload, DeepObject, Union-types and the petstore client tests also passed on HTTP 2.0 locally. No 1.x regression.

🤖 Generated with Claude Code

@tanmaykm

Copy link
Copy Markdown
Member

Companion code generator change: OpenAPITools/openapi-generator#24102. Required only when HTTP.jl 2.x is used with code-generated servers that extract parameters from headers.

Claude review comments:


What it does: Adds HTTP.jl 2.0 support while keeping 1.x working. The core changes are a load-time _HTTP_V2 feature flag plus small shims in the HTTP.jl client backend (src/client/httplibs/juliaweb_http.jl) that bridge API differences —
statustext, header lookup, content-type, error fields, timeout keyword/units, and streaming reads. src/server.jl switches HTTP.Forms.Multipart → HTTP.Multipart and adds a case-insensitive header fallback in get_param. The rest is test
fixtures, generated server stubs, a CI http: ['1','2'] matrix, and a version bump to 0.3.0.

The shim layer is well-organized and the comments are genuinely helpful. I verified the two things I was most suspicious of and both are clean:

  • The \d+ milliseconds regex match is safe — client.timeout is constrained to ::Integer (client.jl:47), so the 1.x e.readtimeout * 1000 stays an Int.
  • get_response_header → HTTP.header(resp, …) (juliaweb_http.jl:84, unchanged) is on the hot path for every response (client.jl:145), and the PR's "client: JSON ✓ on HTTP 2.4" result exercises it — so the HTTP.header removal was
    correctly scoped to the ::Vector method only.

Findings

  1. src/server.jl:64 — case-insensitive matching leaks from headers into query/path/form params (low severity, wrong-depth fix).
    The fallback lives in the shared get_param(source::Dict, …), but that method also serves query, path, and form params — not just headers, which are the only params that are case-insensitive per RFC. A conforming client never hits the
    fallback (generated code passes the spec's exact name and HTTP.queryparams/path params preserve wire case), so this doesn't break correct requests; it silently makes the server lenient — e.g. a request with ?Name=foo would now satisfy a
    query param declared as name, where before it resolved to nothing. The maintainer action is to scope the case-insensitive lookup to the header path (e.g. normalize the header dict where the generated code builds
    Dict{String,String}(req.headers), or a header-specific lookup) rather than the shared Dict method.

  2. test/server/timeouttest/timeouttest_server.jl:56 — _HTTP_V2 detection duplicated (reuse).
    http_v2 = isdefined(Base, :pkgversion) && something(pkgversion(HTTP), v"1") >= v"2" is a verbatim copy of the canonical OpenAPI.Clients._HTTP_V2 (juliaweb_http.jl:28). Other test files (utilstests.jl, allany/runtests.jl) already
    reference OpenAPI.Clients._HTTP_V2, so reuse the constant here too — a second copy of the version-detection logic can drift if the detection rule ever changes.

No correctness bugs survived verification. Both findings are low-severity polish; the PR is in good shape.


krynju and others added 9 commits June 23, 2026 13:17
Add compatibility for HTTP.jl 2.0's reworked public API while keeping
1.x working. A load-time feature flag (_HTTP_V2) and small shims in the
HTTP.jl client backend bridge the differences:

- statustext: HTTP.Messages removed -> fall back to Response.reason
- header lookup: HTTP.Header / HTTP.header(::Vector) removed -> plain
  case-insensitive lookup over the request headers
- content_type returns a String on 2.0 (was a Pair on 1.x)
- TimeoutError.readtimeout -> .timeout_ns; ConnectError.error -> .cause
- timeout milliseconds forced to Integer so the reason string still
  matches the `\d+ milliseconds` regex in is_longpoll_timeout
- read(io, n) removed -> readbytes!; drop the verbose kwarg unsupported
  by 2.0 HTTP.open; map readtimeout -> read_idle_timeout on 2.0

server.jl: HTTP.Forms.Multipart -> HTTP.Multipart (same type on 1.x).

Tests/fixtures updated for both versions, and the timeout server fixture
uses HTTP.listen! on 2.0 (serve!(...; stream=true) on 1.x).

CI now tests both HTTP majors via an http matrix dimension; a step pins
the HTTP compat per cell. Julia 1.6 + HTTP 2 is excluded (HTTP 2.0
requires Julia >= 1.10).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The unscoped sed matched the HTTP UUID line in [deps] too, rewriting it
to a bare version and breaking resolution (Malformed UUID string).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Use pkgversion(HTTP) >= v"2" (guarded for Julia < 1.9, where only HTTP
1.x can be installed) rather than checking for the internal HTTP.Messages
submodule. Explicit about intent and not coupled to an internal name.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
HTTP.jl 2.x introduced a dedicated HTTP.DNSError type for name-resolution
failures. It stringifies as "HTTP.DNSError(...)", whereas 1.x wrapped a
Sockets.DNSError in a ConnectError that stringified as "DNSError: ...".
This broke the client's reason-string contract (tests assert the reason
starts with "DNSError").

Add a version-guarded _http_error_message specialization that normalizes the
2.x DNSError into "DNSError: could not resolve host "<host>"". Elided under
1.x via @static if _HTTP_V2, so 1.x behavior is unchanged.
HTTP.jl 2.x removed the 1-arg HTTP.headers(req) form that returns all
headers; only the 2-arg lookup headers(req, key) remains. Generated server
read-handlers used Dict{String,String}(HTTP.headers(req)) to collect header
params, so any endpoint with a header param (e.g. delete_pet's api_key)
threw a MethodError -> HTTP 500 under HTTP 2.

Use req.headers, which works on both 1.x and 2.x.

NOTE: this is generated code. The matching change belongs upstream in the
openapi-generator Julia server template; these checked-in fixtures are
hand-patched to match until that lands.
Two HTTP.jl 2.x behavior changes broke test_debug:

1. verbose=true output: 1.x (and the curl backend) write the raw exchange to
   stderr; 2.x writes a '[http] ... via h1' format to stdout. The test
   captured stderr and asserted 'HTTP/1.1 200 OK', so under 2.x the pipe got
   no data and readavailable() blocked forever (hung the whole suite).
   Capture the right stream per version, assert the matching format, and
   close the pipe write-end before reading so an empty stream yields EOF
   instead of hanging.

2. response body framing: 1.x servers chunk the body ('27\r\n{json}\r\n0...'),
   2.x sends it unframed with Content-Length. The custom-verbose test parsed
   the JSON via split(str,'\n')[2] (assuming a chunk-size line) -> BoundsError
   under 2.x. Extract the JSON object by brace span instead, framing-agnostic.
HTTP.jl 2.x canonicalizes incoming request header field names (e.g.
"api_key" arrives as "Api_key"), whereas 1.x preserves the sent case.
The server's get_param did a case-sensitive Dict lookup, so header
params silently resolved to nothing under HTTP 2.x — a required header
param would 400, an optional one would be dropped.

Add a case-insensitive fallback in get_param (src/server.jl) that fires
only when the exact lookup misses. Header field names are case-
insensitive per RFC, so this is spec-correct and version-agnostic; it
also fixes already-generated servers without regeneration. Query/path
param dicts are not canonicalized, so exact match always wins for them.

Add regression tests in test/param_deserialize.jl covering canonicalized
key resolution, exact-match precedence, missing/required params, and the
to_param end-to-end path.
Server processes launched via run_server use --pkgimages=no, forcing a
full recompile on each launch. Under HTTP.jl 2.x (slower precompile) on
cold CI runners this regularly exceeds the 20s wait, so the server tests
time out and get skipped, and timed-out-but-still-launching processes
orphan the port for the next testset. Bump the wait to 90s.
@tanmaykm tanmaykm marked this pull request as ready for review June 23, 2026 07:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants