Encode square brackets in path segments

[eyupcanakman]

HttpUrl.Builder.addPathSegment() and setPathSegment() did not percent-encode [ and ] in path segments. RFC 3986 section 3.3 reserves brackets for IPv6 literals in the authority component, so they should be percent-encoded when they appear in path segments.

Added a PATH_SEGMENT_ENCODE_SET_RAW constant that includes [] and applied it to raw-input methods (addPathSegment, addPathSegments, setPathSegment). addEncodedPathSegment(), setEncodedPathSegment(), and URL parsing preserve literal brackets as before.

Fixes #6790

[eyupcanakman]

@swankjesse on your 2023 ask about confirming browsers. Browsers per WHATWG leave [ and ] alone in path segments (neither is in the path percent-encode set), so new URL("https://x/a[0]").pathname stays /a[0] in Chrome and Firefox. The fix here matches RFC 3986 and Java URI (which already rejects [ in paths) but differs from what browsers do, which CONTRIBUTING prefers.

Should I close this? addEncodedPathSegment() and parsing already preserve literal brackets, so the only behavioral change is on the raw-input methods.

[JakeWharton]

The "encoded" variants of the functions are the "raw" ones. These functions are the ones which are used by 99.9% of callers. We should fix these. Retrofit wants this!

[eyupcanakman]

Kept it open. addPathSegment and setPathSegment now encode [ and ], and addEncodedPathSegment plus parsing pass them through unchanged.

[swankjesse]

I would prefer we not change the existing behavior here. Even if a different behavior is more correct per some specs, it’ll result in different HTTP calls being made to servers, which means different data being served to callers.

https://publicobject.com/2017/08/01/url-encoding-is-material/

Now . . . introducing a new function here? Hmmm... I’m unsure how I feel about this.

[swankjesse]

Not sure if this is a compelling argument, but if I type http://httpbin.org/anything/foo[1] into my browser’s URL bar, the square braces are retained verbatim.

[eyupcanakman]

Yeah, same result here, [ and ] aren't in the WHATWG path set so the URL bar keeps them. That's the parse path though, and the PR doesn't touch parsing or addEncodedPathSegment, so "/foo[1]".toHttpUrl() keeps the braces too. The only thing that moves is addPathSegment("foo[1]") on raw input, which is the case #6790 hit a 400 on and the one Retrofit wants. Happy to go either way on whether that's worth the wire change.