#2001 Document adaptive scheduler metadata

[Yurii201811]

Summary

• document the complete metadata set persisted by AdaptiveScheduler
• use the exact lowercase last-modified key from the implementation
• include fetch.statusCode, which the persisted-status path also requires

Fixes #2001

Validation

• parsed the documented key list as YAML
• git diff --check
• attempted mvn -pl docs -am -DskipTests package; Maven Central returned HTTP 403 while resolving org.apache:apache:pom:39

Checklist

• The issue is referenced in the commit and PR
• The title starts with the issue number
• The branch is based on the current target branch
• The contribution is one commit
• No code or dependencies changed

[jnioche]

Looks good, are we sure last-modified needs lowercasing?

[rzo1]

Looks good, are we sure last-modified needs lowercasing?

Good question — I checked the code. Short answer: for main (and thus the next release) the case doesn't matter, and lowercase is the better choice; for 3.5.1 and earlier it would actually have to be Last-Modified.

Details:

• Since fix(core): make Metadata keys consistently case-insensitive #1827 (bb33f92, make Metadata keys consistently case-insensitive), Metadata normalizes all keys to lowercase on both write and read, so last-modified and Last-Modified in metadata.persist behave identically. Lowercase matches the internal stored form and the AdaptiveScheduler javadoc, which also documents last-modified.
• On 3.5.1 and earlier (no fix(core): make Metadata keys consistently case-insensitive #1827), AdaptiveScheduler stores the value under org.apache.http.HttpHeaders.LAST_MODIFIED = "Last-Modified" verbatim, and the metadata.persist filter lookup can't match a capitalized stored key from a lowercase config entry — so last-modified would silently fail to persist there. That's presumably why [Improvement] Document metadata to be persisted for adaptive scheduling #2001 listed the capitalized form.

Since this doc change ships with the next release (which includes #1827), lowercase is correct here — it just shouldn't be backported to docs for older releases.

One small suggestion while we're at it: the sentence "The last-modified and protocol.etag fields allow conditional requests when scheduler.adaptive.setLastModified is enabled" is slightly misleading — setLastModified only controls whether the scheduler writes last-modified (→ If-Modified-Since); the protocol.etag / If-None-Match path works independently of that setting. Maybe reword to:

The last-modified field (written when scheduler.adaptive.setLastModified is enabled) and protocol.etag allow conditional requests (If-Modified-Since / If-None-Match).

[sebastian-nagel]

Thanks, @Yurii201811!

Maybe reword to

+1 I'd second this.

Before protocol.md.prefix has been introduced by #776 (or if an empty prefix is configured), setLastModified controls whether the last-modified value sent by the responding server is overwritten in the persisted metadata.

On 3.5.1 and earlier (no #1827), AdaptiveScheduler stores the value under org.apache.http.HttpHeaders.LAST_MODIFIED = "Last-Modified" verbatim.

It was changed to lowercase in 527f936 and then back to Last-Modified by #1587.

While AdaptiveScheduler was introduced, it was a lower-case string constant. That's how its Javadoc describes it. But maybe the history it not really relevant, except you have to deal with an old status index.

[Yurii201811]

Updated that sentence as suggested in e20cbca. Thanks for clarifying how the no-prefix case behaved.