[WARC] Format WARC-IP-Address header values following the WARC community recommendations

[sebastian-nagel]

Format the IP address in the WARC-IP-Address header following iipc/warc-specifications#100.

Especially, IPv6 addresses should be given in the short form specified by RFC 5952:

Current:

WARC/1.0
WARC-Type: request
WARC-IP-Address: 2620:cb:2000:0:0:0:0:1

Expected:

WARC/1.0
WARC-Type: request
WARC-IP-Address: 2620:cb:2000::1

Same for WARC response records.

[dpol1]

Looks good overall! Just a couple of quick questions:

• InetAddresses comes from Guava: should external/warc declare it directly instead of relying on the transitive dependency from core?
• Guava turns ::ffff:192.0.2.128 into IPv4. Is that intentional? Also, the test comment should probably say “IPv4-mapped” rather than “IPv4-compatible”.
• Maybe assert the final header value in a request/response test. This could also be covered in [Improvement] WARC writer: WARC-Protocol header to follow WARC field proposals #1998 if you plan to expand the WARC tests there.

[sebastian-nagel]

Thanks, for the careful review, @dpol1! I'll update the PR to address your comments.

Guava turns ::ffff:192.0.2.128 into IPv4. Is that intentional?

Well, it's rather enforced by the behavior of Java's InetAddress class:

jshell> InetAddress.getByName("::ffff:127.0.0.1").getClass()
$1 ==> class java.net.Inet4Address

jshell> InetAddress.getByName("127.0.0.1").equals(InetAddress.getByName("::ffff:127.0.0.1"))
$2 ==> true

But very good point. Looks like, there is no recommendation how IPv4-mapped addresses are recorded. Needs clarification. For now we can keep the unit tests, I think.

The unit test replicates the behavior of jwarc which archives IPv6-mapped IPv4 addresses as IPv4:

WARC-IP-Address: 127.0.0.1
WARC-Target-URI: http://[0:0:0:0:0:ffff:127.0.0.1]/

Other WARC-archiving tools behave different, e.g., warcprox:

WARC-Target-URI: http://[0:0:0:0:0:ffff:127.0.0.1]/
WARC-IP-Address: ::ffff:127.0.0.1

But it's maybe an open question whether this is the desired behavior, cf. internetarchive/warcprox#277.

[dpol1]

Thanks @sebastian-nagel - looks good to me now!