Add OrderedListState support in Go SDK

[hnnsgstfssn]

The Go SDK stateful DoFn API was missing OrderedListState, the only
state type already supported in Java and Python but absent from the Go
plumbing. This adds the missing pieces.

• state.OrderedList[T] with Add, Read, ReadRange, Clear, ClearRange
  operating on int64 sort keys
• Provider interface wired through exec, graphx translate, and the
  harness state manager to the Fn API beam:coder:ordered_list_state:v1
  URN
• Unit tests (state, userstate, sideinput) and integration tests against
  the prism runner

Relates to the following issues.

• [Go SDK] Implement State and Timer support #20510 tracking issue for Go SDK state & timer support
  (closed, but ordered list was not included)
• [Task]: Implement State in Go #22736 the original "Implement State in Go" task (closed with
  Value/Bag/Combining only)
• Go SDK support for portable user state #18493 open, tracks full portable user state coverage including
  ordered list
• [Feature Request][Proposal][Go SDK] Composite State and Timers  #25894 open, composite state proposal that explicitly depends on
  ordered list as a primitive

---

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

• Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
• Update CHANGES.md with noteworthy changes.
• If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

[gemini-code-assist]

Summary of Changes

Hello @hnnsgstfssn, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the Apache Beam Go SDK by introducing OrderedListState, a crucial state primitive that was previously available in Java and Python SDKs. This addition allows Go developers to manage timestamped, ordered collections of data per key within stateful DoFns, bringing the Go SDK closer to feature parity with other language SDKs and enabling more complex stateful processing patterns.

Highlights

• OrderedListState support in Go SDK: Added the OrderedList[T] state type to the Go SDK's stateful DoFn API, enabling operations like Add, Read, ReadRange, Clear, and ClearRange using int64 sort keys.
• Plumbing and Integration: Integrated OrderedListState through the exec package, graphx translation, and the harness state manager, including the beam:coder:ordered_list_state:v1 URN.
• Examples and Tests: Included a new example pipeline demonstrating OrderedListState usage, a snippet for documentation, and comprehensive unit and integration tests to validate the new functionality.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• CHANGES.md
  • Added an entry for OrderedListState support in the Go SDK.
• sdks/go/examples/ordered_list_state/ordered_list_state.go
  • Added a new example demonstrating the use of OrderedListState to accumulate and summarize timestamped events.
• sdks/go/examples/snippets/04transforms.go
  • Added a new code snippet illustrating OrderedListState usage, including Add, ReadRange, and ClearRange operations.
• sdks/go/pkg/beam/core/graph/fn.go
  • Updated the validateState function to recognize state.TypeOrderedList as a supported state type.
• sdks/go/pkg/beam/core/runtime/exec/data.go
  • Extended the StateReader interface with methods for OpenOrderedListUserStateReader, OpenOrderedListUserStateAppender, and OpenOrderedListUserStateClearer.
• sdks/go/pkg/beam/core/runtime/exec/sideinput_test.go
  • Added no-op implementations for the new OrderedListState reader, appender, and clearer methods in testStateReader.
• sdks/go/pkg/beam/core/runtime/exec/translate.go
  • Modified the makeLink function to handle OrderedListSpec when translating state specifications.
• sdks/go/pkg/beam/core/runtime/exec/userstate.go
  • Imported math and protowire packages.
  • Added initialOrderedListByKey to stateProvider to cache initial ordered list state values.
  • Implemented ReadOrderedListState, WriteOrderedListState, and ClearOrderedListState methods in stateProvider.
  • Added helper functions getOrderedListReader, getOrderedListAppender, and getOrderedListClearer.
  • Implemented encodeOrderedListEntry and decodeOrderedListEntry for serializing and deserializing ordered list entries.
  • Initialized initialOrderedListByKey in userStateAdapter.NewStateProvider.
• sdks/go/pkg/beam/core/runtime/exec/userstate_test.go
  • Initialized initialOrderedListByKey in buildStateProvider.
• sdks/go/pkg/beam/core/runtime/graphx/translate.go
  • Defined new URNs URNOrderedListUserState and URNOrderedListStateProtocol.
  • Added URNOrderedListStateProtocol to the list of Go capabilities.
  • Extended the addMultiEdge function to generate StateSpec_OrderedListSpec for state.TypeOrderedList.
• sdks/go/pkg/beam/core/runtime/harness/statemgr.go
  • Implemented OpenOrderedListUserStateReader, OpenOrderedListUserStateAppender, and OpenOrderedListUserStateClearer in ScopedStateReader.
  • Added factory functions newOrderedListUserStateReader, newOrderedListUserStateWriter, and newOrderedListUserStateClearer for creating state key readers/writers.
• sdks/go/pkg/beam/core/state/state.go
  • Imported cmp, math, and slices packages.
  • Defined TypeOrderedList as a new TypeEnum.
  • Introduced OrderedListEntry and TimestampedValue structs for ordered list elements.
  • Defined the OrderedList[T] struct with Add, Read, ReadRange, Clear, and ClearRange methods.
  • Implemented StateKey, KeyCoderType, CoderType, and StateType methods for OrderedList[T].
  • Added MakeOrderedListState factory function.
• sdks/go/pkg/beam/core/state/state_test.go
  • Imported the math package.
  • Added initialOrderedListState to fakeProvider.
  • Implemented ReadOrderedListState, WriteOrderedListState, and ClearOrderedListState methods in fakeProvider.
  • Added comprehensive unit tests for OrderedListRead, OrderedListAdd, OrderedListClear, and OrderedListClearRange.
• sdks/go/test/integration/primitives/state.go
  • Registered orderedListStateFn for DoFn processing.
  • Added orderedListStateFn struct and its ProcessElement method to demonstrate ordered list state.
  • Added OrderedListStateParDo integration test function.
• sdks/go/test/integration/primitives/state_test.go
  • Added TestOrderedListState to run the OrderedListStateParDo integration test.
• website/www/site/content/en/documentation/programming-guide.md
  • Added a code sample for ordered_list_state to the programming guide documentation.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[github-actions]

Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment assign set of reviewers

[github-actions]

Assigning reviewers:

R: @lostluck for label go.
R: @damccorm for label website.

Note: If you would like to opt out of this review, comment assign to next reviewer.

Available commands:

• stop reviewer notifications - opt out of the automated review tooling
• remind me after tests pass - tag the comment author after tests pass
• waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

[damccorm]

@shunping could you take a look at this one?

[lostluck]

Seems fairly reasonable on a quick pass at least. Just one naming nit.

I like the clear framing that the sort key can be a millisecond time representation, but doesn't have to be. Outside of overloads or a 2nd method, this is preferable to forcing users to get their sort keys represented as time.Time for us to interpret.