The monotonic timestamp counter added in #83 breaks event ordering when more than one process writes to the same AgentCore Memory session. Consecutive turns served by different instances end up interleaved, and for tool-using agents that means toolResult can land before toolUse on reload, which the model API rejects.
Two separate problems here.
1. The 1-second increment is wrong, and it isnt needed.
_get_monotonic_timestamp() bumps _last_timestamp by a full second on collision. The comment says its because boto3 cant do sub-second resolution. Thats not true. The bedrock-agentcore client serializes eventTimestamp as a unix timestamp with millisecond precision, and the service keeps it. You can see it in the eventId prefix, which is a millisecond epoch. So a single turn with a few tool calls gets stamped several seconds into the future for no reason.
2. The counter is per-process, so it cant coordinate across instances.
_last_timestamp is a class attribute. It isnt shared between processes. When the next turn lands on a different instance, that instances counter starts from current wall-clock time, which is earlier than the future-dated timestamps the previous instance already wrote. New events collide with or come before the old ones, and the turns interleave.
Its also class-level instead of instance-level, so two session managers for different sessions in the same process step on each others counter.
What I think the fix looks like:
- Pass through real millisecond timestamps instead of inflating to whole seconds. Only break genuine same-millisecond ties, by 1ms.
- Make the timestamp state instance-scoped, not class-level.
- Seed the floor from the latest already-persisted event at session init. The load path already reads events, so a fresh instance can continue after whatever another instance wrote instead of restarting from wall-clock. If theres nothing to load, fall back to current time like it does today.
The monotonic timestamp counter added in #83 breaks event ordering when more than one process writes to the same AgentCore Memory session. Consecutive turns served by different instances end up interleaved, and for tool-using agents that means
toolResultcan land beforetoolUseon reload, which the model API rejects.Two separate problems here.
1. The 1-second increment is wrong, and it isnt needed.
_get_monotonic_timestamp()bumps_last_timestampby a full second on collision. The comment says its because boto3 cant do sub-second resolution. Thats not true. Thebedrock-agentcoreclient serializeseventTimestampas a unix timestamp with millisecond precision, and the service keeps it. You can see it in theeventIdprefix, which is a millisecond epoch. So a single turn with a few tool calls gets stamped several seconds into the future for no reason.2. The counter is per-process, so it cant coordinate across instances.
_last_timestampis a class attribute. It isnt shared between processes. When the next turn lands on a different instance, that instances counter starts from current wall-clock time, which is earlier than the future-dated timestamps the previous instance already wrote. New events collide with or come before the old ones, and the turns interleave.Its also class-level instead of instance-level, so two session managers for different sessions in the same process step on each others counter.
What I think the fix looks like: