Skip to content

fix: normalize ISO date equality filters#11963

Open
Aarkin7 wants to merge 4 commits into
deepset-ai:mainfrom
Aarkin7:fix/iso-date-filter-equality
Open

fix: normalize ISO date equality filters#11963
Aarkin7 wants to merge 4 commits into
deepset-ai:mainfrom
Aarkin7:fix/iso-date-filter-equality

Conversation

@Aarkin7

@Aarkin7 Aarkin7 commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Related Issues

Proposed Changes:

This fixes a gap in metadata filtering where ==!=in, and not in were still comparing ISO timestamps as raw strings. That meant equivalent values like 2025-02-03T12:45:46Z and 2025-02-03T12:45:46+00:00 could silently miss matching documents.

The fix normalizes date-like values for equality-style filters so they behave consistently with the ordering operators. I also added regression tests for both the filter helper and the InMemoryDocumentStore path, plus a release note.

How did you test it?

  • Ran hatch run fmt on the touched files
  • Ran hatch run test:unit test/utils/test_filters.py
  • Ran targeted InMemoryDocumentStore filter tests
  • Manually verified that == and in now return matches for equivalent ISO timestamps

Notes for the reviewer

The main logic change is in haystack/utils/filters.py.

The most useful review points are:

  • the equality normalization helper in filters.py
  • the new regressions in test/utils/test_filters.py
  • the end-to-end store check in test/document_stores/test_in_memory.py

Checklist

  • I have read the contributors guidelines and the code of conduct.
  • I have updated the related issue with new insights and changes.
  • I have added unit tests and updated the docstrings.
  • I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test: and added ! in case the PR includes breaking changes.
  • I have documented my code.
  • I have added a release note file, following the contributors guidelines.
  • I have run pre-commit hooks and fixed any issue.

@Aarkin7
Aarkin7 requested a review from a team as a code owner July 10, 2026 17:54
@Aarkin7
Aarkin7 requested review from julian-risch and removed request for a team July 10, 2026 17:54
@vercel

vercel Bot commented Jul 10, 2026

Copy link
Copy Markdown

@Aarkin7 is attempting to deploy a commit to the deepset Team on Vercel.

A member of the Team first needs to authorize it.

@Aarkin7

Aarkin7 commented Jul 12, 2026

Copy link
Copy Markdown
Contributor Author

The remaining CI failures appear unrelated to this PR. They’re in test/components/generators/chat/test_openai_responses*.py with InputTokensDetails.cache_write_tokens validation errors. This PR only touches filter logic/tests and a release note.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

'==' and 'in' filters silently miss equivalent ISO timestamps (Z vs +00:00)

1 participant