Skip to content

fix: increase batch payload size limit from 32KB to 5MB#110

Open
cat-ph wants to merge 1 commit intomasterfrom
fix/increase-batch-size-limit
Open

fix: increase batch payload size limit from 32KB to 5MB#110
cat-ph wants to merge 1 commit intomasterfrom
fix/increase-batch-size-limit

Conversation

@cat-ph
Copy link
Copy Markdown

@cat-ph cat-ph commented Mar 28, 2026

Summary

  • The PHP SDK enforces a 6-year-old 32KB batch payload size limit across all three consumers (ForkCurl, LibCurl, Socket), silently dropping batches that exceed it
  • Any non-trivial batch with feature flags or rich properties can hit this easily
  • The PostHog /batch/ endpoint accepts up to 20MB
  • The Python SDK uses 5MB, Ruby uses 500KB, JS has no client-side limit
  • This bumps the PHP limit to 5MB to match Python

Changes

  • lib/Consumer/ForkCurl.php - 32 * 1024 -> 5 * 1024 * 1024
  • lib/Consumer/LibCurl.php - same
  • lib/Consumer/Socket.php - same

Test plan

  • Existing tests pass (no tests assert the 32KB limit on master)
  • Verified >32KB batch (~55KB) sends successfully against local PostHog instance

The 32KB limit was inherited from 6yo  library and is far
too restrictive. The PostHog /batch/ endpoint accepts
up to 20MB, and the Python SDK already uses a 5MB limit.
@cat-ph
Copy link
Copy Markdown
Author

cat-ph commented Mar 28, 2026

I think this is probably a simpler solution for #31 than #82 and #83 since 5mb gives quite a bit more leeway than 5kb, I tested this against localhost with >32kb and it seems to go through just fine 🤔

@cat-ph cat-ph requested review from a team, ablaszkiewicz and hpouillot March 28, 2026 11:31
// Verify message size is below than 32KB
if (strlen($payload) >= 32 * 1024) {
// Verify message size is below than 5MB
if (strlen($payload) >= 5 * 1024 * 1024) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 things to consider

  • move 5 * 1024 * 1024 to a const so its computed at compile time instead of runtime for every call
  • maybe the payload is small because PHP is single threaded and will block the request until its finished (is that the case?), and 5MB takes much longer to upload than 32KB, we might consider something smaller?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants