diff --git a/CHANGES.md b/CHANGES.md new file mode 100644 index 000000000..4730f098f --- /dev/null +++ b/CHANGES.md @@ -0,0 +1,428 @@ +# Pin Drop OpenPanel — Full Change Log + +Detailed rundown of every change made in this branch, grouped by +feature. Written for Justin + anyone else reviewing the diff before +deploy. Nothing here requires a schema migration — all changes are +code-only on the dashboard, API, and shared packages. + +--- + +## 1. Bug fixes + +### 1.1 Pagination broken past page 2 +**Before:** On Profiles → Identified / Anonymous / Power Users, clicking +past page 2 returned an empty table. + +**Root cause:** +- `getProfileListCount` was querying the raw `profiles` table. Because + profiles use a ReplacingMergeTree engine, historical row versions + were still being counted — inflating `meta.count` and making the UI + render pages that had no actual data. +- `powerUsers` returned `meta.count = data.length` rather than the + total distinct profile count, so `pageCount` was always stuck at 1 + even though the UI let you click "next". + +**Fix:** `getProfileListCount` now reads from `profiles FINAL`; +`powerUsers` (now folded into the unified `list` endpoint) runs a +proper `count(distinct profile_id)`. + +Files: `packages/db/src/services/profile.service.ts`, +`packages/trpc/src/routers/profile.ts`. + +### 1.2 Text bleeding across cells +**Before:** Long values in the Profile Information grid bled into +neighbouring cells (the "Biography" label overlapping the value). In +the profile-summary metric strip, long times like "25 minutes ago" +overflowed the tile and overwrote the next tile. + +**Root cause:** CSS grid's default track sizing is `minmax(auto, 1fr)` +which allows content to push a column wider than its nominal share. +Without explicit `min-w-0` on the child, `truncate` can't fire. + +**Fix:** +- `KeyValueGrid` cells use explicit `grid-cols-[minmax(0,1fr)_…]` + templates; each cell has `min-w-0 overflow-hidden`. +- `OverviewMetricCard` button gets `min-w-0 overflow-hidden` so grid + tracks resolve to `minmax(0, 1fr)`. + +Files: `apps/start/src/components/ui/key-value-grid.tsx`, +`apps/start/src/components/overview/overview-metric-card.tsx`. + +### 1.3 "Last seen" on Power Users was actually first seen +**Before:** The Power Users tab header said "Last seen" but the cell +rendered the profile's `createdAt` (when we first identified them). + +**Fix:** The unified `getEnrichedProfileList` returns both real +`lastSeen` (max of event created_at) and `firstSeenActivity` (min of +event created_at) from the events table. The columns now map to the +correct data. + +Files: `packages/db/src/services/profile.service.ts`, +`apps/start/src/components/profiles/table/columns.tsx`. + +### 1.4 Latest Events widget collapsing to zero height +**Before:** The Latest Events card on the profile page would render +empty even when there was data, because its scroll area height was +computed from the outer Widget's bounding rect — which resolved to 0 +when the grid row around it was short. + +**Fix:** Swapped the self-measured scroll for a fixed `max-h-[420px]` +scroll area. Added a proper empty state ("No events for this profile +yet"). Also hides the per-row profile name when rendered on a +profile's own page (it's always the same person). + +Files: `apps/start/src/components/profiles/latest-events.tsx`, +`apps/start/src/components/events/event-list-item.tsx`. + +### 1.5 "5 minutes ago" overflow +**Before:** `timeAgo` in the metric cards produced strings like +"5 minutes ago" that wouldn't fit in a narrow tile. + +**Fix:** Added `timeAgoShort` which normalises both the library's +`short` format and the long form to a consistent terse output (e.g. +"5 mins ago", "3 hrs ago", "2 mos ago", "1 yr ago"). Singular "1 min" +stays singular. + +Files: `apps/start/src/utils/date.ts`, +`apps/start/src/components/overview/overview-metric-card.tsx`. + +### 1.6 Group member count mismatch +**Before:** Groups page showed, say, "Umbrella Holdings · 15 members" +but clicking in showed 20 in the Members tab. The two counts used +different data sources. + +**Root cause:** `getGroupStats` was doing +`uniqExact(profile_id) FROM events WHERE has(groups, ...)` — +event-based. The group detail's Members tab does +`FROM profiles FINAL WHERE has(groups, ...)` — profile-based. Any +profile assigned to a group but without group-tagged events got +dropped from the stats count. + +**Fix:** Split `getGroupStats` into two parallel queries: members +from `profiles FINAL` (matches the Members tab exactly), activity +from `events` (still event-derived because *activity* is an event +concept, not a membership one). + +Files: `packages/db/src/services/group.service.ts`. + +### 1.7 "Os" / "Id" / acronym capitalisation +**Before:** Keys like `os`, `id`, `url` rendered as "Os", "Id", "Url". + +**Fix:** `camelCaseToWords` now knows about common acronyms (OS, URL, +ID, API, IP, UI, URI, UTM, SDK, SSL, UUID, iOS, etc.) and uppercases +them in place. + +Files: `apps/start/src/utils/casing.ts`. + +--- + +## 2. Unified profile tables + sorting + +### 2.1 One column set across Identified / Anonymous / Power Users +**Before:** Different columns per tab, including Referrer (low value +at table level) and Browser (rarely actionable). + +**After:** All three tabs share this set: + +| Column | Notes | +| -------------- | --------------------------------------------------- | +| Name | Avatar + profile name (links to detail) | +| Plan | Solo / Team / Team+ / Team Pro badge + subscriber ✓ | +| Events | Total event count | +| Session time | Total session duration across all platforms | +| Country | Flag + city | +| OS | Icon + OS name | +| Model | Brand / model | +| First seen | Earliest event | +| Last seen | Most recent event | +| Groups (hidden default) | Any team badges | + +Removed: Referrer, Browser. + +### 2.2 Sortable headers +Every column is clickable. Sort direction toggles asc/desc. URL is +synced (`?sort=eventCount&dir=desc`) so the state survives reload and +is bookmarkable. + +Default sort: `createdAt DESC` for Identified/Anonymous, `eventCount +DESC` for Power Users. + +### 2.3 Unified backend query +New `getEnrichedProfileList` in `profile.service.ts` does the whole +thing in one ClickHouse query: + +- `profiles FINAL` joined with two CTEs (event aggregates, session + aggregates) on `profile_id`. +- Filter: `is_external = true/false` or neither (depending on tab). +- Sort: profile column or any aggregate (event count, session time, + first/last seen activity, session count). +- Pagination: `LIMIT take OFFSET cursor * take`. + +`profile.powerUsers` is now a thin wrapper over `profile.list` with +`isExternal: true` and `sortBy: 'eventCount'` defaults. + +Files: `packages/db/src/services/profile.service.ts`, +`packages/trpc/src/routers/profile.ts`, +`apps/start/src/components/profiles/table/columns.tsx`, +`apps/start/src/components/profiles/table/index.tsx`, +`apps/start/src/components/ui/data-table/data-table-hooks.tsx` (new +`useDataTableSort` hook), +`apps/start/src/routes/_app.$organizationId.$projectId.profiles._tabs.*.tsx`. + +--- + +## 3. Profile detail redesign + +### 3.1 Merged Profile + Properties tabs +**Before:** Two tabs. Profile tab had curated fields, Properties tab +dumped every raw property including noise like `browser`, `path`, +`__referrer`. + +**After:** Single card, one view. Curated order: + +1. First name +2. Last name +3. Email +4. Is Subscriber +5. Plan +6. Team Name (fetched from `profile.groups`, resolves first group of + type `team`, falls back to the first group, or "N/A") +7. ID +8. Created at +9. Last Known Location (City, Country — flagged as "last known" + because users travel) + +Any additional custom properties Pin Drop sends via `identify()` (not +in the hidden list) appear below automatically. + +**Hidden on both previous tabs:** +`browser`, `browser_version`, `os`, `os_version`, `device`, `brand`, +`model`, `path`, `referrer`, `referrer_name`, `referrer_type`, +`__referrer`, `__query`, `__version`, `__buildNumber`, `biography`, +`bio`, `description`, plus any string value longer than 120 chars. + +Everything is still in ClickHouse — we're just filtering for UI +clarity. + +Files: `apps/start/src/components/profiles/profile-properties.tsx`. + +### 3.2 Source card (new) +Answers "where did this user come from?". Two sections: + +- **First seen via** — first session's referrer / UTM / channel with + the campaign name + search keyword + entry page. +- **All sources** — every distinct source/UTM combination this user + has arrived through, ranked by session count. + +Channel classifier rules live in `apps/start/src/utils/source.ts`: + +- **Paid search / Paid social / Paid video** — when utm_source is + set and utm_medium is paid (`cpc`, `ppc`, `paid`, `paidsocial`, + `display`, `cpm`, `retargeting`, `sponsored`, `video`, etc.). + Recognises Google Ads, Meta, TikTok, Apple Search Ads, LinkedIn, + Reddit, X/Twitter Ads, Pinterest, Snapchat, Microsoft (Bing Ads), + YouTube. +- **Email** — `utm_medium=email` or `newsletter`. +- **Organic social** — referrer from a social site without paid UTM. +- **Organic search** — referrer type `search`. +- **Referral** — any other referrer. Also falls into here when only + `utm_campaign` is set (e.g. QR codes). +- **Direct** — nothing at all. + +Search keyword extraction tries `utm_term` first, then parses the +referrer URL for query params from Google / Bing / DuckDuckGo / Yahoo +/ Yandex / Baidu / Ecosia / Brave / Startpage. Google strips their +`q=` for organic so usually only utm_term will have a value there. + +Files: `apps/start/src/components/profiles/profile-source.tsx`, +`apps/start/src/utils/source.ts`, +new `profile.source` procedure in `packages/trpc/src/routers/profile.ts`. + +### 3.3 Platforms card (new) +Answers "does this user use web + app, and which one most?". Stacked +bar visualising session share + per-platform legend, rows show +session count, events, last seen, and: + +- Web rows: comma-separated list of every distinct browser + version + the user has used (e.g. "Chrome 124.0, Safari 17.4"). +- iOS / Android rows: current app version + build number + (`v4.12.0 · build 4120`), pulled via `argMax` over the + `__version` / `__buildNumber` event properties. + +Platform recognition: `sdk_name` header maps `web`/`js`/`browser` → +Web, `op-ios`/`swift` → iOS, `op-android`/`kotlin` → Android, +`react-native` → React Native, `node` → Server. Falls back to OS when +the SDK field is empty. + +Files: `apps/start/src/components/profiles/profile-platforms.tsx`, +new `profile.platforms` procedure in `packages/trpc/src/routers/profile.ts`. + +### 3.4 Metric strip changes +- **Conversion Events tile removed** (low signal on a single profile). +- **Total Session Time tile added** (replaces Bounce Rate, which was + a cohort metric that didn't make sense per-profile). Data source: + `round(sum(duration) / 60, 2)` in `getProfileMetrics`, rendered via + the existing `fancyMinutes` formatter so it shows "1h 24m" etc. +- Revenue tile no longer hides on zero — "$0" is real information. + +Files: `apps/start/src/components/profiles/profile-metrics.tsx`, +`packages/db/src/services/profile.service.ts`. + +### 3.5 Avatar with Gravatar fallback +Resolution order: +1. Pin Drop avatar if `profile.avatar` is a URL. +2. Gravatar — SHA-256 of the lowercased email, requested with + `d=404` so missing matches fail cleanly. +3. Facehash (the pre-existing deterministic initials fallback). + +Hashed in-browser via `crypto.subtle.digest` — no extra dependency. + +Files: `apps/start/src/components/profiles/profile-avatar.tsx`. + +### 3.6 Layout restructure +- **Source** + **Platforms** in the first content row after the + summary. +- **Activity heatmap** full-width, 3 months side-by-side (previously + 4 months, 2-col, made the card tall). +- **Latest Events** on the left of the next row; **Popular Events + + Most Visited Pages** stacked on the right (fills what used to be + an empty cell). Popular events and Most visited pages scroll + internally at `max-h-[220px]`. + +### 3.7 Header additions +- **Breadcrumb** above the name: `Profiles › Identified › [Name]` or + `Anonymous` depending on `isExternal`. +- **Power User badge** next to the name when `totalEvents ≥ 100` + (constant exported at the top of the route file for easy tuning). + +Files: `apps/start/src/routes/_app.$organizationId.$projectId.profiles.$profileId._tabs.tsx`, +`apps/start/src/routes/_app.$organizationId.$projectId.profiles.$profileId._tabs.index.tsx`. + +--- + +## 4. Groups redesign + +### 4.1 Groups list +- **"Add group" button removed.** Description updated to explain + groups flow in from Stripe / RevenueCat webhooks. The modal + tRPC + mutation remain so SDK integrations can still upsert. +- **Type column removed** (all groups are `team` in Pin Drop's setup). +- **Plan column added** — reads `properties.plan`, maps to the four + SKUs (Solo / Team / Team+ / Team Pro), coloured badge per plan. + Mapping supports `solo`, `free`, `team`, `team-plus`/`team+`, + `team-pro`/`pro`/`enterprise`. +- **Member count unified** with the Members tab (see bug fix 1.6). + +Files: `apps/start/src/routes/_app.$organizationId.$projectId.groups.tsx`, +`apps/start/src/components/groups/table/columns.tsx`, +`packages/db/src/services/group.service.ts`. + +### 4.2 Group detail — commercial info block +Replaced the old raw property dump with a curated 9-field grid: + +| Field | Source | +| ------------------ | ------------------------------------------- | +| Name | `group.name` | +| Plan | `properties.plan` → badge-mapped label | +| Team Members | Count from `getGroupMemberProfiles` | +| Owner | `properties.owner_name` | +| Created at | `group.createdAt` → DD/MM/YYYY | +| Subscription term | `properties.subscription_term` (Monthly / Annual / 24 months) | +| Deal amount | `properties.deal_amount` formatted as currency (`properties.currency` or USD) | +| Renewal date | `properties.renewal_date` → DD/MM/YYYY | +| Stripe ID | `properties.stripe_customer_id` | + +Any field with no value shows "—". + +Files: `apps/start/src/routes/_app.$organizationId.$projectId.groups_.$groupId._tabs.index.tsx`. + +### 4.3 Group detail — metrics +Replaced **First Seen** + **Last Seen** tiles with **Total Sessions** ++ **Total Session Time**. Total session aggregation joins the +sessions table against every member profile (so it covers sessions +not explicitly tagged with the group). + +Files: `packages/trpc/src/routers/group.ts` (extended `group.metrics`). + +### 4.4 Group detail — new cards +- **GroupPlatforms** (same visuals as ProfilePlatforms but aggregated + across every member) — shows team-wide web-vs-app split with app + versions + browser lists. +- **GroupTopMembers** (new) — top 5 members ranked by event count, + each row links to the profile. Flame icon badge. Empty state when + the group has no events. + +New tRPC procedures: `group.platforms`, `group.topMembers`. + +Files: `apps/start/src/components/groups/group-platforms.tsx`, +`apps/start/src/components/groups/group-top-members.tsx`, +`packages/trpc/src/routers/group.ts`. + +### 4.5 Group detail — layout + header +- Breadcrumb added (`Groups › [Name]`) matching the profile page. +- Activity heatmap full-width. +- Popular routes full-width (balances the row). + +--- + +## 5. Dev tooling (optional, dev-only) + +These are in the repo but don't affect production. Leave them in or +delete on deploy — either is fine. + +- **`LOCAL_SETUP.md`** — bring-up guide for a local dev instance + (Docker Desktop, Node 20+, pnpm, env, seed). +- **`scripts/seed-local.mjs`** — synthetic data generator. Creates + ~220 profiles with a realistic mix of web-only / iOS-only / + android-only / web+iOS / web+android / all-three usage patterns, + plus 17 fake teams split across the four SKUs with realistic + commercial properties (owner, deal amount, term, renewal, Stripe + ID). Assigns ~60% of identified profiles to a team with a weighted + pick that respects the declared team size. Solo groups get exactly + one member each. Also stamps each tracked event with the assigned + team so the Groups aggregates populate. + +--- + +## 6. Files touched + +**Modified (28):** +- `apps/start/src/components/events/event-list-item.tsx` +- `apps/start/src/components/groups/table/columns.tsx` +- `apps/start/src/components/overview/overview-metric-card.tsx` +- `apps/start/src/components/profiles/latest-events.tsx` +- `apps/start/src/components/profiles/most-events.tsx` +- `apps/start/src/components/profiles/popular-routes.tsx` +- `apps/start/src/components/profiles/profile-activity.tsx` +- `apps/start/src/components/profiles/profile-avatar.tsx` +- `apps/start/src/components/profiles/profile-metrics.tsx` +- `apps/start/src/components/profiles/profile-properties.tsx` +- `apps/start/src/components/profiles/table/columns.tsx` +- `apps/start/src/components/profiles/table/index.tsx` +- `apps/start/src/components/ui/data-table/data-table-hooks.tsx` +- `apps/start/src/components/ui/key-value-grid.tsx` +- `apps/start/src/routes/_app.$organizationId.$projectId.groups.tsx` +- `apps/start/src/routes/_app.$organizationId.$projectId.groups_.$groupId._tabs.index.tsx` +- `apps/start/src/routes/_app.$organizationId.$projectId.groups_.$groupId._tabs.tsx` +- `apps/start/src/routes/_app.$organizationId.$projectId.profiles.$profileId._tabs.index.tsx` +- `apps/start/src/routes/_app.$organizationId.$projectId.profiles.$profileId._tabs.tsx` +- `apps/start/src/routes/_app.$organizationId.$projectId.profiles._tabs.anonymous.tsx` +- `apps/start/src/routes/_app.$organizationId.$projectId.profiles._tabs.identified.tsx` +- `apps/start/src/routes/_app.$organizationId.$projectId.profiles._tabs.power-users.tsx` +- `apps/start/src/utils/casing.ts` +- `apps/start/src/utils/date.ts` +- `packages/db/src/services/group.service.ts` +- `packages/db/src/services/profile.service.ts` +- `packages/trpc/src/routers/group.ts` +- `packages/trpc/src/routers/profile.ts` + +**New (7):** +- `apps/start/src/components/groups/group-platforms.tsx` +- `apps/start/src/components/groups/group-top-members.tsx` +- `apps/start/src/components/profiles/profile-platforms.tsx` +- `apps/start/src/components/profiles/profile-source.tsx` +- `apps/start/src/utils/source.ts` +- `scripts/seed-local.mjs` (dev only) +- `LOCAL_SETUP.md` (dev only) + +Plus this file and `DEPLOYMENT.md` at the repo root. diff --git a/DEPLOYMENT.md b/DEPLOYMENT.md new file mode 100644 index 000000000..861f95c66 --- /dev/null +++ b/DEPLOYMENT.md @@ -0,0 +1,263 @@ +# Deployment — Pin Drop OpenPanel customisations + +Handoff doc for Justin to roll these UI + API changes onto the live +self-hosted OpenPanel instance. + +## What changed (high level) + +All changes are scoped to three packages + the scripts folder — no +schema migrations, no breaking API changes for the SDKs. + +**Dashboard (`apps/start`)** +- New profile detail cards: **Source** (referrer + UTM + channel + classification + search keyword extraction) and **Platforms** (Web + / iOS / Android split with app version + browser list). +- New profile metric tile: **Total Session Time** (replaces Bounce + Rate); Conversion Events tile removed. +- Unified Profile Information card: curated field set (name, email, + is subscriber, plan, team name, id, created, last known location). + "Properties" tab merged into the Profile tab; noisy / duplicate + keys (browser, os, model, path, referrer\_\*, biography, etc.) are + hidden but still available in ClickHouse. +- Sortable column headers on the Identified / Anonymous / Power Users + tables; all three tabs share one unified column set (Name / Plan / + Events / Session time / Country / OS / Model / First seen / Last + seen). Pagination bug fixed (wasn't counting deduped rows). +- Latest Events widget: replaced fragile self-measured scroll with a + fixed max-height; empty state added. Hides profile name when already + on that profile's page. +- Activity heatmap: 2 months by default on the profile page, 3 months + on the group page where the card is full-width. +- Avatar fallback chain: Pin Drop avatar → Gravatar (SHA-256 of the + email, zero extra deps) → facehash. +- Breadcrumbs added to profile + group detail pages; "Power user" + badge on profiles with ≥100 events. +- Group list: removed Type column, added Plan column with coloured + badges for the four Pin Drop SKUs (Solo / Team / Team+ / Team Pro). + "Add group" button removed — groups flow in from Stripe/RevenueCat. +- Group detail: redesigned Group Information block (Name / Plan / + Team Members / Owner / Created / Subscription term / Deal amount / + Renewal date / Stripe ID); new **Platforms** and **Power users in + this team** cards; Total Sessions + Total Session Time tiles. +- Layout fixes for overflowing text in the KeyValueGrid and metric + cards (the "biography bleeding into next cell" and "25 minutes ago + overflowing the tile" issues). Short timeAgo formatter ("5 mins + ago" instead of "5 minutes ago"). + +**API + packages** +- `packages/db/src/services/profile.service.ts` — new + `getEnrichedProfileList` that joins profiles with event + session + aggregates in a single query. Pagination count now uses `profiles + FINAL`. Added `totalSessionDuration` to `getProfileMetrics`. +- `packages/trpc/src/routers/profile.ts` — `profile.list` / + `profile.powerUsers` now accept `sortBy` + `sortDirection`; new + procedures: `profile.source`, `profile.platforms`. +- `packages/trpc/src/routers/group.ts` — new procedures: + `group.platforms`, `group.topMembers`; `group.metrics` now returns + total sessions + total session duration; `getGroupStats` unified + onto profile-based member count so the table matches the detail + page. +- `packages/db/src/services/group.service.ts` — getGroupStats split + into two parallel queries (members from profiles table, activity + from events table). + +**Dev-only** +- `scripts/seed-local.mjs` — synthetic data generator for local dev. + Not used in production. +- `LOCAL_SETUP.md` — doc for bringing up a local instance. Also not + used in production but harmless to leave in the repo. + +## Files changed + +28 modified, 7 new. Full list at the bottom of this file. + +## Handoff options + +Pick whichever suits Pin Drop's workflow best. (1) is the cleanest if +you have a GitHub fork of OpenPanel you control; (2) is fine as a +one-shot. + +### 1. Git branch / PR (preferred) + +From the folder that holds the local checkout on Andy's Mac: + +```bash +cd /Users/andy/Documents/Claude/Projects/OpenPanel/openpanel + +# Sanity check you're on main and clean otherwise. +git status + +# Branch it. +git checkout -b pindrop/dashboard-customisations + +# Stage + commit. The scope is large so a single commit is fine. +git add -A +git commit -m "feat(dashboard): Pin Drop customisations + +- Unified profile tables with sortable columns + pagination fix +- New Source / Platforms / Power-user cards on profile detail +- Group detail redesign (commercial info block + platforms + top members) +- Curated Profile Information card (merged tabs) +- Avatar fallback via Gravatar +- Seed + LOCAL_SETUP for local dev" + +# Push to your fork. +git push -u origin pindrop/dashboard-customisations +``` + +Justin then either opens a PR against your fork's `main` and merges, +or pulls the branch directly onto his deployment copy. + +### 2. Patch file + +If Pin Drop doesn't have a fork to push to, or Justin prefers to +apply a single file: + +```bash +cd /Users/andy/Documents/Claude/Projects/OpenPanel/openpanel + +# Bundle every change (including new files) into one patch. +git add -A +git diff --cached --binary > pindrop-dashboard.patch + +# Share pindrop-dashboard.patch (email / Slack / Drive). +``` + +Justin applies it on the production checkout: + +```bash +cd /path/to/openpanel +git checkout -b pindrop/dashboard-customisations +git apply --index pindrop-dashboard.patch +git commit -m "Apply Pin Drop dashboard customisations" +``` + +### 3. Whole-folder handover + +If Justin doesn't already have this OpenPanel checkout: zip the repo +(excluding `node_modules/`, `.git/` optional, `docker/data/`) and +send. Justin treats it as a new clone. + +## Deploying to the self-hosted instance + +### If Pin Drop runs OpenPanel from a built image in a registry + +Most self-hosted setups use `docker compose` pointing at pre-built +images (e.g. `lindesvard/openpanel-dashboard:latest`). To ship our +changes, Justin needs to build and push custom images: + +```bash +cd /path/to/openpanel-with-changes + +# Build the three container images that embed the code we changed. +# (Adjust tags to your registry.) +docker build -f apps/start/Dockerfile \ + -t ghcr.io/pindrop/openpanel-dashboard:custom-$(date +%Y%m%d) . + +docker build -f apps/api/Dockerfile \ + -t ghcr.io/pindrop/openpanel-api:custom-$(date +%Y%m%d) . + +docker build -f apps/worker/Dockerfile \ + -t ghcr.io/pindrop/openpanel-worker:custom-$(date +%Y%m%d) . + +# Push. +docker push ghcr.io/pindrop/openpanel-dashboard:custom-20260416 +docker push ghcr.io/pindrop/openpanel-api:custom-20260416 +docker push ghcr.io/pindrop/openpanel-worker:custom-20260416 +``` + +Then on the production server, update `docker-compose.yml` to point +at the new tags and: + +```bash +docker compose pull dashboard api worker +docker compose up -d dashboard api worker +``` + +ClickHouse / Postgres / Redis don't need restarting — no schema +migrations are required. + +### If Pin Drop builds from source on the server + +Simpler — pull the branch and rebuild in place: + +```bash +cd /path/to/openpanel +git fetch origin +git checkout pindrop/dashboard-customisations +git pull + +# Rebuild + restart the three services. Whichever build command +# your existing deploy script uses. +pnpm install +pnpm -r build +docker compose up -d --build dashboard api worker +``` + +## Rollback + +No data-layer changes are involved, so rollback is plain container +revert: + +```bash +# Point docker-compose back at the previous image tag or commit and +docker compose up -d dashboard api worker +``` + +## Smoke test after deploy + +1. Log into the live dashboard. +2. Navigate to **Profiles → Power Users** — confirm pagination works + past page 2 and the sortable columns reorder data when clicked. +3. Open any profile with recorded sessions — confirm the Source and + Platforms cards render. +4. Open the **Groups** page — confirm the Plan badge column is + populated and the "Add group" button is gone. +5. Open any group detail — confirm the Group Information block shows + the curated fields and the Platforms + Power users cards render. + +If any card shows a runtime error, check the dashboard container's +logs first (`docker compose logs dashboard --tail 100`); most +regressions would surface as a tRPC schema mismatch — a restart of +the api + worker usually clears it. + +## Full list of files touched + +### Modified +- apps/start/src/components/events/event-list-item.tsx +- apps/start/src/components/groups/table/columns.tsx +- apps/start/src/components/overview/overview-metric-card.tsx +- apps/start/src/components/profiles/latest-events.tsx +- apps/start/src/components/profiles/most-events.tsx +- apps/start/src/components/profiles/popular-routes.tsx +- apps/start/src/components/profiles/profile-activity.tsx +- apps/start/src/components/profiles/profile-avatar.tsx +- apps/start/src/components/profiles/profile-metrics.tsx +- apps/start/src/components/profiles/profile-properties.tsx +- apps/start/src/components/profiles/table/columns.tsx +- apps/start/src/components/profiles/table/index.tsx +- apps/start/src/components/ui/data-table/data-table-hooks.tsx +- apps/start/src/components/ui/key-value-grid.tsx +- apps/start/src/routes/_app.$organizationId.$projectId.groups.tsx +- apps/start/src/routes/_app.$organizationId.$projectId.groups_.$groupId._tabs.index.tsx +- apps/start/src/routes/_app.$organizationId.$projectId.groups_.$groupId._tabs.tsx +- apps/start/src/routes/_app.$organizationId.$projectId.profiles.$profileId._tabs.index.tsx +- apps/start/src/routes/_app.$organizationId.$projectId.profiles.$profileId._tabs.tsx +- apps/start/src/routes/_app.$organizationId.$projectId.profiles._tabs.anonymous.tsx +- apps/start/src/routes/_app.$organizationId.$projectId.profiles._tabs.identified.tsx +- apps/start/src/routes/_app.$organizationId.$projectId.profiles._tabs.power-users.tsx +- apps/start/src/utils/casing.ts +- apps/start/src/utils/date.ts +- packages/db/src/services/group.service.ts +- packages/db/src/services/profile.service.ts +- packages/trpc/src/routers/group.ts +- packages/trpc/src/routers/profile.ts + +### New +- apps/start/src/components/groups/group-platforms.tsx +- apps/start/src/components/groups/group-top-members.tsx +- apps/start/src/components/profiles/profile-platforms.tsx +- apps/start/src/components/profiles/profile-source.tsx +- apps/start/src/utils/source.ts +- scripts/seed-local.mjs (dev only) +- LOCAL_SETUP.md (dev only) diff --git a/LOCAL_SETUP.md b/LOCAL_SETUP.md new file mode 100644 index 000000000..afdc464d5 --- /dev/null +++ b/LOCAL_SETUP.md @@ -0,0 +1,143 @@ +# OpenPanel — Local Development Setup (for Andy) + +This is a step-by-step to get OpenPanel running on your Mac so you can see the +fixes in action against a local, empty database. Once it's working, we'll +figure out the path to load Pin Drop's live data into it. + +## 1. Prerequisites + +Install these once if you don't already have them: + +- **Docker Desktop** — https://www.docker.com/products/docker-desktop/ + After install, open it once so the docker daemon is running. +- **Node.js 20+** — easiest via https://github.com/nvm-sh/nvm: + `nvm install 20 && nvm use 20` +- **pnpm** (package manager this repo uses): + `npm install -g pnpm@10.6.2` + +Verify in a Terminal: + +```bash +docker --version # any recent version is fine +node --version # v20.x +pnpm --version # 10.x +``` + +## 2. Set up the project + +Open Terminal and `cd` into the repo: + +```bash +cd /Users/andy/Websites/openpanel +``` + +Create a `.env` file from the example. The defaults match the docker-compose +services so you don't need to change anything: + +```bash +cp .env.example .env +``` + +Install dependencies (this takes a few minutes the first time): + +```bash +pnpm install +``` + +## 3. Start the databases + +Postgres, ClickHouse and Redis all run in Docker. One command: + +```bash +pnpm dock:up +``` + +To check they're up: + +```bash +docker compose ps +``` + +You should see `op-db`, `op-kv`, and `op-ch` all listed as running. + +## 4. Run database migrations + +This creates the Postgres tables OpenPanel needs: + +```bash +pnpm migrate +``` + +ClickHouse migrations run automatically when the API boots. + +## 5. Start the apps + +In one terminal window, run everything in dev mode: + +```bash +pnpm dev +``` + +This starts: + +- **Dashboard** (the UI you'll see) — http://localhost:3000 +- **API** (event ingestion) — http://localhost:3333 +- **Worker** (background jobs) + +Open http://localhost:3000 in your browser and create an account. + +## 6. Sanity check + +Once you're signed in, you'll have an empty project. To see the fixes I'm +making take effect, you need at least a handful of profiles + events. Two +options: + +### Option A — Send a few test events + +In the dashboard, create a project and copy the client ID. Then from a +terminal: + +```bash +curl -X POST http://localhost:3333/track \ + -H "Content-Type: application/json" \ + -H "openpanel-client-id: " \ + -d '{ + "type": "track", + "payload": { + "name": "page_view", + "properties": { + "path": "/test", + "referrer": "https://google.com/search?q=pin+drop" + } + } + }' +``` + +Repeat a few times with different identified profiles to see the user table. + +### Option B — Load a snapshot from your live instance (safer than connecting directly) + +Once you can SSH into your self-hosted OpenPanel server, we can dump the +ClickHouse `events`, `profiles`, and `sessions` tables, then load them into +your local instance. Tell me when you have access and I'll write the +exact commands. + +## 7. Common issues + +- **Port already in use** — something else is running on 3000/3333/5432/6379/8123. + Stop it or change the port in `.env`. +- **`pnpm dev` errors about missing types** — run `pnpm codegen` once. +- **Want to wipe and start over** — `pnpm dock:down` then delete + `docker/data/` then `pnpm dock:up && pnpm migrate`. + +## 8. Where the fixes I'm making live + +- `packages/db/src/services/profile.service.ts` — Postgres/ClickHouse queries +- `packages/trpc/src/routers/profile.ts` — API for the user/power-user tables +- `apps/start/src/components/profiles/` — UI for the user pages +- `apps/start/src/components/overview/overview-metric-card.tsx` — metric tiles +- `apps/start/src/components/ui/key-value-grid.tsx` — key/value grid used in many places + +When we want to push fixes to your live self-hosted instance, the workflow is: +build the docker images from this branch, push them somewhere your server can +pull from, then `docker compose pull && docker compose up -d` on the server. diff --git a/apps/start/src/components/events/event-list-item.tsx b/apps/start/src/components/events/event-list-item.tsx index 5908d70b2..c37783274 100644 --- a/apps/start/src/components/events/event-list-item.tsx +++ b/apps/start/src/components/events/event-list-item.tsx @@ -8,11 +8,16 @@ import { pushModal } from '@/modals'; import { cn } from '@/utils/cn'; import { getProfileName } from '@/utils/getters'; -type EventListItemProps = IServiceEventMinimal | IServiceEvent; +type EventListItemProps = (IServiceEventMinimal | IServiceEvent) & { + /** Hide the profile name/link at the right of the row. Used in + * single-profile contexts (e.g. the profile detail page) where + * every row belongs to the same person and the name is redundant. */ + hideProfile?: boolean; +}; export function EventListItem(props: EventListItemProps) { const { organizationId, projectId } = useAppParams(); - const { createdAt, name, path, meta } = props; + const { createdAt, name, path, meta, hideProfile } = props; const profile = 'profile' in props ? props.profile : null; const renderName = () => { @@ -61,7 +66,7 @@ export function EventListItem(props: EventListItemProps) {
- {profile && ( + {profile && !hideProfile && ( = { + Web: 'bg-sky-500', + iOS: 'bg-slate-900', + Android: 'bg-emerald-500', + 'React Native': 'bg-indigo-500', + Server: 'bg-amber-500', + Unknown: 'bg-muted-foreground', +}; + +function PlatformIcon({ label }: { label: string }) { + const className = 'size-4 shrink-0 text-muted-foreground'; + if (label === 'Web') return ; + if (label === 'iOS' || label === 'Android' || label === 'React Native') + return ; + if (label === 'Server') return ; + return ; +} + +/** + * "Platforms" card for a group — aggregates every member's sessions + * by SDK so you can see the web-vs-app split for this team at a + * glance. Mirrors the profile version visually so the same mental + * model works at both levels. + */ +export function GroupPlatforms({ groupId, projectId }: Props) { + const trpc = useTRPC(); + const { data } = useSuspenseQuery( + trpc.group.platforms.queryOptions({ id: groupId, projectId }), + ); + + if (!data.length) { + return null; + } + + const totalSessions = data.reduce((acc, p) => acc + p.sessions, 0) || 1; + + return ( + + +
Platforms (team-wide)
+
+ +
+
+ {data.map((p) => { + const pct = (p.sessions / totalSessions) * 100; + return ( +
+ ); + })} +
+ +
    + {data.map((p) => { + const pct = Math.round((p.sessions / totalSessions) * 100); + return ( +
  • +
    + + +
    +
    + {p.label} + + {pct}% + +
    + {p.appVersion ? ( +
    + v{p.appVersion} + {p.buildNumber ? ` · build ${p.buildNumber}` : null} +
    + ) : p.browsers && p.browsers.length > 0 ? ( +
    + {p.browsers.join(', ')} +
    + ) : null} +
    +
    +
    +
    + {p.sessions.toLocaleString()}{' '} + {p.sessions === 1 ? 'session' : 'sessions'} +
    + {p.lastSeen ? ( +
    {timeAgoShort(new Date(p.lastSeen))}
    + ) : null} +
    +
  • + ); + })} +
+
+ + ); +} diff --git a/apps/start/src/components/groups/group-top-members.tsx b/apps/start/src/components/groups/group-top-members.tsx new file mode 100644 index 000000000..8e5af5dbd --- /dev/null +++ b/apps/start/src/components/groups/group-top-members.tsx @@ -0,0 +1,74 @@ +import { useSuspenseQuery } from '@tanstack/react-query'; +import { FlameIcon } from 'lucide-react'; +import { useTRPC } from '@/integrations/trpc/react'; +import { Widget, WidgetEmptyState } from '@/components/widget'; +import { WidgetHead } from '@/components/overview/overview-widget'; +import { ProfileAvatar } from '@/components/profiles/profile-avatar'; +import { ProjectLink } from '@/components/links'; +import { timeAgoShort } from '@/utils/date'; +import { getProfileName } from '@/utils/getters'; + +type Props = { + groupId: string; + projectId: string; +}; + +/** + * "Power users in this team" — the members generating the most events + * inside this group. Useful for account-managers: if the two biggest + * users churn, is the team at risk? Who are our champions worth + * nurturing? Top 5 only to keep the card compact — the full member + * list is one click away under the "Members" tab. + */ +export function GroupTopMembers({ groupId, projectId }: Props) { + const trpc = useTRPC(); + const { data } = useSuspenseQuery( + trpc.group.topMembers.queryOptions({ id: groupId, projectId, take: 5 }), + ); + + return ( + + +
+ + Power users in this team +
+
+ + {data.length === 0 ? ( + + ) : ( +
    + {data.map((m) => ( +
  • + + +
    +
    + {getProfileName(m.profile)} +
    +
    + Last active{' '} + {m.lastSeen ? timeAgoShort(new Date(m.lastSeen)) : '—'} +
    +
    +
    +
    +
    + {m.eventCount.toLocaleString()} +
    +
    events
    +
    +
  • + ))} +
+ )} +
+ ); +} diff --git a/apps/start/src/components/groups/table/columns.tsx b/apps/start/src/components/groups/table/columns.tsx index 87d7fedcf..607fe93fe 100644 --- a/apps/start/src/components/groups/table/columns.tsx +++ b/apps/start/src/components/groups/table/columns.tsx @@ -5,6 +5,33 @@ import { Link } from '@tanstack/react-router'; import type { ColumnDef } from '@tanstack/react-table'; import type { IServiceGroup } from '@openpanel/db'; +/** Friendly plan labels for the Groups table. Matches the names Pin + * Drop uses in Stripe/RevenueCat so the column reads the way support + * expects. Keys are lowercased so the lookup is case-insensitive. */ +/** Pin Drop's four SKUs. Team Pro is the enterprise tier — they're the + * same product, just two names depending on where you land. */ +const PLAN_LABELS: Record = { + solo: 'Solo', + free: 'Solo', + team: 'Team', + 'team-plus': 'Team+', + team_plus: 'Team+', + teamplus: 'Team+', + 'team+': 'Team+', + pro: 'Team Pro', + 'team-pro': 'Team Pro', + team_pro: 'Team Pro', + teampro: 'Team Pro', + enterprise: 'Team Pro', +}; + +const PLAN_BADGE_STYLE: Record = { + Solo: 'bg-slate-100 text-slate-700 border-slate-200', + Team: 'bg-blue-50 text-blue-700 border-blue-200', + 'Team+': 'bg-purple-50 text-purple-700 border-purple-200', + 'Team Pro': 'bg-amber-50 text-amber-700 border-amber-200', +}; + export type IServiceGroupWithStats = IServiceGroup & { memberCount: number; lastActiveAt: Date | null; @@ -40,11 +67,30 @@ export function useGroupColumns(): ColumnDef[] { ), }, { - accessorKey: 'type', - header: 'Type', - cell: ({ row }) => ( - {row.original.type} - ), + // Surface the subscription plan from the group's `properties.plan` + // (which your Stripe/RevenueCat webhook handler is expected to + // set on every `group` event). Falls back to the raw value — + // e.g. "Starter" — when we don't have a friendly mapping for it. + accessorKey: 'plan', + header: 'Plan', + cell: ({ row }) => { + const raw = (row.original.properties as any)?.plan as + | string + | undefined; + if (!raw) { + return ; + } + const label = PLAN_LABELS[raw.toLowerCase()] ?? raw; + const badgeClass = + PLAN_BADGE_STYLE[label] ?? 'bg-muted text-foreground border-border'; + return ( + + {label} + + ); + }, }, { accessorKey: 'memberCount', diff --git a/apps/start/src/components/overview/overview-metric-card.tsx b/apps/start/src/components/overview/overview-metric-card.tsx index a2ea8310f..4641b1123 100644 --- a/apps/start/src/components/overview/overview-metric-card.tsx +++ b/apps/start/src/components/overview/overview-metric-card.tsx @@ -10,7 +10,7 @@ import { Skeleton } from '../skeleton'; import { Tooltiper } from '../ui/tooltip'; import { fancyMinutes, useNumber } from '@/hooks/use-numer-formatter'; import { cn } from '@/utils/cn'; -import { formatDate, timeAgo } from '@/utils/date'; +import { formatDate, timeAgo, timeAgoShort } from '@/utils/date'; interface MetricCardProps { id: string; @@ -74,7 +74,9 @@ export function OverviewMetricCard({ if (!value) { return <>{'N/A'}; } - return <>{timeAgo(new Date(value))}; + // Use the short variant so "25 minutes ago" becomes "25 mins ago" — + // fits inside the metric tile without truncation. + return <>{timeAgoShort(new Date(value))}; } if (unit === 'min') { @@ -131,7 +133,12 @@ export function OverviewMetricCard({ - ))} - - {tab === 'profile' && profile && ( + {profile ? ( ({ - ...item, - event: { - ...profile, - ...profile.properties, - } as unknown as IServiceEvent, - }))} + data={data} /> - )} - - {tab === 'properties' && profile && ( - value !== undefined && value !== '') - .map(([key, value]) => ({ - name: key, - value: value, - event: { - ...profile, - ...profile.properties, - } as unknown as IServiceEvent, - }))} - /> - )} - {(!profile || !profile.properties) && ( - + ) : ( + )} ); diff --git a/apps/start/src/components/profiles/profile-source.tsx b/apps/start/src/components/profiles/profile-source.tsx new file mode 100644 index 000000000..6031a525c --- /dev/null +++ b/apps/start/src/components/profiles/profile-source.tsx @@ -0,0 +1,204 @@ +import { useSuspenseQuery } from '@tanstack/react-query'; +import { + Globe, + Link as LinkIcon, + Mail, + Megaphone, + Search, + Share2, + Video, +} from 'lucide-react'; +import { useTRPC } from '@/integrations/trpc/react'; +import { Widget } from '@/components/widget'; +import { WidgetHead } from '@/components/overview/overview-widget'; +import { SerieIcon } from '@/components/report-chart/common/serie-icon'; +import { formatDateTime } from '@/utils/date'; +import { classifySource, type SourceChannel } from '@/utils/source'; + +type Props = { + profileId: string; + projectId: string; +}; + +function ChannelIcon({ channel }: { channel: SourceChannel }) { + const Icon = + channel === 'paid-search' || channel === 'paid-social' + ? Megaphone + : channel === 'paid-video' + ? Video + : channel === 'organic-search' + ? Search + : channel === 'email' + ? Mail + : channel === 'organic-social' + ? Share2 + : channel === 'referral' + ? Globe + : LinkIcon; + return ; +} + +const CHANNEL_BADGE_CLASS: Record = { + 'paid-search': 'bg-blue-50 text-blue-700 border-blue-200', + 'paid-social': 'bg-purple-50 text-purple-700 border-purple-200', + 'paid-video': 'bg-pink-50 text-pink-700 border-pink-200', + 'organic-search': 'bg-emerald-50 text-emerald-700 border-emerald-200', + 'organic-social': 'bg-teal-50 text-teal-700 border-teal-200', + email: 'bg-amber-50 text-amber-700 border-amber-200', + referral: 'bg-slate-100 text-slate-700 border-slate-200', + direct: 'bg-muted text-muted-foreground border-border', +}; + +function ChannelBadge({ channel }: { channel: SourceChannel }) { + const readable = + channel === 'paid-search' + ? 'Paid search' + : channel === 'paid-social' + ? 'Paid social' + : channel === 'paid-video' + ? 'Paid video' + : channel === 'organic-search' + ? 'Organic search' + : channel === 'organic-social' + ? 'Organic social' + : channel === 'email' + ? 'Email' + : channel === 'referral' + ? 'Referral' + : 'Direct'; + return ( + + {readable} + + ); +} + +/** + * "Source" card — shown on the profile detail page. Surfaces where the + * profile originally arrived from (first session) plus every distinct + * source they've used since, ranked by session count. This takes over + * the acquisition-story UI job that used to live on the profile list + * table (which had a tiny Referrer column). + */ +export function ProfileSource({ profileId, projectId }: Props) { + const trpc = useTRPC(); + const { data } = useSuspenseQuery( + trpc.profile.source.queryOptions({ profileId, projectId }), + ); + + const firstClassified = data.first ? classifySource(data.first) : null; + + // Collapse identical sources (the server already groups by the main + // dimensions, but nothing stops you from having two separate distinct + // tags that classify the same way). Keep the highest count. + const ranked = data.sources + .map((s) => ({ ...s, classified: classifySource(s) })) + .sort((a, b) => b.count - a.count); + + return ( + + +
Source
+
+ + {firstClassified ? ( +
+
+ First seen via +
+
+ + + {firstClassified.platform ?? firstClassified.label} + + +
+ {firstClassified.keyword ? ( +
+ Search term:{' '} + "{firstClassified.keyword}" +
+ ) : null} + {firstClassified.campaign ? ( +
+ Campaign:{' '} + {firstClassified.campaign} +
+ ) : null} + {data.first?.entryPath ? ( +
+ Landed on{' '} + {data.first.entryPath} +
+ ) : null} +
+ {data.first?.createdAt + ? formatDateTime(new Date(data.first.createdAt)) + : null} +
+
+ ) : ( +
+
+ First seen via +
+
+ + Direct or untracked + +
+
+ No referrer or campaign data recorded for this profile yet. +
+
+ )} + + {ranked.length > 0 && ( +
+
+ All sources +
+
    + {ranked.map((s, i) => ( +
  • +
    + +
    +
    + + {s.classified.platform ?? s.classified.label} + + +
    + {s.classified.campaign ? ( +
    + Campaign:{' '} + + {s.classified.campaign} + +
    + ) : null} + {s.classified.keyword ? ( +
    + "{s.classified.keyword}" +
    + ) : null} +
    +
    +
    + {s.count.toLocaleString()}{' '} + {s.count === 1 ? 'session' : 'sessions'} +
    +
  • + ))} +
+
+ )} +
+ ); +} diff --git a/apps/start/src/components/profiles/table/columns.tsx b/apps/start/src/components/profiles/table/columns.tsx index e5f90e39d..c7d4c9c2b 100644 --- a/apps/start/src/components/profiles/table/columns.tsx +++ b/apps/start/src/components/profiles/table/columns.tsx @@ -1,16 +1,90 @@ import type { IServiceProfile } from '@openpanel/db'; -import type { ColumnDef } from '@tanstack/react-table'; +import type { ColumnDef, SortDirection } from '@tanstack/react-table'; +import { ArrowDown, ArrowUp, ArrowUpDown, CheckCircle2 } from 'lucide-react'; import { ProfileAvatar } from '../profile-avatar'; import { ColumnCreatedAt } from '@/components/column-created-at'; import { ProjectLink } from '@/components/links'; import { SerieIcon } from '@/components/report-chart/common/serie-icon'; import { getProfileName } from '@/utils/getters'; +import { cn } from '@/utils/cn'; + +// Enriched profile shape returned by `profile.list` / `profile.powerUsers`. +// Keeping these optional on `IServiceProfile` here (rather than re-declaring +// the full shape) so existing callers that still receive the non-enriched +// type continue to compile. +type EnrichedProfile = IServiceProfile & { + eventCount?: number; + sessionCount?: number; + totalDuration?: number; + lastSeen?: Date | string | null; + firstSeenActivity?: Date | string | null; + plan?: string | null; + isSubscriber?: boolean; +}; + +/** Format a duration in seconds into `1h 24m` / `3m 20s` / `45s`. */ +function formatDuration(seconds: number): string { + if (!seconds || seconds <= 0) { + return '—'; + } + const h = Math.floor(seconds / 3600); + const m = Math.floor((seconds % 3600) / 60); + const s = Math.floor(seconds % 60); + if (h > 0) { + return `${h}h ${m}m`; + } + if (m > 0) { + return `${m}m ${s}s`; + } + return `${s}s`; +} + +/** Header cell with a clickable sort affordance. Works with TanStack Table's + * manual sorting — the column is sortable if its `meta.sortable` is true. */ +function SortableHeader({ + label, + direction, + onToggle, + align = 'left', +}: { + label: string; + direction: false | SortDirection; + onToggle: () => void; + align?: 'left' | 'right'; +}) { + return ( + + ); +} export function useColumns(type: 'profiles' | 'power-users') { - const columns: ColumnDef[] = [ + const columns: ColumnDef[] = [ { accessorKey: 'name', - header: 'Name', + meta: { sortable: true, sortKey: 'name' }, + header: ({ column }) => ( + void} + /> + ), cell: ({ row }) => { const profile = row.original; return ( @@ -26,35 +100,94 @@ export function useColumns(type: 'profiles' | 'power-users') { }, }, { - accessorKey: 'referrer', - header: 'Referrer', + accessorKey: 'plan', + meta: { sortable: true, sortKey: 'plan' }, + header: ({ column }) => ( + void} + /> + ), cell({ row }) { - const { referrer, referrer_name } = row.original.properties; - const ref = referrer_name || referrer; + const { plan, isSubscriber } = row.original; + if (!plan && !isSubscriber) { + return ; + } return ( -
- - {ref} +
+ {isSubscriber && ( + + )} + {plan || 'subscriber'}
); }, }, + { + accessorKey: 'eventCount', + meta: { sortable: true, sortKey: 'eventCount' }, + header: ({ column }) => ( + void} + /> + ), + cell: ({ row }) => ( +
+ {row.original.eventCount?.toLocaleString() ?? 0} +
+ ), + }, + { + accessorKey: 'totalDuration', + meta: { sortable: true, sortKey: 'totalDuration' }, + header: ({ column }) => ( + void} + /> + ), + cell: ({ row }) => ( +
+ {formatDuration(row.original.totalDuration ?? 0)} +
+ ), + }, { accessorKey: 'country', - header: 'Country', + meta: { sortable: true, sortKey: 'country' }, + header: ({ column }) => ( + void} + /> + ), cell({ row }) { const { country, city } = row.original.properties; return (
- {city} + {city || country}
); }, }, { accessorKey: 'os', - header: 'OS', + meta: { sortable: true, sortKey: 'os' }, + header: ({ column }) => ( + void} + /> + ), cell({ row }) { const { os } = row.original.properties; return ( @@ -65,41 +198,70 @@ export function useColumns(type: 'profiles' | 'power-users') { ); }, }, - { - accessorKey: 'browser', - header: 'Browser', - cell({ row }) { - const { browser } = row.original.properties; - return ( -
- - {browser} -
- ); - }, - }, { accessorKey: 'model', - header: 'Model', + meta: { sortable: true, sortKey: 'model' }, + header: ({ column }) => ( + void} + /> + ), cell({ row }) { const { model, brand } = row.original.properties; + if (!model && !brand) { + return ; + } return (
- {brand} / {model} + {[brand, model].filter(Boolean).join(' / ')}
); }, }, { - accessorKey: 'createdAt', - header: 'First seen', + accessorKey: 'firstSeenActivity', + meta: { sortable: true, sortKey: 'firstSeenActivity' }, size: ColumnCreatedAt.size, + header: ({ column }) => ( + void} + /> + ), cell: ({ row }) => { - const item = row.original; - return {item.createdAt}; + // Prefer the earliest event for this profile (when they actually + // did something) and fall back to the profile row's createdAt. + const value = + row.original.firstSeenActivity ?? row.original.createdAt ?? null; + if (!value) { + return ; + } + return {value}; + }, + }, + { + accessorKey: 'lastSeen', + meta: { sortable: true, sortKey: 'lastSeen' }, + size: ColumnCreatedAt.size, + header: ({ column }) => ( + void} + /> + ), + cell: ({ row }) => { + const value = row.original.lastSeen ?? null; + if (!value) { + return ; + } + return {value}; }, }, { @@ -131,17 +293,9 @@ export function useColumns(type: 'profiles' | 'power-users') { }, ]; - if (type === 'power-users') { - columns.unshift({ - accessorKey: 'count', - header: 'Events', - cell: ({ row }) => { - const profile = row.original; - // @ts-expect-error - return
{profile.count}
; - }, - }); - } - + // All three tabs (Identified / Anonymous / Power Users) now share the + // same column set; `type` is currently only used to drive different + // default sorts on the route side. + void type; return columns; } diff --git a/apps/start/src/components/profiles/table/index.tsx b/apps/start/src/components/profiles/table/index.tsx index 9c534b313..875bfd06c 100644 --- a/apps/start/src/components/profiles/table/index.tsx +++ b/apps/start/src/components/profiles/table/index.tsx @@ -1,7 +1,12 @@ import type { IServiceProfile } from '@openpanel/db'; import type { UseQueryResult } from '@tanstack/react-query'; import { useNavigate } from '@tanstack/react-router'; -import type { PaginationState, Table, Updater } from '@tanstack/react-table'; +import type { + PaginationState, + SortingState, + Table, + Updater, +} from '@tanstack/react-table'; import { getCoreRowModel, useReactTable } from '@tanstack/react-table'; import { memo, useCallback } from 'react'; import { useColumns } from './columns'; @@ -9,6 +14,7 @@ import { DataTable } from '@/components/ui/data-table/data-table'; import { useDataTableColumnVisibility, useDataTablePagination, + useDataTableSort, } from '@/components/ui/data-table/data-table-hooks'; import { AnimatedSearchInput, @@ -52,6 +58,15 @@ export const ProfilesTable = memo( ); const { setPage, state: pagination } = useDataTablePagination(pageSize); + const defaultSortBy = + type === 'power-users' ? 'eventCount' : 'createdAt'; + const { sortBy, sortDirection, setSort } = useDataTableSort( + defaultSortBy, + 'desc', + ); + const sortingState: SortingState = sortBy + ? [{ id: sortBy, desc: sortDirection === 'desc' }] + : []; const { columnVisibility, setColumnVisibility, @@ -75,11 +90,20 @@ export const ProfilesTable = memo( }, state: { pagination, + sorting: sortingState, columnVisibility, columnOrder, }, onColumnVisibilityChange: setColumnVisibility, onColumnOrderChange: setColumnOrder, + onSortingChange: (updaterOrValue: Updater) => { + const next = + typeof updaterOrValue === 'function' + ? updaterOrValue(sortingState) + : updaterOrValue; + setPage(1); + setSort(next[0] ?? null); + }, onPaginationChange: (updaterOrValue: Updater) => { const nextPagination = typeof updaterOrValue === 'function' diff --git a/apps/start/src/components/ui/data-table/data-table-hooks.tsx b/apps/start/src/components/ui/data-table/data-table-hooks.tsx index a7fde2e5f..e64cb2fd3 100644 --- a/apps/start/src/components/ui/data-table/data-table-hooks.tsx +++ b/apps/start/src/components/ui/data-table/data-table-hooks.tsx @@ -3,10 +3,47 @@ import type { PaginationState, VisibilityState, } from '@tanstack/react-table'; -import { parseAsInteger, useQueryState } from 'nuqs'; +import { parseAsInteger, parseAsString, parseAsStringEnum, useQueryState } from 'nuqs'; import { useEffect, useState } from 'react'; import { useLocalStorage, useReadLocalStorage } from 'usehooks-ts'; +/** + * Sync a table's manual sort state to the URL query string. The `sortBy` + * value is passed straight into the tRPC input, so the set of allowed + * values should match the server-side `ProfileListSortBy` enum. + */ +export const useDataTableSort = ( + defaultSortBy: string | null = null, + defaultDirection: 'asc' | 'desc' = 'desc', +) => { + const [sortBy, setSortBy] = useQueryState( + 'sort', + parseAsString + .withDefault(defaultSortBy ?? '') + .withOptions({ clearOnDefault: true, history: 'push' }), + ); + const [direction, setDirection] = useQueryState( + 'dir', + parseAsStringEnum(['asc', 'desc']) + .withDefault(defaultDirection) + .withOptions({ clearOnDefault: true, history: 'push' }), + ); + const effectiveSortBy = sortBy || defaultSortBy || null; + return { + sortBy: effectiveSortBy, + sortDirection: direction, + setSort: (next: { id: string; desc: boolean } | null) => { + if (!next) { + setSortBy(''); + setDirection(defaultDirection); + return; + } + setSortBy(next.id); + setDirection(next.desc ? 'desc' : 'asc'); + }, + }; +}; + export const useDataTablePagination = (pageSize = 10) => { const [page, setPage] = useQueryState( 'page', diff --git a/apps/start/src/components/ui/key-value-grid.tsx b/apps/start/src/components/ui/key-value-grid.tsx index ea0fbd410..df3ac5746 100644 --- a/apps/start/src/components/ui/key-value-grid.tsx +++ b/apps/start/src/components/ui/key-value-grid.tsx @@ -69,11 +69,16 @@ export function KeyValueGrid({ ); }; + // Explicit `minmax(0, 1fr)` tracks are important here — the default + // `grid-cols-N` in Tailwind resolves to `minmax(auto, 1fr)`, which + // lets a long value (e.g. a long `biography` property) push its + // column wider than its share and bleed over into the neighbouring + // cell regardless of overflow-hidden on the child. const gridCols = { - 1: 'grid-cols-1', - 2: 'grid-cols-1 md:grid-cols-2', - 3: 'grid-cols-1 md:grid-cols-2 lg:grid-cols-3', - 4: 'grid-cols-1 md:grid-cols-2 lg:grid-cols-4', + 1: 'grid-cols-[minmax(0,1fr)]', + 2: 'grid-cols-[minmax(0,1fr)] md:grid-cols-[minmax(0,1fr)_minmax(0,1fr)]', + 3: 'grid-cols-[minmax(0,1fr)] md:grid-cols-[minmax(0,1fr)_minmax(0,1fr)] lg:grid-cols-[minmax(0,1fr)_minmax(0,1fr)_minmax(0,1fr)]', + 4: 'grid-cols-[minmax(0,1fr)] md:grid-cols-[minmax(0,1fr)_minmax(0,1fr)] lg:grid-cols-[minmax(0,1fr)_minmax(0,1fr)_minmax(0,1fr)_minmax(0,1fr)]', }; return ( @@ -84,7 +89,11 @@ export function KeyValueGrid({
+ {/* "Add group" button intentionally removed — in a real Pin Drop + * self-hosted setup groups flow in from Stripe / RevenueCat + * webhooks rather than being hand-created in this UI. Kept the + * modal + API so integrations can still upsert via the SDK. */} pushModal('AddGroup')}> - - Add group - - } className="mb-8" - description="Groups represent companies, teams, or other entities that events belong to." + description="Groups represent companies or paying teams. They're created automatically when your billing system (Stripe, RevenueCat) posts to the OpenPanel API." title="Groups" /> diff --git a/apps/start/src/routes/_app.$organizationId.$projectId.groups_.$groupId._tabs.index.tsx b/apps/start/src/routes/_app.$organizationId.$projectId.groups_.$groupId._tabs.index.tsx index 058afd3ca..0fdf6e5c9 100644 --- a/apps/start/src/routes/_app.$organizationId.$projectId.groups_.$groupId._tabs.index.tsx +++ b/apps/start/src/routes/_app.$organizationId.$projectId.groups_.$groupId._tabs.index.tsx @@ -2,6 +2,8 @@ import { useSuspenseQuery } from '@tanstack/react-query'; import { createFileRoute } from '@tanstack/react-router'; import FullPageLoadingState from '@/components/full-page-loading-state'; import { GroupMemberGrowth } from '@/components/groups/group-member-growth'; +import { GroupPlatforms } from '@/components/groups/group-platforms'; +import { GroupTopMembers } from '@/components/groups/group-top-members'; import { OverviewMetricCard } from '@/components/overview/overview-metric-card'; import { WidgetHead } from '@/components/overview/overview-widget'; import { MostEvents } from '@/components/profiles/most-events'; @@ -10,9 +12,91 @@ import { ProfileActivity } from '@/components/profiles/profile-activity'; import { KeyValueGrid } from '@/components/ui/key-value-grid'; import { Widget } from '@/components/widget'; import { useTRPC } from '@/integrations/trpc/react'; -import { formatDateTime } from '@/utils/date'; import { createProjectTitle } from '@/utils/title'; +/** Format a date as DD/MM/YYYY — what finance / support expect. */ +function formatDateShort(value: string | Date | null | undefined): string { + if (!value) return '—'; + const d = value instanceof Date ? value : new Date(value); + if (Number.isNaN(d.getTime())) return '—'; + const dd = String(d.getDate()).padStart(2, '0'); + const mm = String(d.getMonth() + 1).padStart(2, '0'); + const yyyy = d.getFullYear(); + return `${dd}/${mm}/${yyyy}`; +} + +const PLAN_LABEL: Record = { + solo: 'Solo', + free: 'Solo', + team: 'Team', + 'team-plus': 'Team+', + team_plus: 'Team+', + 'team+': 'Team+', + pro: 'Team Pro', + 'team-pro': 'Team Pro', + team_pro: 'Team Pro', + enterprise: 'Team Pro', +}; + +const SUBSCRIPTION_TERM_LABEL: Record = { + monthly: 'Monthly', + annual: 'Annual', + yearly: 'Annual', + '24m': '24 months', + '24-months': '24 months', + '24months': '24 months', +}; + +/** Curated + ordered fields for the Group Information grid. */ +function buildGroupInfoRows( + g: { + id: string; + name: string; + createdAt: Date | string; + properties: Record; + }, + memberCount: number, +) { + const p = g.properties as Record; + const planRaw = (p.plan as string | undefined)?.trim() || ''; + const plan = planRaw ? PLAN_LABEL[planRaw.toLowerCase()] ?? planRaw : '—'; + + const termRaw = (p.subscription_term as string | undefined)?.trim() || ''; + const term = termRaw + ? SUBSCRIPTION_TERM_LABEL[termRaw.toLowerCase()] ?? termRaw + : '—'; + + const dealRaw = p.deal_amount as number | string | undefined; + const deal = + typeof dealRaw === 'number' + ? new Intl.NumberFormat(undefined, { + style: 'currency', + currency: (p.currency as string) || 'USD', + maximumFractionDigits: 0, + }).format(dealRaw) + : typeof dealRaw === 'string' && dealRaw.trim() + ? dealRaw + : '—'; + + return [ + { name: 'name', value: g.name || '—' }, + { name: 'plan', value: plan }, + { name: 'teamMembers', value: memberCount.toLocaleString() }, + { name: 'owner', value: (p.owner_name as string) || '—' }, + { name: 'createdAt', value: formatDateShort(g.createdAt) }, + { name: 'subscriptionTerm', value: term }, + { name: 'dealAmount', value: deal }, + { + name: 'renewalDate', + value: formatDateShort(p.renewal_date as string | undefined), + }, + { + name: 'stripeId', + value: (p.stripe_customer_id as string) || '—', + }, + ]; +} + export const Route = createFileRoute( '/_app/$organizationId/$projectId/groups_/$groupId/_tabs/' )({ @@ -25,6 +109,19 @@ export const Route = createFileRoute( projectId: params.projectId, }) ), + context.queryClient.prefetchQuery( + context.trpc.group.platforms.queryOptions({ + id: params.groupId, + projectId: params.projectId, + }) + ), + context.queryClient.prefetchQuery( + context.trpc.group.topMembers.queryOptions({ + id: params.groupId, + projectId: params.projectId, + take: 5, + }) + ), ]); }, pendingComponent: FullPageLoadingState, @@ -63,8 +160,6 @@ function Component() { return null; } - const properties = g.properties as Record; - return (
{/* Metrics */} @@ -89,31 +184,29 @@ function Component() { />
)} - {/* Properties */} + {/* Group Information — curated grid matching the fields Pin Drop + * cares about for a customer record. Custom properties fall + * through below the curated block for anything extra the Stripe + * / RevenueCat webhook sends. Dates render DD/MM/YYYY for + * consistency with finance/support expectations. */}
@@ -123,27 +216,23 @@ function Component() { className="border-0" columns={3} copyable - data={[ - { name: 'id', value: g.id }, - { name: 'name', value: g.name }, - { name: 'type', value: g.type }, - { - name: 'createdAt', - value: formatDateTime(new Date(g.createdAt)), - }, - ...Object.entries(properties) - .filter(([, v]) => v !== undefined && v !== '') - .map(([k, v]) => ({ - name: k, - value: String(v), - })), - ]} + data={buildGroupInfoRows(g, m?.uniqueProfiles ?? 0)} />
- {/* Activity heatmap */} + {/* Platforms across the whole team */}
+ +
+ + {/* Power users in the team */} +
+ +
+ + {/* Activity heatmap — full width now that the row above is paired */} +
@@ -158,7 +247,7 @@ function Component() {
{/* Popular routes */} -
+
diff --git a/apps/start/src/routes/_app.$organizationId.$projectId.groups_.$groupId._tabs.tsx b/apps/start/src/routes/_app.$organizationId.$projectId.groups_.$groupId._tabs.tsx index 3ba26ac44..d30c1b227 100644 --- a/apps/start/src/routes/_app.$organizationId.$projectId.groups_.$groupId._tabs.tsx +++ b/apps/start/src/routes/_app.$organizationId.$projectId.groups_.$groupId._tabs.tsx @@ -9,10 +9,11 @@ import { useNavigate, useRouter, } from '@tanstack/react-router'; -import { Building2Icon, PencilIcon, Trash2Icon } from 'lucide-react'; +import { Building2Icon, ChevronRight, PencilIcon, Trash2Icon } from 'lucide-react'; import FullPageLoadingState from '@/components/full-page-loading-state'; import { PageContainer } from '@/components/page-container'; import { PageHeader } from '@/components/page-header'; +import { ProjectLink } from '@/components/links'; import { Button } from '@/components/ui/button'; import { Tabs, TabsList, TabsTrigger } from '@/components/ui/tabs'; import { usePageTabs } from '@/hooks/use-page-tabs'; @@ -98,6 +99,16 @@ function Component() { return ( + {/* Breadcrumb — clickable trail back to the Groups list so the + * navigation experience matches the profile detail page. */} + + diff --git a/apps/start/src/routes/_app.$organizationId.$projectId.profiles.$profileId._tabs.index.tsx b/apps/start/src/routes/_app.$organizationId.$projectId.profiles.$profileId._tabs.index.tsx index f3bd8b954..00d14e032 100644 --- a/apps/start/src/routes/_app.$organizationId.$projectId.profiles.$profileId._tabs.index.tsx +++ b/apps/start/src/routes/_app.$organizationId.$projectId.profiles.$profileId._tabs.index.tsx @@ -7,6 +7,8 @@ import { ProfileCharts } from '@/components/profiles/profile-charts'; import { ProfileGroups } from '@/components/profiles/profile-groups'; import { ProfileMetrics } from '@/components/profiles/profile-metrics'; import { ProfileProperties } from '@/components/profiles/profile-properties'; +import { ProfileSource } from '@/components/profiles/profile-source'; +import { ProfilePlatforms } from '@/components/profiles/profile-platforms'; import { useTRPC } from '@/integrations/trpc/react'; import { PAGE_TITLES, createProjectTitle } from '@/utils/title'; import { useSuspenseQuery } from '@tanstack/react-query'; @@ -43,6 +45,18 @@ export const Route = createFileRoute( projectId: params.projectId, }), ), + context.queryClient.prefetchQuery( + context.trpc.profile.source.queryOptions({ + profileId: params.profileId, + projectId: params.projectId, + }), + ), + context.queryClient.prefetchQuery( + context.trpc.profile.platforms.queryOptions({ + profileId: params.profileId, + projectId: params.projectId, + }), + ), ]); }, pendingComponent: FullPageLoadingState, @@ -115,12 +129,27 @@ function Component() { ) : null}
- {/* Heatmap / Activity */} + {/* Source (referrer + UTM attribution) */}
+ +
+ + {/* Platforms (web vs app split, current app version) */} +
+ +
+ + {/* Heatmap / Activity */} +
- {/* Latest events */} + {/* Latest events fills the left column; Popular events + Most + * visited pages stack on the right so the bottom row never has + * an awkward empty cell. */}
- {/* Most events */} -
+
-
- - {/* Popular routes */} -
diff --git a/apps/start/src/routes/_app.$organizationId.$projectId.profiles.$profileId._tabs.tsx b/apps/start/src/routes/_app.$organizationId.$projectId.profiles.$profileId._tabs.tsx index 315c80c38..6cdc7e95b 100644 --- a/apps/start/src/routes/_app.$organizationId.$projectId.profiles.$profileId._tabs.tsx +++ b/apps/start/src/routes/_app.$organizationId.$projectId.profiles.$profileId._tabs.tsx @@ -4,11 +4,19 @@ import { PageHeader } from '@/components/page-header'; import { ProfileAvatar } from '@/components/profiles/profile-avatar'; import { SerieIcon } from '@/components/report-chart/common/serie-icon'; import { Tabs, TabsList, TabsTrigger } from '@/components/ui/tabs'; +import { ProjectLink } from '@/components/links'; import { usePageTabs } from '@/hooks/use-page-tabs'; import { useTRPC } from '@/integrations/trpc/react'; import { getProfileName } from '@/utils/getters'; -import { useSuspenseQuery } from '@tanstack/react-query'; +import { useQuery, useSuspenseQuery } from '@tanstack/react-query'; import { Outlet, createFileRoute, useRouter } from '@tanstack/react-router'; +import { ChevronRight, Flame } from 'lucide-react'; + +/** Threshold above which a profile is surfaced as a "Power User" in + * the page header. Matches the gut-feel threshold the Power Users + * tab uses to rank profiles — anything above 100 events is clearly + * an engaged user. Tweak as you learn what's typical for Pin Drop. */ +const POWER_USER_EVENT_THRESHOLD = 100; export const Route = createFileRoute( '/_app/$organizationId/$projectId/profiles/$profileId/_tabs', @@ -37,6 +45,25 @@ function Component() { }), ); + // Metrics are already prefetched by the overview child route, so + // this usually resolves instantly from the cache. Used only to + // decide whether to show the "Power User" badge. + const metrics = useQuery( + trpc.profile.metrics.queryOptions({ + profileId, + projectId, + }), + ); + + const isPowerUser = + !!metrics.data && + metrics.data.totalEvents >= POWER_USER_EVENT_THRESHOLD; + + // Breadcrumb parent: identified profiles for external users, + // anonymous for everyone else. + const parentTab = profile.data?.isExternal ? 'identified' : 'anonymous'; + const parentLabel = profile.data?.isExternal ? 'Identified' : 'Anonymous'; + const { activeTab, tabs } = usePageTabs([ { id: '/$organizationId/$projectId/profiles/$profileId', @@ -55,6 +82,28 @@ function Component() { return ( + {/* Breadcrumb — clickable trail back to the Profiles list so you + * can get back to where you came from without relying on the + * browser back button. */} + + @@ -64,6 +113,12 @@ function Component() { ? getProfileName(profile.data, false) : 'User not identified'} + {isPowerUser && ( + + + Power user + + )}
} > diff --git a/apps/start/src/routes/_app.$organizationId.$projectId.profiles._tabs.anonymous.tsx b/apps/start/src/routes/_app.$organizationId.$projectId.profiles._tabs.anonymous.tsx index b2af79aad..762d37613 100644 --- a/apps/start/src/routes/_app.$organizationId.$projectId.profiles._tabs.anonymous.tsx +++ b/apps/start/src/routes/_app.$organizationId.$projectId.profiles._tabs.anonymous.tsx @@ -1,5 +1,8 @@ import { ProfilesTable } from '@/components/profiles/table'; -import { useDataTablePagination } from '@/components/ui/data-table/data-table-hooks'; +import { + useDataTablePagination, + useDataTableSort, +} from '@/components/ui/data-table/data-table-hooks'; import { useSearchQueryState } from '@/hooks/use-search-query-state'; import { useTRPC } from '@/integrations/trpc/react'; import { PAGE_TITLES, createEntityTitle } from '@/utils/title'; @@ -26,6 +29,7 @@ function Component() { const trpc = useTRPC(); const { page } = useDataTablePagination(50); const { debouncedSearch } = useSearchQueryState(); + const { sortBy, sortDirection } = useDataTableSort('createdAt', 'desc'); const query = useQuery( trpc.profile.list.queryOptions( { @@ -34,6 +38,8 @@ function Component() { take: 50, search: debouncedSearch, isExternal: false, + sortBy: (sortBy ?? undefined) as any, + sortDirection, }, { placeholderData: keepPreviousData, diff --git a/apps/start/src/routes/_app.$organizationId.$projectId.profiles._tabs.identified.tsx b/apps/start/src/routes/_app.$organizationId.$projectId.profiles._tabs.identified.tsx index d6265a6b4..449aeb76a 100644 --- a/apps/start/src/routes/_app.$organizationId.$projectId.profiles._tabs.identified.tsx +++ b/apps/start/src/routes/_app.$organizationId.$projectId.profiles._tabs.identified.tsx @@ -1,5 +1,8 @@ import { ProfilesTable } from '@/components/profiles/table'; -import { useDataTablePagination } from '@/components/ui/data-table/data-table-hooks'; +import { + useDataTablePagination, + useDataTableSort, +} from '@/components/ui/data-table/data-table-hooks'; import { useSearchQueryState } from '@/hooks/use-search-query-state'; import { useTRPC } from '@/integrations/trpc/react'; import { PAGE_TITLES, createEntityTitle } from '@/utils/title'; @@ -27,6 +30,7 @@ function Component() { const { page } = useDataTablePagination(50); const { debouncedSearch } = useSearchQueryState(); + const { sortBy, sortDirection } = useDataTableSort('createdAt', 'desc'); const query = useQuery( trpc.profile.list.queryOptions( @@ -36,6 +40,8 @@ function Component() { take: 50, search: debouncedSearch, isExternal: true, + sortBy: (sortBy ?? undefined) as any, + sortDirection, }, { placeholderData: keepPreviousData, diff --git a/apps/start/src/routes/_app.$organizationId.$projectId.profiles._tabs.power-users.tsx b/apps/start/src/routes/_app.$organizationId.$projectId.profiles._tabs.power-users.tsx index 12070a701..f3349b2c7 100644 --- a/apps/start/src/routes/_app.$organizationId.$projectId.profiles._tabs.power-users.tsx +++ b/apps/start/src/routes/_app.$organizationId.$projectId.profiles._tabs.power-users.tsx @@ -1,5 +1,9 @@ import { ProfilesTable } from '@/components/profiles/table'; -import { useDataTablePagination } from '@/components/ui/data-table/data-table-hooks'; +import { + useDataTablePagination, + useDataTableSort, +} from '@/components/ui/data-table/data-table-hooks'; +import { useSearchQueryState } from '@/hooks/use-search-query-state'; import { useTRPC } from '@/integrations/trpc/react'; import { PAGE_TITLES, createEntityTitle } from '@/utils/title'; import { keepPreviousData, useQuery } from '@tanstack/react-query'; @@ -24,12 +28,17 @@ function Component() { const { projectId } = Route.useParams(); const trpc = useTRPC(); const { page } = useDataTablePagination(50); + const { debouncedSearch } = useSearchQueryState(); + const { sortBy, sortDirection } = useDataTableSort('eventCount', 'desc'); const query = useQuery( trpc.profile.powerUsers.queryOptions( { cursor: page - 1, projectId, take: 50, + search: debouncedSearch, + sortBy: (sortBy ?? undefined) as any, + sortDirection, }, { placeholderData: keepPreviousData, diff --git a/apps/start/src/utils/casing.ts b/apps/start/src/utils/casing.ts index fa278cbb6..0ecd6d74e 100644 --- a/apps/start/src/utils/casing.ts +++ b/apps/start/src/utils/casing.ts @@ -1,9 +1,44 @@ +/** + * Acronyms that should always render upper-case rather than + * title-case when they appear as standalone words. Without this, + * `os` → `Os`, `url` → `Url`, `id` → `Id`, which reads oddly in + * key/value grids. + */ +const ACRONYMS = new Set([ + 'os', + 'url', + 'id', + 'api', + 'ip', + 'ui', + 'uri', + 'utm', + 'sdk', + 'ssl', + 'png', + 'jpg', + 'gif', + 'csv', + 'pdf', + 'uuid', + 'ios', +]); + export const camelCaseToWords = (str: string) => { - return str + const titled = str .replaceAll('_', ' ') .trim() .replaceAll(/([A-Z])/g, ' $1') .trim() - .replace(/^./, (str) => str.toUpperCase()) - .replaceAll(/\s./g, (str) => str.toUpperCase()); + .replace(/^./, (s) => s.toUpperCase()) + .replaceAll(/\s./g, (s) => s.toUpperCase()); + + // Upper-case any known acronym, operating on each space-separated + // word so we don't accidentally match a mid-word substring. + return titled + .split(' ') + .map((word) => + ACRONYMS.has(word.toLowerCase()) ? word.toUpperCase() : word, + ) + .join(' '); }; diff --git a/apps/start/src/utils/date.ts b/apps/start/src/utils/date.ts index 46f6d6384..3382964f1 100644 --- a/apps/start/src/utils/date.ts +++ b/apps/start/src/utils/date.ts @@ -54,6 +54,38 @@ export function timeAgo(date: Date, style?: FormatStyleName) { return ta.format(new Date(date), style); } +/** + * Shorter variant used in places where space is tight (e.g. the profile + * summary metric cards). Produces e.g. "25 mins ago" instead of + * "25 minutes ago", "2 mos ago" instead of "2 months ago". + * + * We don't rely on javascript-time-ago's `short` style alone — different + * locale bundles format the short style differently (some produce + * "5 min. ago", others still "5 minutes ago"). The post-processing + * below normalises both possibilities to a consistent terse output. + */ +export function timeAgoShort(date: Date) { + return ta + .format(new Date(date), 'short') + // Long forms → short forms. + .replace(/\bseconds?\b/g, 'secs') + .replace(/\bminutes?\b/g, 'mins') + .replace(/\bhours?\b/g, 'hrs') + .replace(/\bmonths?\b/g, 'mos') + .replace(/\byears?\b/g, 'yrs') + // Any "short" form that already abbreviates with a trailing period + // (library-locale-dependent) gets the period stripped. + .replace(/\bsec\./g, 'secs') + .replace(/\bmin\./g, 'mins') + .replace(/\bhr\./g, 'hrs') + .replace(/\bmo\./g, 'mos') + .replace(/\byr\./g, 'yrs') + // "1 mins" / "1 hrs" reads weird; drop the pluralising s after 1. + .replace(/\b1 (secs|mins|hrs|mos|yrs)\b/g, (_, unit) => + `1 ${unit.slice(0, -1)}`, + ); +} + export function formatTimeAgoOrDateTime(date: Date) { if (Math.abs(differenceInHours(date, new Date())) < 3) { return timeAgo(date); diff --git a/apps/start/src/utils/source.ts b/apps/start/src/utils/source.ts new file mode 100644 index 000000000..ab8db0595 --- /dev/null +++ b/apps/start/src/utils/source.ts @@ -0,0 +1,253 @@ +/** + * Given a session's referrer + UTM data, classify the acquisition source + * into a human-readable channel/platform and (when possible) extract the + * search keyword the visitor used. + * + * Rules are heuristic — there's no single universal standard for UTM + * tagging — but the patterns here cover the conventions used by the + * major ad platforms (Google / Meta / TikTok / Apple Search Ads / Reddit + * / LinkedIn / X / Pinterest / Microsoft / Snapchat) and the main search + * engines that still leak query terms in the referrer. + */ + +export type SourceChannel = + | 'paid-search' + | 'paid-social' + | 'paid-video' + | 'organic-search' + | 'organic-social' + | 'email' + | 'referral' + | 'direct'; + +export type SourceInput = { + referrer?: string | null; + referrerName?: string | null; + referrerType?: string | null; + utmSource?: string | null; + utmMedium?: string | null; + utmCampaign?: string | null; + utmTerm?: string | null; + utmContent?: string | null; +}; + +export type ClassifiedSource = { + channel: SourceChannel; + /** "Google Ads", "Meta Ads", "Organic search", "Direct", etc. */ + label: string; + /** Platform/brand name if we recognise one (e.g. "Google", "TikTok"). */ + platform: string | null; + /** Search query the visitor used, if we can recover one. */ + keyword: string | null; + /** The campaign name (utm_campaign) passed through verbatim. */ + campaign: string | null; +}; + +const PAID_MEDIUMS = new Set([ + 'cpc', + 'ppc', + 'paid', + 'paidsearch', + 'paid-search', + 'paid_search', + 'paidsocial', + 'paid-social', + 'paid_social', + 'display', + 'banner', + 'cpm', + 'retargeting', + 'sponsored', +]); + +// utm_source → friendly platform name. Matched case-insensitively and +// partially so `google_ads`, `googleads`, `google-ads` all resolve to +// "Google". +const PLATFORM_PATTERNS: Array<{ match: RegExp; platform: string }> = [ + { match: /google[_-]?ads?|adwords|^google$|\bgads\b/i, platform: 'Google' }, + { match: /facebook|\bfb\b|meta|instagram|\big\b/i, platform: 'Meta' }, + { match: /tiktok/i, platform: 'TikTok' }, + { match: /apple[_-]?search|\basa\b/i, platform: 'Apple Search Ads' }, + { match: /linkedin/i, platform: 'LinkedIn' }, + { match: /reddit/i, platform: 'Reddit' }, + { match: /twitter|\bx[_-]?ads\b|^x$/i, platform: 'X' }, + { match: /pinterest/i, platform: 'Pinterest' }, + { match: /snapchat|\bsnap\b/i, platform: 'Snapchat' }, + { match: /microsoft|\bbing[_-]?ads?\b/i, platform: 'Microsoft' }, + { match: /youtube/i, platform: 'YouTube' }, +]; + +const SEARCH_ENGINE_HOSTS: Record = { + google: ['q'], + bing: ['q'], + duckduckgo: ['q'], + yahoo: ['p', 'q'], + yandex: ['text'], + baidu: ['wd', 'word'], + ecosia: ['q'], + brave: ['q'], + startpage: ['q', 'query'], +}; + +function recognisePlatform(utmSource?: string | null): string | null { + if (!utmSource) { + return null; + } + for (const { match, platform } of PLATFORM_PATTERNS) { + if (match.test(utmSource)) { + return platform; + } + } + // Fall back to capitalising the raw utm_source so "newsletter" → "Newsletter". + const trimmed = utmSource.trim(); + if (!trimmed) { + return null; + } + return trimmed.charAt(0).toUpperCase() + trimmed.slice(1); +} + +/** + * Try to pull the search query out of a referrer URL. Mostly useful for + * search engines that still leak the query (Bing, DuckDuckGo, Yahoo) — + * Google has stripped `q=` from organic referrers for years, so for + * Google hits the keyword usually only appears if utm_term is set. + */ +function extractKeywordFromReferrer(referrer?: string | null): string | null { + if (!referrer) { + return null; + } + try { + const url = new URL(referrer); + const host = url.hostname.replace(/^www\./, '').toLowerCase(); + for (const [engine, params] of Object.entries(SEARCH_ENGINE_HOSTS)) { + if (host.includes(engine)) { + for (const param of params) { + const value = url.searchParams.get(param); + if (value) { + return value; + } + } + } + } + } catch { + return null; + } + return null; +} + +export function classifySource(input: SourceInput): ClassifiedSource { + const utmSource = input.utmSource?.trim() || ''; + const utmMedium = (input.utmMedium ?? '').trim().toLowerCase(); + const utmTerm = input.utmTerm?.trim() || ''; + const utmCampaign = input.utmCampaign?.trim() || null; + const referrer = input.referrer?.trim() || ''; + const referrerName = input.referrerName?.trim() || ''; + const referrerType = (input.referrerType ?? '').trim().toLowerCase(); + + const platform = recognisePlatform(utmSource) ?? referrerName ?? null; + const keyword = + utmTerm || extractKeywordFromReferrer(referrer) || null; + const isPaid = PAID_MEDIUMS.has(utmMedium) || utmMedium.includes('paid'); + + // Paid channels first — presence of utm_source + a paid medium is the + // strongest signal we have. + if (isPaid && utmSource) { + const platformName = platform ?? 'Unknown'; + if (/youtube/i.test(utmSource) || utmMedium === 'video') { + return { + channel: 'paid-video', + label: `${platformName} Ads`, + platform: platformName, + keyword, + campaign: utmCampaign, + }; + } + if (/(facebook|instagram|meta|tiktok|linkedin|reddit|twitter|x|pinterest|snapchat)/i.test(utmSource)) { + return { + channel: 'paid-social', + label: `${platformName} Ads`, + platform: platformName, + keyword, + campaign: utmCampaign, + }; + } + return { + channel: 'paid-search', + label: `${platformName} Ads`, + platform: platformName, + keyword, + campaign: utmCampaign, + }; + } + + // Email campaigns (newsletters, transactional link-outs). + if (utmMedium === 'email' || utmMedium === 'newsletter') { + return { + channel: 'email', + label: platform ? `Email · ${platform}` : 'Email', + platform, + keyword: null, + campaign: utmCampaign, + }; + } + + // Untagged organic-social (user shared a link to Twitter etc). + if (referrerType === 'social' || referrerName && /social/i.test(referrerName)) { + return { + channel: 'organic-social', + label: `Organic · ${referrerName || platform || 'Social'}`, + platform: referrerName || platform, + keyword: null, + campaign: utmCampaign, + }; + } + + // Organic search — referrer from a known search engine, or + // referrerType explicitly set by the ingestion pipeline. + if (referrerType === 'search') { + return { + channel: 'organic-search', + label: `Organic search · ${referrerName || 'Search engine'}`, + platform: referrerName || null, + keyword, + campaign: utmCampaign, + }; + } + + // Any other referrer = referral traffic (news site, blog, partner, …). + if (referrer || referrerName) { + return { + channel: 'referral', + label: `Referral · ${referrerName || referrer}`, + platform: referrerName || null, + keyword, + campaign: utmCampaign, + }; + } + + // No referrer and no platform, but the link was tagged with a campaign + // (e.g. a printed QR code or an SMS blast where `utm_campaign=spring24` + // is the only signal we get). Surface it as a referral so the campaign + // name is still visible — otherwise the source card would just say + // "Direct" and we'd lose the attribution. + if (utmCampaign || utmSource) { + return { + channel: 'referral', + label: utmSource + ? `Campaign · ${utmSource}` + : `Campaign · ${utmCampaign}`, + platform: platform, + keyword, + campaign: utmCampaign, + }; + } + + // Nothing at all. + return { + channel: 'direct', + label: 'Direct', + platform: null, + keyword: null, + campaign: null, + }; +} diff --git a/packages/db/src/services/group.service.ts b/packages/db/src/services/group.service.ts index baa79b8ff..9ef5221d1 100644 --- a/packages/db/src/services/group.service.ts +++ b/packages/db/src/services/group.service.ts @@ -258,33 +258,65 @@ export async function getGroupStats( return new Map(); } - const rows = await chQuery<{ - group_id: string; - member_count: number; - last_active_at: string; - }>(` - SELECT - g AS group_id, - uniqExact(profile_id) AS member_count, - max(created_at) AS last_active_at - FROM ${TABLE_NAMES.events} - ARRAY JOIN groups AS g - WHERE project_id = ${sqlstring.escape(projectId)} - AND g IN (${groupIds.map((id) => sqlstring.escape(id)).join(',')}) - AND profile_id != device_id - GROUP BY g - `); + const escapedIds = groupIds.map((id) => sqlstring.escape(id)).join(','); + + // Member count is derived from the **profiles** table so that it + // matches exactly what the "Members" tab on a group detail page + // shows. Previously we were counting distinct `profile_id`s from + // events instead, which produced a smaller number than the members + // list itself (profiles with a group assignment but no tagged events + // were missing) — making the two views disagree. + // + // `lastActiveAt` still comes from events because that's a genuine + // activity signal, not a membership one. + const [memberRows, activityRows] = await Promise.all([ + chQuery<{ group_id: string; member_count: number }>(` + SELECT + g AS group_id, + count() AS member_count + FROM ${TABLE_NAMES.profiles} FINAL + ARRAY JOIN groups AS g + WHERE project_id = ${sqlstring.escape(projectId)} + AND g IN (${escapedIds}) + GROUP BY g + `), + chQuery<{ group_id: string; last_active_at: string }>(` + SELECT + g AS group_id, + max(created_at) AS last_active_at + FROM ${TABLE_NAMES.events} + ARRAY JOIN groups AS g + WHERE project_id = ${sqlstring.escape(projectId)} + AND g IN (${escapedIds}) + GROUP BY g + `), + ]); - return new Map( - rows.map((r) => [ - r.group_id, - { - groupId: r.group_id, - memberCount: r.member_count, - lastActiveAt: r.last_active_at ? new Date(r.last_active_at) : null, - }, - ]) + const activityByGroup = new Map( + activityRows.map((r) => [r.group_id, r.last_active_at]), ); + + const result = new Map(); + for (const id of groupIds) { + result.set(id, { + groupId: id, + memberCount: 0, + lastActiveAt: null, + }); + } + for (const row of memberRows) { + const existing = result.get(row.group_id); + if (existing) { + existing.memberCount = row.member_count; + } + } + for (const [groupId, lastActive] of activityByGroup) { + const existing = result.get(groupId); + if (existing && lastActive) { + existing.lastActiveAt = new Date(lastActive); + } + } + return result; } export async function getGroupsByIds( diff --git a/packages/db/src/services/profile.service.ts b/packages/db/src/services/profile.service.ts index 7464b0bfc..15917370b 100644 --- a/packages/db/src/services/profile.service.ts +++ b/packages/db/src/services/profile.service.ts @@ -24,6 +24,9 @@ export interface IProfileMetrics { sessions: number; durationAvg: number; durationP90: number; + /** Total session time across every session the profile has had, + * expressed in minutes (so the UI can format it with `fancyMinutes`). */ + totalSessionDuration: number; totalEvents: number; uniqueDaysActive: number; bounceRate: number; @@ -52,10 +55,11 @@ export function getProfileMetrics(profileId: string, projectId: string) { SELECT count(*) as sessions FROM ${TABLE_NAMES.events} WHERE name = 'session_start' AND profile_id = ${sqlstring.escape(profileId)} AND project_id = ${sqlstring.escape(projectId)} ), duration AS ( - SELECT - round(avg(duration) / 1000 / 60, 2) as durationAvg, - round(quantilesExactInclusive(0.9)(duration)[1] / 1000 / 60, 2) as durationP90 - FROM ${TABLE_NAMES.events} + SELECT + round(avg(duration) / 1000 / 60, 2) as durationAvg, + round(quantilesExactInclusive(0.9)(duration)[1] / 1000 / 60, 2) as durationP90, + round(sum(duration) / 1000 / 60, 2) as totalSessionDuration + FROM ${TABLE_NAMES.events} WHERE name = 'session_end' AND duration != 0 AND profile_id = ${sqlstring.escape(profileId)} AND project_id = ${sqlstring.escape(projectId)} ), totalEvents AS ( @@ -88,8 +92,9 @@ export function getProfileMetrics(profileId: string, projectId: string) { (SELECT firstSeen FROM firstSeen) as firstSeen, (SELECT screenViews FROM screenViews) as screenViews, (SELECT sessions FROM sessions) as sessions, - (SELECT durationAvg FROM duration) as durationAvg, + (SELECT durationAvg FROM duration) as durationAvg, (SELECT durationP90 FROM duration) as durationP90, + (SELECT totalSessionDuration FROM duration) as totalSessionDuration, (SELECT totalEvents FROM totalEvents) as totalEvents, (SELECT uniqueDaysActive FROM uniqueDaysActive) as uniqueDaysActive, (SELECT bounceRate FROM bounceRate) as bounceRate, @@ -208,13 +213,181 @@ export async function getProfileList({ return data.map(transformProfile); } +// Columns the UI is allowed to sort the profile list by. `name` / `country` +// / `os` / `model` / `plan` / `createdAt` are resolved on the profile row +// itself; `lastSeen` / `firstSeenActivity` / `eventCount` come from the +// event aggregate and `totalDuration` / `sessionCount` from the session +// aggregate — we still sort those in the same query by joining the +// aggregates in as CTEs, so pagination works correctly either way. +export type ProfileListSortBy = + | 'name' + | 'country' + | 'os' + | 'model' + | 'plan' + | 'createdAt' + | 'lastSeen' + | 'firstSeenActivity' + | 'eventCount' + | 'totalDuration' + | 'sessionCount'; + +export type ProfileListSortDirection = 'asc' | 'desc'; + +interface GetEnrichedProfileListOptions extends GetProfileListOptions { + sortBy?: ProfileListSortBy; + sortDirection?: ProfileListSortDirection; +} + +export interface IEnrichedServiceProfile extends IServiceProfile { + eventCount: number; + sessionCount: number; + totalDuration: number; + lastSeen: Date | null; + firstSeenActivity: Date | null; + plan: string | null; + isSubscriber: boolean; +} + +/** + * Return a page of profiles joined with per-profile aggregates (event count, + * session count, total session duration, first/last activity, plan). This is + * the data source behind the unified "Identified / Anonymous / Power Users" + * tables in the dashboard. All three tables share the same columns and just + * differ by filter (`isExternal`) and default sort. + */ +export async function getEnrichedProfileList({ + take, + cursor, + projectId, + search, + isExternal, + sortBy = 'createdAt', + sortDirection = 'desc', +}: GetEnrichedProfileListOptions) { + const sortColumnMap: Record = { + name: 'lower(concat(p.first_name, p.last_name, p.email))', + country: "p.properties['country']", + os: "p.properties['os']", + model: "p.properties['model']", + plan: 'plan', + createdAt: 'p.created_at', + lastSeen: 'last_seen', + firstSeenActivity: 'first_seen_activity', + eventCount: 'event_count', + totalDuration: 'total_duration', + sessionCount: 'session_count', + }; + const orderByExpr = sortColumnMap[sortBy] ?? 'p.created_at'; + const direction = sortDirection === 'asc' ? 'ASC' : 'DESC'; + + const filterExternal = + isExternal !== undefined + ? `AND p.is_external = ${isExternal ? 'true' : 'false'}` + : ''; + const filterSearch = search + ? `AND (p.email ILIKE '%${search}%' OR p.first_name ILIKE '%${search}%' OR p.last_name ILIKE '%${search}%')` + : ''; + + const sql = ` + WITH event_stats AS ( + SELECT + profile_id, + count(*) AS event_count, + max(created_at) AS last_seen, + min(created_at) AS first_seen + FROM ${TABLE_NAMES.events} + WHERE profile_id != '' + AND project_id = ${sqlstring.escape(projectId)} + GROUP BY profile_id + ), + session_stats AS ( + SELECT + profile_id, + sum(duration) AS total_duration, + count() AS session_count + FROM ${TABLE_NAMES.sessions} FINAL + WHERE profile_id != '' + AND project_id = ${sqlstring.escape(projectId)} + GROUP BY profile_id + ) + SELECT + p.id AS id, + p.project_id AS project_id, + p.first_name AS first_name, + p.last_name AS last_name, + p.email AS email, + p.avatar AS avatar, + p.is_external AS is_external, + p.properties AS properties, + p.created_at AS created_at, + p.groups AS groups, + coalesce(p.properties['plan'], '') AS plan, + coalesce(e.event_count, 0) AS event_count, + e.last_seen AS last_seen, + e.first_seen AS first_seen_activity, + coalesce(s.total_duration, 0) AS total_duration, + coalesce(s.session_count, 0) AS session_count + FROM ${TABLE_NAMES.profiles} AS p FINAL + LEFT JOIN event_stats AS e ON p.id = e.profile_id + LEFT JOIN session_stats AS s ON p.id = s.profile_id + WHERE p.project_id = ${sqlstring.escape(projectId)} + ${filterExternal} + ${filterSearch} + ORDER BY ${orderByExpr} ${direction} + LIMIT ${take} OFFSET ${Math.max(0, (cursor ?? 0) * take)} + `; + + type EnrichedRow = IClickhouseProfile & { + event_count: number; + session_count: number; + total_duration: number; + last_seen: string | null; + first_seen_activity: string | null; + plan: string; + }; + + const rows = await chQuery(sql); + + return rows.map((row): IEnrichedServiceProfile => { + const base = transformProfile(row); + const plan = row.plan && row.plan.length > 0 ? row.plan : null; + // The SDK-facing convention: `is_subscriber` can be set either as an + // explicit boolean property, or implied from a non-free `plan`. This + // keeps the UI honest whether Pin Drop sends plan, is_subscriber, or + // both. + const rawIsSub = row.properties?.is_subscriber; + const explicit = + rawIsSub === 'true' || rawIsSub === '1' || (rawIsSub as any) === true; + const impliedFromPlan = + !!plan && plan !== 'free' && plan.toLowerCase() !== 'free'; + return { + ...base, + eventCount: Number(row.event_count) || 0, + sessionCount: Number(row.session_count) || 0, + totalDuration: Number(row.total_duration) || 0, + lastSeen: row.last_seen ? convertClickhouseDateToJs(row.last_seen) : null, + firstSeenActivity: row.first_seen_activity + ? convertClickhouseDateToJs(row.first_seen_activity) + : null, + plan, + isSubscriber: explicit || impliedFromPlan, + }; + }); +} + export async function getProfileListCount({ projectId, isExternal, search, }: Omit) { const { sb, getSql } = createSqlBuilder(); - sb.from = 'profiles'; + // Use FINAL so the count matches the deduped row set returned by + // getProfileList (which also reads from `profiles FINAL`). Without + // FINAL the ReplacingMergeTree's older row versions inflate the + // count, which causes the UI to show pages that have no data + // beyond the real distinct profile count. + sb.from = `${TABLE_NAMES.profiles} FINAL`; sb.select.count = 'count(id) as count'; sb.where.project_id = `project_id = ${sqlstring.escape(projectId)}`; sb.groupBy.project_id = 'project_id'; diff --git a/packages/trpc/src/routers/group.ts b/packages/trpc/src/routers/group.ts index 0711c941b..85f0a4680 100644 --- a/packages/trpc/src/routers/group.ts +++ b/packages/trpc/src/routers/group.ts @@ -10,6 +10,7 @@ import { getGroupStats, getGroupsByIds, getGroupTypes, + getProfiles, TABLE_NAMES, toNullIfDefaultMinDate, updateGroup, @@ -82,7 +83,7 @@ export const groupRouter = createTRPCRouter({ metrics: protectedProcedure .input(z.object({ id: z.string(), projectId: z.string() })) .query(async ({ input: { id, projectId } }) => { - const [eventData, profileData] = await Promise.all([ + const [eventData, profileData, sessionData] = await Promise.all([ chQuery<{ totalEvents: number; firstSeen: string; lastSeen: string }>(` SELECT count() AS totalEvents, @@ -98,11 +99,30 @@ export const groupRouter = createTRPCRouter({ WHERE project_id = ${sqlstring.escape(projectId)} AND has(groups, ${sqlstring.escape(id)}) `), + // Session aggregates come from the sessions table FINAL, joined + // on profile_id so we cover every session every member of the + // group has ever had — even if that specific session wasn't + // tagged with the group. + chQuery<{ totalSessions: number; totalSessionDuration: number }>(` + SELECT + count() AS totalSessions, + -- minutes + round(sum(duration) / 60, 2) AS totalSessionDuration + FROM ${TABLE_NAMES.sessions} FINAL + WHERE project_id = ${sqlstring.escape(projectId)} + AND profile_id IN ( + SELECT id FROM ${TABLE_NAMES.profiles} FINAL + WHERE project_id = ${sqlstring.escape(projectId)} + AND has(groups, ${sqlstring.escape(id)}) + ) + `), ]); return { totalEvents: eventData[0]?.totalEvents ?? 0, uniqueProfiles: profileData[0]?.uniqueProfiles ?? 0, + totalSessions: sessionData[0]?.totalSessions ?? 0, + totalSessionDuration: sessionData[0]?.totalSessionDuration ?? 0, firstSeen: toNullIfDefaultMinDate(eventData[0]?.firstSeen), lastSeen: toNullIfDefaultMinDate(eventData[0]?.lastSeen), }; @@ -140,6 +160,130 @@ export const groupRouter = createTRPCRouter({ `); }), + /** + * Platform breakdown for a group — web vs iOS vs Android, aggregated + * across every member's events. Mirrors `profile.platforms` so we + * can reuse the same card on the group page. + */ + platforms: protectedProcedure + .input(z.object({ id: z.string(), projectId: z.string() })) + .query(async ({ input: { id, projectId } }) => { + const rows = await chQuery<{ + sdk_name: string; + os: string; + browsers: string[]; + sessions: number; + events: number; + last_seen: string; + app_version: string; + build_number: string; + }>( + `SELECT + sdk_name, + any(os) as os, + arrayFilter( + x -> length(x) > 0, + arrayMap( + x -> trim(concat(tupleElement(x, 1), ' ', tupleElement(x, 2))), + groupUniqArray(tuple(browser, browser_version)) + ) + ) as browsers, + count(distinct session_id) as sessions, + count() as events, + max(created_at) as last_seen, + argMax(properties['__version'], created_at) as app_version, + argMax(properties['__buildNumber'], created_at) as build_number + FROM ${TABLE_NAMES.events} + WHERE project_id = ${sqlstring.escape(projectId)} + AND has(groups, ${sqlstring.escape(id)}) + GROUP BY sdk_name + ORDER BY sessions DESC`, + ); + + function friendlyLabel(sdkName: string, os: string) { + const s = (sdkName || '').toLowerCase(); + if (!s) { + if (os.toLowerCase().includes('ios')) return 'iOS'; + if (os.toLowerCase().includes('android')) return 'Android'; + return 'Unknown'; + } + if (s.includes('web') || s === 'js' || s.includes('browser')) { + return 'Web'; + } + if (s.includes('ios') || s.includes('swift')) return 'iOS'; + if (s.includes('android') || s.includes('kotlin')) return 'Android'; + if (s.includes('react-native') || s.includes('reactnative')) { + return 'React Native'; + } + if (s.includes('node')) return 'Server'; + return sdkName; + } + + return rows.map((r) => ({ + sdkName: r.sdk_name || null, + label: friendlyLabel(r.sdk_name, r.os), + sessions: Number(r.sessions) || 0, + events: Number(r.events) || 0, + lastSeen: r.last_seen, + appVersion: r.app_version || null, + buildNumber: r.build_number || null, + browsers: Array.isArray(r.browsers) ? r.browsers : [], + })); + }), + + /** + * Top N members of the group, ranked by event count. Powers the + * "Power users in this team" card on the group detail page. + */ + topMembers: protectedProcedure + .input( + z.object({ + id: z.string(), + projectId: z.string(), + take: z.number().default(5), + }), + ) + .query(async ({ input: { id, projectId, take } }) => { + // First pick the top profile_ids by event count for this group, + // then hydrate them into full profile records. + const rows = await chQuery<{ + profile_id: string; + event_count: number; + last_seen: string; + }>( + `SELECT + profile_id, + count() as event_count, + max(created_at) as last_seen + FROM ${TABLE_NAMES.events} + WHERE project_id = ${sqlstring.escape(projectId)} + AND has(groups, ${sqlstring.escape(id)}) + AND profile_id != '' + GROUP BY profile_id + ORDER BY event_count DESC + LIMIT ${take}`, + ); + + if (rows.length === 0) return []; + + const profiles = await getProfiles( + rows.map((r) => r.profile_id), + projectId, + ); + + return rows + .map((r) => { + const profile = profiles.find((p) => p.id === r.profile_id); + if (!profile) return null; + return { + profile, + eventCount: Number(r.event_count) || 0, + lastSeen: r.last_seen, + }; + }) + .filter((x): x is NonNullable => x !== null); + }), + listProfiles: protectedProcedure .input( z.object({ diff --git a/packages/trpc/src/routers/profile.ts b/packages/trpc/src/routers/profile.ts index 64159f493..18701999f 100644 --- a/packages/trpc/src/routers/profile.ts +++ b/packages/trpc/src/routers/profile.ts @@ -6,6 +6,7 @@ import { TABLE_NAMES, chQuery, createSqlBuilder, + getEnrichedProfileList, getProfileById, getProfileList, getProfileListCount, @@ -13,6 +14,23 @@ import { getProfiles, } from '@openpanel/db'; +const zSortBy = z + .enum([ + 'name', + 'country', + 'os', + 'model', + 'plan', + 'createdAt', + 'lastSeen', + 'firstSeenActivity', + 'eventCount', + 'totalDuration', + 'sessionCount', + ]) + .optional(); +const zSortDirection = z.enum(['asc', 'desc']).optional(); + import { createTRPCRouter, protectedProcedure } from '../trpc'; export const profileRouter = createTRPCRouter({ @@ -28,6 +46,165 @@ export const profileRouter = createTRPCRouter({ return getProfileMetrics(profileId, projectId); }), + /** + * Platform/client breakdown per profile: Web vs iOS vs Android vs + * React Native, with sessions, events, last-seen-per-platform and + * (when the SDK sends them) the most recent app version / build. + * Lets the UI answer "does this user use both the web and the app, + * and which one do they reach for most?". + */ + platforms: protectedProcedure + .input(z.object({ profileId: z.string(), projectId: z.string() })) + .query(async ({ input: { profileId, projectId } }) => { + const rows = await chQuery<{ + sdk_name: string; + os: string; + browsers: string[]; + sessions: number; + events: number; + last_seen: string; + app_version: string; + build_number: string; + }>( + `SELECT + sdk_name, + any(os) as os, + -- Return every distinct browser + version this profile has + -- used on this platform. Filter out empty strings (native + -- apps don't populate browser) and trim to avoid stray spaces + -- when either field is empty. + arrayFilter( + x -> length(x) > 0, + arrayMap( + x -> trim(concat(tupleElement(x, 1), ' ', tupleElement(x, 2))), + groupUniqArray(tuple(browser, browser_version)) + ) + ) as browsers, + count(distinct session_id) as sessions, + count() as events, + max(created_at) as last_seen, + argMax(properties['__version'], created_at) as app_version, + argMax(properties['__buildNumber'], created_at) as build_number + FROM ${TABLE_NAMES.events} + WHERE profile_id = ${sqlstring.escape(profileId)} + AND project_id = ${sqlstring.escape(projectId)} + GROUP BY sdk_name + ORDER BY sessions DESC`, + ); + + // Map raw sdk_name values to a friendly label. Falls back to OS + // when the SDK field is empty (older events / imports). + function friendlyLabel(sdkName: string, os: string) { + const s = (sdkName || '').toLowerCase(); + if (!s) { + if (os.toLowerCase().includes('ios')) return 'iOS'; + if (os.toLowerCase().includes('android')) return 'Android'; + return 'Unknown'; + } + if (s.includes('web') || s === 'js' || s.includes('browser')) { + return 'Web'; + } + if (s.includes('ios') || s.includes('swift')) return 'iOS'; + if (s.includes('android') || s.includes('kotlin')) return 'Android'; + if (s.includes('react-native') || s.includes('reactnative')) { + return 'React Native'; + } + if (s.includes('node')) return 'Server'; + return sdkName; + } + + return rows.map((r) => ({ + sdkName: r.sdk_name || null, + label: friendlyLabel(r.sdk_name, r.os), + sessions: Number(r.sessions) || 0, + events: Number(r.events) || 0, + lastSeen: r.last_seen, + appVersion: r.app_version || null, + buildNumber: r.build_number || null, + browsers: Array.isArray(r.browsers) ? r.browsers : [], + })); + }), + + /** + * Where did this profile come from? Returns the first recorded session + * (the acquisition event) plus a rollup of every distinct source the + * profile has arrived through since. Used by the "Source" card on the + * profile detail page. + */ + source: protectedProcedure + .input(z.object({ profileId: z.string(), projectId: z.string() })) + .query(async ({ input: { profileId, projectId } }) => { + const whereClause = `profile_id = ${sqlstring.escape(profileId)} AND project_id = ${sqlstring.escape(projectId)}`; + + const [firstRows, allRows] = await Promise.all([ + chQuery<{ + referrer: string; + referrer_name: string; + referrer_type: string; + utm_source: string; + utm_medium: string; + utm_campaign: string; + utm_term: string; + utm_content: string; + entry_path: string; + entry_origin: string; + created_at: string; + }>( + `SELECT referrer, referrer_name, referrer_type, + utm_source, utm_medium, utm_campaign, utm_term, utm_content, + entry_path, entry_origin, created_at + FROM ${TABLE_NAMES.sessions} FINAL + WHERE ${whereClause} + ORDER BY created_at ASC + LIMIT 1`, + ), + chQuery<{ + referrer_name: string; + referrer_type: string; + utm_source: string; + utm_medium: string; + utm_campaign: string; + count: number; + }>( + `SELECT referrer_name, referrer_type, + utm_source, utm_medium, utm_campaign, + count() as count + FROM ${TABLE_NAMES.sessions} FINAL + WHERE ${whereClause} + GROUP BY referrer_name, referrer_type, utm_source, utm_medium, utm_campaign + ORDER BY count DESC + LIMIT 10`, + ), + ]); + + const first = firstRows[0] ?? null; + return { + first: first + ? { + referrer: first.referrer, + referrerName: first.referrer_name, + referrerType: first.referrer_type, + utmSource: first.utm_source, + utmMedium: first.utm_medium, + utmCampaign: first.utm_campaign, + utmTerm: first.utm_term, + utmContent: first.utm_content, + entryPath: first.entry_path, + entryOrigin: first.entry_origin, + createdAt: first.created_at, + } + : null, + sources: allRows.map((r) => ({ + referrerName: r.referrer_name, + referrerType: r.referrer_type, + utmSource: r.utm_source, + utmMedium: r.utm_medium, + utmCampaign: r.utm_campaign, + count: Number(r.count) || 0, + })), + }; + }), + activity: protectedProcedure .input(z.object({ profileId: z.string(), projectId: z.string() })) .query(async ({ input: { profileId, projectId } }) => { @@ -81,11 +258,13 @@ export const profileRouter = createTRPCRouter({ take: z.number().default(50), search: z.string().optional(), isExternal: z.boolean().optional(), + sortBy: zSortBy, + sortDirection: zSortDirection, }), ) .query(async ({ input }) => { const [data, count] = await Promise.all([ - getProfileList(input), + getEnrichedProfileList(input), getProfileListCount(input), ]); return { @@ -97,46 +276,36 @@ export const profileRouter = createTRPCRouter({ }; }), + // Power Users shares the same enriched row shape as `list` — it's just + // "identified profiles, default-sorted by event_count DESC". Folding it + // onto the same query path means one set of columns for all three tabs. powerUsers: protectedProcedure .input( z.object({ projectId: z.string(), cursor: z.number().optional(), take: z.number().default(50), + search: z.string().optional(), + sortBy: zSortBy, + sortDirection: zSortDirection, }), ) - .query(async ({ input: { projectId, cursor, take } }) => { - const res = await chQuery<{ profile_id: string; count: number }>( - ` - SELECT profile_id, count(*) as count - FROM ${TABLE_NAMES.events} - WHERE - profile_id != '' - AND project_id = ${sqlstring.escape(projectId)} - GROUP BY profile_id - ORDER BY count() DESC - LIMIT ${take} ${cursor ? `OFFSET ${cursor * take}` : ''}`, - ); - const profiles = await getProfiles( - res.map((r) => r.profile_id), - projectId, - ); - - const data = res - .map((item) => { - return { - count: item.count, - ...(profiles.find((p) => p.id === item.profile_id)! ?? {}), - }; - }) - // Make sure we return actual profiles - .filter((item) => item.id); - + .query(async ({ input }) => { + const listInput = { + ...input, + isExternal: true, + sortBy: input.sortBy ?? ('eventCount' as const), + sortDirection: input.sortDirection ?? ('desc' as const), + }; + const [data, count] = await Promise.all([ + getEnrichedProfileList(listInput), + getProfileListCount(listInput), + ]); return { data, meta: { - count: data.length, - pageCount: take, + count, + pageCount: input.take, }, }; }), diff --git a/scripts/seed-local.mjs b/scripts/seed-local.mjs new file mode 100644 index 000000000..0e30d5624 --- /dev/null +++ b/scripts/seed-local.mjs @@ -0,0 +1,457 @@ +#!/usr/bin/env node +// Seed the local OpenPanel instance with fake profiles + events so we can +// verify the dashboard fixes against realistic-looking data. +// +// Usage: +// node scripts/seed-local.mjs +// +// Both come from the dashboard's onboarding screen ("Connect data"). + +const [, , CLIENT_ID, CLIENT_SECRET] = process.argv; + +if (!CLIENT_ID || !CLIENT_SECRET) { + console.error( + 'Usage: node scripts/seed-local.mjs ', + ); + process.exit(1); +} + +const API_URL = process.env.API_URL || 'http://localhost:3333'; +const PROFILE_COUNT = 220; // > 100 so we exceed two pages with pageSize=50 +const headers = { + 'Content-Type': 'application/json', + 'openpanel-client-id': CLIENT_ID, + 'openpanel-client-secret': CLIENT_SECRET, + // pretending to be a browser SDK so device/session creation works + 'user-agent': + 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0 Safari/537.36', + origin: 'http://localhost:3000', + referer: 'http://localhost:3000/', +}; + +const FIRST_NAMES = [ + 'Justin', 'Rakevet', 'Monika', 'Matthew', 'Val', 'Andy', 'Sara', 'Tom', + 'Priya', 'Lin', 'Olu', 'Kenji', 'Maeve', 'Hugo', 'Ada', 'Noor', 'Diego', + 'Nia', 'Bram', 'Eline', +]; +const LAST_NAMES = [ + 'Davies', 'Cohen', 'Patel', 'Nguyen', 'Olsson', 'Smith', 'Garcia', 'Singh', + 'Tanaka', 'Mensah', 'Rossi', 'Schmidt', 'Lee', 'Andersen', 'Costa', +]; +const COUNTRIES = [ + ['US', 'New York'], + ['US', 'Redmond'], + ['US', 'Orlando'], + ['GB', 'Durham'], + ['GB', 'West Bromwich'], + ['GB', 'London'], + ['IL', 'Tel Aviv'], + ['DE', 'Berlin'], + ['NL', 'Amsterdam'], + ['SE', 'Stockholm'], + ['BR', 'São Paulo'], + ['IN', 'Bangalore'], + ['JP', 'Tokyo'], +]; +// Profile of clients that can send events. The Platforms card uses the +// `sdk_name` header to group sessions — mix these so some seeded users +// are web-only, some app-only, and a chunk use both. +const CLIENTS = [ + { + id: 'web', + sdkName: 'web', + sdkVersion: '1.3.2', + userAgent: + 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0 Safari/537.36', + appVersion: null, + buildNumber: null, + os: 'macOS', + }, + { + id: 'ios', + sdkName: 'op-ios', + sdkVersion: '0.8.1', + userAgent: 'PinDrop/4.12.0 CFNetwork/1490 Darwin/23.1.0', + appVersion: '4.12.0', + buildNumber: '4120', + os: 'iOS', + }, + { + id: 'android', + sdkName: 'op-android', + sdkVersion: '0.7.4', + userAgent: + 'PinDrop/4.11.2 (Linux; Android 14; Pixel 8) okhttp/4.12.0', + appVersion: '4.11.2', + buildNumber: '4112', + os: 'Android', + }, +]; + +const REFERRERS = [ + // search engines with keywords (used by the search-keyword feature later) + 'https://www.google.com/search?q=pin+drop+app', + 'https://www.google.com/search?q=share+location+iphone', + 'https://www.bing.com/search?q=pindrop+meeting+place', + 'https://duckduckgo.com/?q=pin+drop+restaurants', + // direct referrers + 'https://twitter.com/somebody/status/123', + 'https://news.ycombinator.com/item?id=1234', + 'https://producthunt.com/posts/pin-drop', + '', // direct + '', // direct +]; +const PATHS = ['/', '/pricing', '/about', '/blog/launch', '/map', '/teams']; +const OS = ['Mac OS', 'Windows', 'iOS', 'Android']; +const BROWSERS = ['Chrome', 'Safari', 'Firefox', 'Edge']; +const DEVICES = ['desktop', 'mobile', 'tablet']; + +const choice = (arr) => arr[Math.floor(Math.random() * arr.length)]; +const randInt = (min, max) => + Math.floor(Math.random() * (max - min + 1)) + min; + +let sent = 0; +let failed = 0; + +async function postEvent(body, client = CLIENTS[0]) { + const res = await fetch(`${API_URL}/track`, { + method: 'POST', + headers: { + ...headers, + 'user-agent': client.userAgent, + // The API's `validateSdkRequest` keys off these headers to stamp + // `sdk_name` / `sdk_version` onto the event, which the Platforms + // card groups by. + 'openpanel-sdk-name': client.sdkName, + 'openpanel-sdk-version': client.sdkVersion, + }, + body: JSON.stringify(body), + }); + if (!res.ok) { + failed++; + if (failed <= 3) { + console.error(` ✗ ${res.status} ${await res.text()}`); + } + return; + } + sent++; + if (sent % 250 === 0) { + process.stdout.write(` …${sent} events sent\n`); + } +} + +/** Pick a per-event client based on the profile's usage pattern. */ +function clientFor(patternId) { + if (patternId === 'web-only') return CLIENTS[0]; + if (patternId === 'ios-only') return CLIENTS[1]; + if (patternId === 'android-only') return CLIENTS[2]; + // Multi-platform users: weight so `primary` ~60%, secondary ~30%, tertiary ~10%. + const r = Math.random(); + if (patternId === 'web-and-ios') { + return r < 0.6 ? CLIENTS[0] : CLIENTS[1]; + } + if (patternId === 'web-and-android') { + return r < 0.6 ? CLIENTS[0] : CLIENTS[2]; + } + if (patternId === 'all-three') { + return r < 0.5 ? CLIENTS[0] : r < 0.8 ? CLIENTS[1] : CLIENTS[2]; + } + return CLIENTS[0]; +} + +// Fake teams. In a real Pin Drop setup these would flow in from +// Stripe / RevenueCat webhooks as `group` events with type='team'; +// here we just upsert them the same way an integration would. +// Pin Drop's four SKUs. Member-count rules per SKU: +// - solo: exactly 1 member (individual customer) +// - team: 2-15 members +// - team+: 5+ members +// - team-pro: 5+ members (enterprise tier) +// The `size` here is the sales-record seat count; actual membership +// is driven by `assign_group` below and will approximate this. +const TEAMS = [ + // team-pro (5+, typically large enterprise) + { id: 'team-acme', name: 'Acme Travel Co', plan: 'team-pro', size: 48 }, + { id: 'team-umbrella', name: 'Umbrella Holdings', plan: 'team-pro', size: 94 }, + { id: 'team-stark', name: 'Stark Industries', plan: 'team-pro', size: 22 }, + { id: 'team-hooli', name: 'Hooli', plan: 'team-pro', size: 68 }, + { id: 'team-wayne', name: 'Wayne Enterprises', plan: 'team-pro', size: 41 }, + + // team-plus (5+) + { id: 'team-initech', name: 'Initech', plan: 'team-plus', size: 7 }, + { id: 'team-massive-dynamic', name: 'Massive Dynamic', plan: 'team-plus', size: 15 }, + { id: 'team-cyberdyne', name: 'Cyberdyne Systems', plan: 'team-plus', size: 19 }, + + // team (2-15) + { id: 'team-globex', name: 'Globex', plan: 'team', size: 12 }, + { id: 'team-pied-piper', name: 'Pied Piper', plan: 'team', size: 5 }, + { id: 'team-paper-street', name: 'Paper Street Soap Co', plan: 'team', size: 3 }, + { id: 'team-vandelay', name: 'Vandelay Industries', plan: 'team', size: 4 }, + + // solo (exactly 1 member per group — individual paying customer) + { id: 'solo-nomad-travel', name: 'Nomad Travel (Solo)', plan: 'solo', size: 1 }, + { id: 'solo-maeve-tours', name: "Maeve's Tours (Solo)", plan: 'solo', size: 1 }, + { id: 'solo-diego-photo', name: 'Diego Photo (Solo)', plan: 'solo', size: 1 }, + { id: 'solo-hugo-maps', name: 'Hugo Maps (Solo)', plan: 'solo', size: 1 }, + { id: 'solo-lin-guides', name: 'Lin Guides (Solo)', plan: 'solo', size: 1 }, +]; + +/** Split into buckets the assignment logic below uses. */ +const SOLO_GROUPS = TEAMS.filter((t) => t.plan === 'solo'); +const MULTI_MEMBER_GROUPS = TEAMS.filter((t) => t.plan !== 'solo'); + +// Helpers to give each seeded team realistic-looking commercial data. +const OWNERS = [ + 'Andy Smith', + 'Rakevet Cohen', + 'Hugo Lee', + 'Maeve Rossi', + 'Diego Costa', + 'Lin Tanaka', + 'Sara Mensah', + 'Priya Patel', + 'Olu Ade', + 'Noor Hassan', +]; +const TERMS = ['monthly', 'annual', '24m']; +const PLAN_BASE_ANNUAL = { + solo: 96, + team: 240, + 'team-plus': 720, + 'team-pro': 1800, +}; + +function dealAmountFor(plan, size, term) { + const perSeat = PLAN_BASE_ANNUAL[plan] ?? 100; + const multiplier = term === '24m' ? 2 : term === 'annual' ? 1 : 1 / 12; + return Math.round(perSeat * size * multiplier); +} + +function renewalFor(term) { + const now = new Date(); + const add = term === 'monthly' ? 30 : term === 'annual' ? 365 : 730; + const d = new Date(now.getTime() + add * 24 * 60 * 60 * 1000); + return d.toISOString().slice(0, 10); // YYYY-MM-DD +} + +async function seedGroups() { + for (let i = 0; i < TEAMS.length; i++) { + const team = TEAMS[i]; + const term = team.plan === 'solo' ? 'monthly' : choice(TERMS); + await postEvent({ + type: 'group', + payload: { + id: team.id, + type: 'team', + name: team.name, + properties: { + plan: team.plan, + member_count: team.size, + owner_name: OWNERS[i % OWNERS.length], + subscription_term: term, + deal_amount: dealAmountFor(team.plan, team.size, term), + currency: 'USD', + renewal_date: renewalFor(term), + stripe_customer_id: `cus_${team.id.replace(/[^a-z0-9]/gi, '').slice(-12)}`, + }, + }, + }); + } +} + +const USAGE_PATTERNS = [ + 'web-only', + 'web-only', + 'web-only', + 'ios-only', + 'android-only', + 'web-and-ios', + 'web-and-ios', + 'web-and-android', + 'all-three', +]; +function pickUsagePattern() { + return USAGE_PATTERNS[Math.floor(Math.random() * USAGE_PATTERNS.length)]; +} + +async function seedProfile(i) { + const firstName = choice(FIRST_NAMES); + const lastName = choice(LAST_NAMES); + const profileId = `seed-${i.toString().padStart(4, '0')}`; + const [country, city] = choice(COUNTRIES); + const browser = choice(BROWSERS); + const device = choice(DEVICES); + const pattern = pickUsagePattern(); + // Power-user distribution: most have 1-30 events, a few have 200-600. + const eventCount = i < 15 ? randInt(200, 600) : randInt(1, 30); + const isSubscriber = Math.random() < 0.25; + + // identify — send with the first client so the initial os/device + // doesn't bias the platforms card. + const identifyClient = clientFor(pattern); + await postEvent( + { + type: 'identify', + payload: { + profileId, + firstName, + lastName, + email: `${firstName.toLowerCase()}.${lastName.toLowerCase()}+${i}@example.com`, + properties: { + country, + city, + os: identifyClient.os, + browser, + device, + is_subscriber: isSubscriber, + plan: isSubscriber + ? choice(['team', 'team-plus', 'team-pro']) + : 'solo', + timezone: choice([ + 'Europe/London', + 'Europe/Amsterdam', + 'America/New_York', + 'America/Los_Angeles', + 'Asia/Tokyo', + ]), + locale: choice(['en-GB', 'en-US', 'de-DE', 'fr-FR', 'ja-JP']), + }, + }, + }, + identifyClient, + ); + + // Group assignment rules: + // • Solo groups must have exactly 1 member — the first seeded + // profiles claim a solo slot each, in order, until all solos + // are filled. This guarantees we have real "solo customer" data + // to look at. + // • The rest of the subscriber profiles are distributed across + // team / team-plus / team-pro groups roughly in proportion to + // each group's declared `size`, so the member counts in the + // Groups table look like realistic sales-record ratios. + // • Non-subscribers (free/solo-plan profiles) stay ungrouped so + // the Anonymous / unassigned cohort isn't empty. + let assignedTeamId = null; + if (isSubscriber) { + if (i < SOLO_GROUPS.length) { + // One-to-one mapping into a solo group. + assignedTeamId = SOLO_GROUPS[i].id; + } else { + // Weighted pick across multi-member groups so an enterprise + // team (size=94) gets ~20× more members than a small one (size=5). + const totalSize = MULTI_MEMBER_GROUPS.reduce( + (acc, t) => acc + t.size, + 0, + ); + let pick = Math.random() * totalSize; + for (const team of MULTI_MEMBER_GROUPS) { + pick -= team.size; + if (pick <= 0) { + assignedTeamId = team.id; + break; + } + } + } + if (assignedTeamId) { + await postEvent( + { + type: 'assign_group', + payload: { + groupIds: [assignedTeamId], + profileId, + }, + }, + identifyClient, + ); + } + } + + // events spread across the seeded profile's usage pattern + for (let e = 0; e < eventCount; e++) { + const referrer = choice(REFERRERS); + const client = clientFor(pattern); + await postEvent( + { + type: 'track', + payload: { + name: 'screen_view', + profileId, + // Stamp the team on every event so the Groups page's + // aggregations (member count, last-active, events by team, + // etc. — all of which read from events.groups) have data to + // work with. In a real setup you'd pass this from the SDK + // when you know the user is acting in a team context. + ...(assignedTeamId ? { groups: [assignedTeamId] } : {}), + properties: { + path: choice(PATHS), + referrer, + __referrer: referrer, + __query: referrer.includes('?') + ? Object.fromEntries(new URL(referrer).searchParams.entries()) + : {}, + country, + city, + os: client.os, + browser: client.id === 'web' ? browser : '', + device, + ...(client.appVersion + ? { + __version: client.appVersion, + __buildNumber: client.buildNumber, + } + : {}), + }, + }, + }, + client, + ); + } + + // a few revenue events for some users + if (isSubscriber && Math.random() < 0.5) { + await postEvent({ + type: 'track', + payload: { + name: 'revenue', + profileId, + properties: { + __revenue: randInt(900, 9900), // cents + plan: 'pro', + }, + }, + }); + } +} + +async function main() { + console.log( + `Seeding ${PROFILE_COUNT} profiles into ${API_URL} (client ${CLIENT_ID.slice(0, 8)}…)`, + ); + + // Seed the teams (groups) first so profile `assign_group` calls can + // reference them. In a real integration this is what a Stripe / + // RevenueCat webhook handler would do on customer create/update. + await seedGroups(); + console.log(`Seeded ${TEAMS.length} teams.`); + + // Run in small parallel batches so we don't melt the local API. + const concurrency = 4; + for (let i = 0; i < PROFILE_COUNT; i += concurrency) { + await Promise.all( + Array.from({ length: concurrency }, (_, k) => + i + k < PROFILE_COUNT ? seedProfile(i + k) : null, + ).filter(Boolean), + ); + process.stdout.write(`Profiles ${i + concurrency}/${PROFILE_COUNT}\r`); + } + console.log(`\nDone. ${sent} events sent, ${failed} failed.`); + console.log( + 'Open http://localhost:3000 → Profiles / Groups.', + ); +} + +main().catch((err) => { + console.error(err); + process.exit(1); +});