Skip to content

feat(plotnine): implement heatmap-cohort-retention#9079

Open
github-actions[bot] wants to merge 4 commits into
mainfrom
implementation/heatmap-cohort-retention/plotnine
Open

feat(plotnine): implement heatmap-cohort-retention#9079
github-actions[bot] wants to merge 4 commits into
mainfrom
implementation/heatmap-cohort-retention/plotnine

Conversation

@github-actions

Copy link
Copy Markdown
Contributor

Implementation: heatmap-cohort-retention - python/plotnine

Implements the python/plotnine version of heatmap-cohort-retention.

File: plots/heatmap-cohort-retention/implementations/python/plotnine.py

Parent Issue: #4570


🤖 impl-generate workflow

github-actions Bot added 2 commits June 20, 2026 01:01
Regen from quality 90. Addressed:
- Canvas: fixed to 6×6 in / dpi=400 → exact 2400×2400 square (was 16×9/dpi=300 → non-standard size)
- Save: plot-{THEME}.png (was plot.png, missing theme suffix)
- Theme-adaptive chrome: added ANYPLOT_THEME env var, INK/INK_SOFT/INK_MUTED/PAGE_BG/ELEVATED_BG tokens
- Colormap: switched from viridis-inspired custom palette to Imprint sequential (blue→green, #4467A3→#009E73)
- Font sizes: aligned to library guide (12pt title, 9pt subtitle, 10pt axis, 8pt ticks/legend)
- geom_text size: corrected to 2.8 mm (was 13, wrong scale)
- Tile borders: theme-adaptive PAGE_BG (was hardcoded #f8f9fa)
- Annotation color: theme-adaptive INK_MUTED
- Title: added python language segment (was missing)
- sys.path fix: remove script dir to prevent self-shadowing plotnine import
- Data: stronger cohort improvement trend (trend_bonus i×2.8 vs i×1.5) for clearer storytelling
@claude

claude Bot commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

AI Review - Attempt 1/3

Image Description

Light render (plot-light.png): Warm off-white background (#FAF8F1). Bold title "heatmap-cohort-retention · python · plotnine · anyplot.ai" fills ~80% of width (expected for the long mandated format). Italic subtitle in dark-soft ink. Y-axis labels "Jan 2024 (n=1,200)" through "Oct 2024 (n=1,020)" in dark secondary ink — all clearly readable. X-axis labels M0–M9 and "Months Since Signup" title in dark ink. Triangular staircase heatmap: Jan 2024 (top) has all 10 columns, Oct 2024 (bottom) has just M0. High-retention cells render in brand green (#009E73); lower-retention cells transition through teal to blue (#4467A3). Cell annotations show percentages in adaptive dark/light text. Small annotation "M4: +14pp Jan→Jun 2024" in the lower-right empty zone — visible but on the small side. Colorbar labeled "Retention %" with a themed legend box. All text is readable against the light background.

Dark render (plot-dark.png): Warm near-black background (#1A1A17). Title in light (#F0EFE8) ink — fully readable. Subtitle, y-axis labels, x-axis labels, and x-title all in light secondary text — readable. Legend has dark elevated background (#242420) with light text. Small annotation in lower-right zone in muted light text — visible. Data gradient (blue→green) is identical to the light render — only chrome flipped, Imprint data colors unchanged. Cell text adaptive coloring preserved. No dark-on-dark failures detected. All text is readable against the dark background.

Both paragraphs are required. A review that only describes one render is invalid.

Score: 89/100

Category Score Max
Visual Quality 29 30
Design Excellence 13 20
Spec Compliance 15 15
Data Quality 15 15
Code Quality 10 10
Library Mastery 7 10
Total 89 100

Visual Quality (29/30)

  • VQ-01: Text Legibility (7/8) — All font sizes explicitly set; title/subtitle/axis labels well-proportioned; annotation "M4: +14pp" at size=3.0mm reads slightly small in the large empty zone
  • VQ-02: No Overlap (6/6) — No overlapping elements; annotation correctly placed in empty triangular space
  • VQ-03: Element Visibility (6/6) — Tiles and cell annotations clearly visible; gradient contrast is strong
  • VQ-04: Color Accessibility (2/2) — Imprint palette is CVD-safe; adaptive cell text (dark/light) ensures contrast on all tiles
  • VQ-05: Layout & Canvas (4/4) — Square 2400×2400 canvas; heatmap fills ~60% of canvas; balanced margins
  • VQ-06: Axis Labels & Title (2/2) — "Months Since Signup" descriptive; y-axis labels include cohort size; colorbar labeled "Retention %"
  • VQ-07: Palette Compliance (2/2) — scale_fill_gradient uses Imprint stops (#4467A3#009E73); reversed direction semantically correct (green=high retention); backgrounds #FAF8F1/#1A1A17 correct; chrome theme-adaptive in both renders

Design Excellence (13/20)

  • DE-01: Aesthetic Sophistication (5/8) — Above well-configured default: custom Imprint palette, adaptive cell text coloring, subtitle framing the narrative. Held below 6 because the legend box floats in the large empty right zone reducing overall polish.
  • DE-02: Visual Refinement (4/6) — Grid removed, border removed, ticks removed, themed legend border — solid refinement. No anchoring spine on the heatmap means the triangular shape floats slightly against the warm background.
  • DE-03: Data Storytelling (4/6) — Subtitle and M4 annotation together narrate the product-maturity story; triangular shape inherently communicates the cohort lifecycle; color gradient encodes retention quality at a glance.

Spec Compliance (15/15)

  • SC-01: Plot Type (5/5) — Correct triangular cohort retention heatmap
  • SC-02: Required Features (4/4) — Triangular shape ✓, period 0=100% ✓, cell text annotations ✓, cohort sizes in y-labels ✓, colorbar legend ✓; M0/M1 labels acceptable abbreviation for Month 0/Month 1
  • SC-03: Data Mapping (3/3) — X=periods, Y=cohorts (ordered), fill=retention_rate
  • SC-04: Title & Legend (3/3) — Title exactly "heatmap-cohort-retention · python · plotnine · anyplot.ai"; colorbar labeled "Retention %"

Data Quality (15/15)

  • DQ-01: Feature Coverage (6/6) — 10 cohorts with varying column counts; declining retention within cohorts; improvement trend across newer cohorts
  • DQ-02: Realistic Context (5/5) — SaaS monthly cohort retention Jan–Oct 2024; realistic cohort sizes 980–1,450
  • DQ-03: Appropriate Scale (4/4) — Exponential decay with noise; values 8–100%; product-maturity trend_bonus plausible

Code Quality (10/10)

  • CQ-01: KISS Structure (3/3) — Flat structure: imports → chrome tokens → data → plot composition → save
  • CQ-02: Reproducibility (2/2) — np.random.seed(42) set before data generation
  • CQ-03: Clean Imports (2/2) — All 14 plotnine imports used; os, numpy, pandas all used
  • CQ-04: Code Elegance (2/2) — Clean ggplot grammar; no over-engineering; no fake UI elements
  • CQ-05: Output & API (1/1) — plot.save(f"plot-{THEME}.png", dpi=400, width=6, height=6, units="in") correct

Library Mastery (7/10)

  • LM-01: Idiomatic Usage (4/5) — Correct ggplot grammar of graphics; pd.Categorical with ordered=True for y-axis ordering; scale_color_identity() for pre-computed text colors — very idiomatic
  • LM-02: Distinctive Features (3/5) — scale_color_identity() paired with a pre-computed color column for adaptive cell text is a distinctively ggplot2/plotnine pattern not easily replicated in other libraries

Score Caps Applied

  • None applied

Strengths

  • Triangular heatmap shape correctly implemented — staircase pattern clearly shows cohort lifecycle
  • scale_color_identity() with pre-computed text color column is elegant, idiomatic plotnine
  • Cohort sizes in y-axis labels, period 0=100%, colorbar — all spec requirements met
  • SaaS monthly cohort scenario is realistic, neutral, and business-plausible
  • Imprint palette applied correctly (#4467A3#009E73 sequential, semantically reversed for high=green); all chrome theme-adaptive
  • Subtitle + M4 annotation together provide genuine data storytelling
  • Full spec compliance with perfect data quality

Weaknesses

  • Annotation text "M4: +14pp Jan→Jun 2024" (size=3.0mm) reads small in the large empty lower-right zone — bump to 3.5–4.0mm for better legibility
  • Legend box floats in large empty right zone, increasing visual whitespace imbalance — consider legend_position=("right", 0.85) or tightening the right margin so the legend sits closer to the data
  • No visual anchoring for the triangular heatmap — a hairline bottom spine or subtle border around the tile grid would give the floating triangular shape more definition
  • LM-02: scale_color_identity is a good distinctive feature but the implementation could push further (e.g., use plotnine's geom_text positioning arguments or theme(strip_*) for cohort grouping)

Issues Found

  1. DE-01 (5/8): Aesthetic sophistication is above default but held back by the legend floating in the empty triangular zone and the lack of any visual anchor for the heatmap grid.
    • Fix: Reposition legend closer to data or add a subtle panel_border to the heatmap grid; consider tightening plot_margin to reduce the empty right-side whitespace.
  2. VQ-01 (7/8): Annotation size (3.0mm ≈ 8.5pt) is slightly small relative to the canvas.
    • Fix: Increase annotation size from 3.0 to 3.5–4.0 to improve readability in the empty zone.
  3. DE-02 (4/6): Heatmap floats without visual anchoring on the warm background.
    • Fix: Add a subtle hairline panel_border=element_rect(color=INK_SOFT, fill=None, size=0.3) to give the grid a contained look, or add axis_line_x at the bottom for a grounding spine.

AI Feedback for Next Attempt

Score is 89 — 1 point below the Attempt 1 threshold of 90. To close the gap: (1) increase annotation size from 3.0 to 3.5–4.0mm; (2) reposition the legend or tighten the right margin so it doesn't float in an oversized empty zone; (3) add a subtle panel_border or bottom spine to anchor the triangular heatmap visually. These three targeted fixes should push DE-01 to 6 and VQ-01 to 8, reaching 91–92.

Verdict: REJECTED

@github-actions github-actions Bot added quality:89 Quality score 89/100 ai-rejected Quality not OK, triggers update labels Jun 20, 2026
@github-actions github-actions Bot added ai-attempt-1 First repair attempt and removed ai-rejected Quality not OK, triggers update labels Jun 20, 2026
@github-actions

Copy link
Copy Markdown
Contributor Author

🔧 Repair Attempt 1/4

Applied fixes based on AI review feedback.

Status: Repair completed, re-triggering review...


🤖 impl-repair

@claude

claude Bot commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

AI Review - Attempt 2/3

Image Description

Light render (plot-light.png): Warm off-white background (#FAF8F1) — correct light surface. The triangular heatmap shows 10 cohorts (Jan–Oct 2024) with a green-to-blue gradient (green = high retention, blue = low). All period-0 cells show 100% in bright teal/green. Cell text annotations show retention percentages with dynamic contrast (dark ink on bright tiles, light text on dark blue tiles). Two black annotate("rect") borders highlight the M4 cells for Jan 2024 and Jun 2024. A small annotation "M4: +14pp Jan→Jun 2024" appears in the lower-right data area. Y-axis labels with cohort sizes are readable in dark ink. CRITICAL: The title "heatmap-cohort-retention · python · plotnine · anyplot." is clipped at the right canvas edge — the ".ai" suffix is visibly cut off. The subtitle "SaaS monthly cohorts — newer signups retain significantly better month over m..." is also clipped. All other text is readable against the light background.

Dark render (plot-dark.png): Warm near-black background (#1A1A17) — correct dark surface. Data colors are identical to the light render (green at 100%, blue at low retention — only chrome flips). Y-axis labels, x-axis labels (M0–M9), and legend text are light-colored (#B8B7B0) — no dark-on-dark failures detected. The annotate("rect") highlight borders render in light ink (#F0EFE8) on the dark background. Legend box uses elevated dark background (#242420). Same AR-09 clipping: title and subtitle cut off at the right canvas edge in both renders.

Score: 0/100

(AR-09 auto-reject: title and subtitle clipped at canvas border — see details below)

Category Score Max
Visual Quality 22 30
Design Excellence 13 20
Spec Compliance 14 15
Data Quality 15 15
Code Quality 10 10
Library Mastery 7 10
Total (pre-AR-09) 81 100
Total (post-AR-09) 0 100

Visual Quality (22/30)

  • VQ-01: Text Legibility (5/8) — Font sizes explicitly set (title 12pt, subtitle 9pt, axes 8–10pt, geom_text 2.8mm); but title/subtitle clipped at right edge reduces effective legibility
  • VQ-02: No Overlap (5/6) — No significant overlaps; cell text fits in tiles with minor tightness in corner cells
  • VQ-03: Element Visibility (6/6) — Tiles and cell annotations clearly visible; dynamic text color ensures contrast across gradient
  • VQ-04: Color Accessibility (2/2) — CVD-safe Imprint palette; dynamic cell text color adds redundant contrast signal
  • VQ-05: Layout & Canvas (0/4) — AR-09: Title and subtitle extend beyond right canvas border; pixels are visibly missing
  • VQ-06: Axis Labels & Title (2/2) — "Months Since Signup" descriptive; Y-axis empty but cohort labels are self-descriptive
  • VQ-07: Palette Compliance (2/2) — Continuous cmap uses Imprint endpoints (#4467A3 → #009E73); backgrounds correct (#FAF8F1 light, #1A1A17 dark); chrome theme-correct in both renders

Design Excellence (13/20)

  • DE-01: Aesthetic Sophistication (5/8) — Strong design above defaults: Imprint gradient, dynamic cell text color, highlighted comparison cells, trend annotation
  • DE-02: Visual Refinement (4/6) — Grid removed, axis ticks removed, panel border retained (correct for enclosed heatmap), legend styled with elevated background
  • DE-03: Data Storytelling (4/6) — Annotation quantifies M4 retention improvement (+14pp Jan→Jun 2024); highlighted cells guide the eye; subtitle explains the insight

Spec Compliance (14/15)

  • SC-01: Plot Type (5/5) — Correct triangular cohort heatmap
  • SC-02: Required Features (3/4) — Triangular shape, cell text, cohort sizes in labels, colorbar all present; minor: x-axis uses "M0" instead of "Month 0" as spec requests
  • SC-03: Data Mapping (3/3) — Period on X, cohort on Y, retention_rate as fill; correct orientation
  • SC-04: Title & Legend (3/3) — Title format correct; legend labeled "Retention %"

Data Quality (15/15)

  • DQ-01: Feature Coverage (6/6) — Triangular shape, period-0 = 100%, decay over time, product maturity trend across cohorts
  • DQ-02: Realistic Context (5/5) — Real SaaS monthly cohort scenario, Jan–Oct 2024, realistic cohort sizes
  • DQ-03: Appropriate Scale (4/4) — Exponential decay formula produces realistic curves; 2.8pp/cohort maturity trend is plausible

Code Quality (10/10)

  • CQ-01: KISS Structure (3/3) — Linear flow; no functions or classes
  • CQ-02: Reproducibility (2/2) — np.random.seed(42)
  • CQ-03: Clean Imports (2/2) — All imported symbols are used
  • CQ-04: Code Elegance (2/2) — Clean Pythonic code; no fake interactivity
  • CQ-05: Output & API (1/1) — Saves as plot-{THEME}.png; correct API

Library Mastery (7/10)

  • LM-01: Idiomatic Usage (4/5) — Correct grammar-of-graphics: geom_tile + geom_text layers, aes label/color mapping, scale_color_identity for pre-computed colors, annotate()
  • LM-02: Distinctive Features (3/5) — scale_color_identity() for pre-computed text color array is a neat plotnine-specific technique; annotate("rect", fill="none") for cell highlights

Score Caps Applied

  • AR-09 → Score = 0 (title and subtitle clipped at canvas border in both renders)

Strengths

  • Triangular heatmap shape correctly implemented — recent cohorts show fewer columns
  • Dynamic cell text color (dark on green tiles, light on blue tiles) avoids contrast failures across the full gradient
  • Storytelling annotation with annotate("rect") highlights the M4 comparison and quantifies the improvement (+14pp Jan→Jun 2024)
  • Theme-adaptive chrome fully correct — backgrounds, text, and legend box flip cleanly between light (#FAF8F1) and dark (#1A1A17) themes
  • Imprint palette gradient (#4467A3 → #009E73) is semantically correct for a retention metric (green = high = good)
  • Cohort sizes in y-axis labels match spec requirement

Weaknesses

  • AR-09: Title clipped at right canvas edge — ".ai" suffix cut off in both renders. Fix: reduce plot_title fontsize from 12 to 10pt: element_text(size=10, ha="center", weight="bold", color=INK)
  • AR-09: Subtitle clipped at right canvas edge — "month" truncated. Fix: reduce plot_subtitle fontsize from 9 to 7pt: element_text(size=7, ha="center", color=INK_SOFT, style="italic")
  • X-axis labels use abbreviated "M0, M1, ..." instead of spec-requested "Month 0, Month 1, ...". Fix: labels=[f"Month {i}" for i in range(n_cohorts)]
  • Legend at fractional position (0.87, 0.58) may overlap data in upper-right cells — consider legend_position="right" or "bottom"

Issues Found

  1. AR-09 TITLE CLIPPED: plot_title at fontsize=12pt causes "heatmap-cohort-retention · python · plotnine · anyplot.ai" to overflow right canvas edge — ".ai" is visibly cut off in both light and dark renders
    • Fix: plot_title=element_text(size=10, ha="center", weight="bold", color=INK)
  2. AR-09 SUBTITLE CLIPPED: plot_subtitle at fontsize=9pt causes the subtitle to overflow right canvas edge
    • Fix: plot_subtitle=element_text(size=7, ha="center", color=INK_SOFT, style="italic")
  3. SC-02 MINOR: X-axis labels use "M0" abbreviation instead of "Month 0" as spec requires
    • Fix: labels=[f"Month {i}" for i in range(n_cohorts)]

AI Feedback for Next Attempt

The implementation is high quality (81/100 pre-AR-09) with excellent storytelling and correct Imprint palette usage. Two surgical fixes are needed: (1) reduce plot_title fontsize from 12pt to 10pt and plot_subtitle from 9pt to 7pt to prevent right-edge clipping — the y-axis labels are wide (e.g. "Jan 2024 (n=1,200)") which narrows the effective panel area, pushing a 12pt centered title beyond the canvas edge; (2) change x-axis labels from "M0" to "Month 0" to match the spec. No other changes needed.

Verdict: REJECTED

@github-actions

Copy link
Copy Markdown
Contributor Author

🔧 AI Review Produced No Score — Auto-Retrying

The Claude Code Action ran but didn't write quality_score.txt. Auto-retrying review once...


🤖 impl-review

@claude

claude Bot commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

AI Review - Attempt 2/3

Image Description

Light render (plot-light.png): Background is warm off-white (#FAF8F1) — correct Imprint light surface. The triangular cohort retention heatmap displays 10 cohorts (Jan 2024 – Oct 2024) on the y-axis with cohort sizes (e.g., "Jan 2024 (n=1,200)") and months since signup (M0–M9) on the x-axis. Cells are colored on a blue (#4467A3, low retention) → green (#009E73, high retention) gradient with bold percentage labels inside. A "Retention %" colorbar is positioned inside the plot area. Two cells at period M4 (Jan 2024 and Jun 2024) have bold border-highlight boxes, and an annotation "M4: +14pp Jan→Jun 2024" appears in the empty lower-right area. The title is clipped at the right canvas edge — only "anyplot." is visible; the "ai" characters of "anyplot.ai" are missing. The subtitle is clipped at the right canvas edge — renders as "...month over mo" with "nth" pixel-chopped. All other text (y-axis cohort labels, x-axis ticks and title, colorbar legend) is clearly readable against the light background.

Dark render (plot-dark.png): Same layout on warm near-black (#1A1A17) background. Title, axis labels, tick labels, and legend all render in light text (#B8B7B0 secondary / #F0EFE8 primary) and are clearly readable against the dark background — no dark-on-dark failures for any element. The border-highlight annotations appear in light ink in the dark render. Data colors are identical to the light render (green/blue gradient unchanged — only chrome flips). The same AR-09 title and subtitle clipping is present in the dark render. All non-clipped text passes readability in the dark theme.

Both paragraphs are required. A review that only describes one render is invalid.

Score: 0/100

Category Score Max
Visual Quality 15 30
Design Excellence 13 20
Spec Compliance 13 15
Data Quality 14 15
Code Quality 10 10
Library Mastery 7 10
Total (pre-AR-09) 72 100
AR-09 Override 0 100

Visual Quality (15/30)

  • VQ-01: Text Legibility (0/8) — AR-09: title and subtitle clipped at right edge; per rule, VQ-01=0 when title is unreadable in either render
  • VQ-02: No Overlap (5/6) — no significant collisions; cell labels, annotation text, and legend are well-placed
  • VQ-03: Element Visibility (5/6) — heatmap tiles and cell percentage labels clearly visible
  • VQ-04: Color Accessibility (2/2) — CVD-safe Imprint colors; adaptive cell text color (dark on high-retention, light on low-retention) ensures per-cell contrast
  • VQ-05: Layout & Canvas (0/4) — AR-09 text overflow beyond right canvas boundary
  • VQ-06: Axis Labels & Title (1/2) — "Months Since Signup" is descriptive; no y-axis label (intentional); title clipped
  • VQ-07: Palette Compliance (2/2) — blue (#4467A3) → green (#009E73) uses correct Imprint sequential colors; reversed direction is semantically valid (green=high retention=good); backgrounds #FAF8F1/#1A1A17 correct; chrome adapts correctly in both renders

Design Excellence (13/20)

  • DE-01: Aesthetic Sophistication (5/8) — above defaults: custom Imprint gradient, adaptive in-cell text color via scale_color_identity, border-highlight annotations, legend inside chart area
  • DE-02: Visual Refinement (4/6) — grid removed (appropriate for heatmap), axis ticks removed, panel_border retained for the enclosed grid, clean composition
  • DE-03: Data Storytelling (4/6) — concrete quantified insight (M4 period: +14pp Jan→Jun 2024) with visual highlight boxes and annotation text; subtitle reinforces the narrative

Spec Compliance (13/15)

  • SC-01: Plot Type (5/5) — correct triangular cohort retention heatmap; triangular shape emerges naturally from the data construction
  • SC-02: Required Features (3/4) — triangular shape ✓, gradient colormap ✓, cell % labels ✓, cohort sizes in y-axis ✓, colorbar ✓, period 0 = 100% ✓; minor: spec recommends "Month 0" / "Month 1" notation but implementation uses abbreviated "M0" / "M1"
  • SC-03: Data Mapping (3/3) — period on x-axis, cohort labels on y-axis, retention rate as fill color; all correct
  • SC-04: Title & Legend (2/3) — title format "heatmap-cohort-retention · python · plotnine · anyplot.ai" is correct but clipped (AR-09); "Retention %" colorbar label correct

Data Quality (14/15)

  • DQ-01: Feature Coverage (5/6) — shows triangular shape, exponential retention decay, cohort quality trend (product maturity effect), period-0 = 100%, varying cohort sizes
  • DQ-02: Realistic Context (5/5) — SaaS monthly cohorts Jan–Oct 2024 with plausible retention decay and cohort sizes (980–1,450); neutral business domain
  • DQ-03: Appropriate Scale (4/4) — retention 5–100%, 10 cohorts × up to 10 periods, realistic decay constants; all values in expected ranges

Code Quality (10/10)

  • CQ-01: KISS Structure (3/3) — clean imports → data → plot → save, no functions or classes
  • CQ-02: Reproducibility (2/2) — np.random.seed(42)
  • CQ-03: Clean Imports (2/2) — all imports used
  • CQ-04: Code Elegance (2/2) — clean plotnine grammar; annotate('rect') for cell highlights is correct usage, not fake interactivity
  • CQ-05: Output & API (1/1) — saves as plot-{THEME}.png correctly

Library Mastery (7/10)

  • LM-01: Idiomatic Usage (4/5) — good ggplot grammar: geom_tile + geom_text + annotate layering, scale_fill_gradient, scale_color_identity, pd.Categorical for y-axis ordering
  • LM-02: Distinctive Features (3/5) — scale_color_identity for adaptive in-cell text color (idiomatic ggplot2/plotnine pattern), annotate('rect') for cell border highlights, reversed categorical y-axis ordering via pd.Categorical

Score Caps Applied

  • AR-09 (edge clipping): Final score forced to 0, regardless of category scores.

Strengths

  • Correct triangular cohort heatmap shape — period 0 = 100% and each newer cohort naturally has fewer columns
  • Theme-adaptive chrome correctly implemented: all text tokens (INK, INK_SOFT, INK_MUTED) thread through every element in both renders
  • Strong data storytelling: M4 comparison between Jan 2024 and Jun 2024 highlighted with annotate(rect) borders and annotation text quantifying the improvement (+14pp)
  • Imprint palette used correctly for continuous gradient — semantically valid reversed direction (green=high retention=good, blue=low=bad)
  • Code is clean: seed set, all imports used, idiomatic plotnine grammar, scale_color_identity for adaptive cell text

Weaknesses

  • AR-09 (primary): Subtitle (87 chars at 9pt) overflows the right canvas edge — code has subtitle="SaaS monthly cohorts — newer signups retain significantly better month over month" but render shows only "...month over mo" with "nth" pixel-clipped. Fix: reduce plot_subtitle font size from 9pt to 7pt, OR shorten subtitle to ≤70 characters
  • AR-09 (secondary): Title "heatmap-cohort-retention · python · plotnine · anyplot.ai" clipped at right edge — "anyplot." is visible but "ai" is cut off. Fix: reduce plot_title font size from 12pt to 10pt, which applies the style-guide's linear scaling formula for square canvases where effective text width is narrower than landscape
  • X-axis labels use abbreviated "M0""M9" notation instead of the spec-recommended "Month 0""Month 9" format

Issues Found

  1. AR-09 SUBTITLE CLIP: plot_subtitle=element_text(size=9) with an 87-character subtitle overflows the 2400px square canvas right edge
    • Fix: Change to size=7 or shorten subtitle to "SaaS cohorts — newer signups retain significantly better over time" (≤65 chars)
  2. AR-09 TITLE CLIP: plot_title=element_text(size=12) on the square (6-inch) canvas clips the mandatory title at the right edge
    • Fix: Change to size=10 — at 10pt on a 6-inch canvas the title fits with adequate margin
  3. MINOR SC-02: X-axis labels should use "Month 0", "Month 1" per spec rather than abbreviated "M0", "M1"

AI Feedback for Next Attempt

Critical fix: Both the title and subtitle are clipped at the right canvas edge (AR-09). Reduce plot_title size from 12 to 10pt and plot_subtitle size from 9 to 7pt. These values are appropriate for the 2400×2400 square canvas where the effective text width is narrower than landscape due to y-axis label margins. Alternatively, shorten the subtitle to ≤65 characters. Also update x-axis labels from "M0""M9" to "Month 0""Month 9" per spec. The underlying implementation is otherwise solid — triangular shape, Imprint palette, theme-adaptive chrome, and storytelling annotation are all correct.

Verdict: REJECTED

@github-actions github-actions Bot added the ai-review-failed AI review action failed or timed out label Jun 20, 2026
@github-actions

Copy link
Copy Markdown
Contributor Author

❌ AI Review Failed (auto-retry exhausted)

The AI review action completed but did not produce valid output files. Auto-retry already tried once.

What happened:

  • The Claude Code Action ran
  • No quality_score.txt file was created

Manual rerun:

gh workflow run impl-review.yml -f pr_number=9079

🤖 impl-review

@github-actions github-actions Bot added ai-review-rescued Review re-dispatched once after ai-review-failed and removed ai-review-failed AI review action failed or timed out labels Jun 20, 2026
@claude

claude Bot commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

AI Review - Attempt 2/3

Image Description

Light render (plot-light.png): Triangular cohort retention heatmap on a warm off-white #FAF8F1 background. 10 cohorts (Jan–Oct 2024) on the y-axis with cohort sizes (e.g. "Jan 2024 (n=1,200)"), and months M0–M9 on the x-axis. Tiles are colored on a blue-to-green gradient (low retention = #4467A3 blue, high retention = #009E73 green) matching the Imprint palette. Retention percentages are printed inside each cell with dark or light ink depending on tile brightness. Two cells (Jan 2024 M4 and Jun 2024 M4) are highlighted with a thick dark border for comparison, with an annotation "M4: +14pp / Jan→Jun 2024" in the lower-right empty area. Colorbar legend "Retention %" floats in the empty upper-right triangle. CRITICAL FAILURE: The title "heatmap-cohort-retention · python · plotnine · anyplot.ai" is clipped at the right canvas edge — the final "ai" is missing. The subtitle "SaaS monthly cohorts — newer signups retain significantly better month over mo..." is also cut off at the right. AR-09 confirmed. Otherwise axis labels and tick labels are readable against the light background.

Dark render (plot-dark.png): Same layout on a warm near-black #1A1A17 background. Chrome flips correctly: white/light text on dark surface, legend box uses elevated dark bg #242420. Data colors (green high, blue low) are identical to the light render — the Imprint palette holds across themes. Text in cells uses dark/light ink correctly. Same AR-09 clipping failure: title and subtitle are cut off at the right canvas edge in the dark render as well. No dark-on-dark text failure in chrome elements — axis labels, tick labels, and legend text are all light-colored and readable.

Both paragraphs required; both renders confirm AR-09.

Score: 0/100

AR-09 AUTO-REJECT: Title "heatmap-cohort-retention · python · plotnine · anyplot.ai" and subtitle are both clipped at the right canvas edge in both renders. Pixels of the title text are visibly missing at the canvas boundary. Score forced to 0; full scoring below is provided for repair guidance.

Category Score Max
Visual Quality 19 30
Design Excellence 14 20
Spec Compliance 14 15
Data Quality 15 15
Code Quality 10 10
Library Mastery 7 10
Total (pre-reject) 79 100
Total (post-AR-09) 0 100

Visual Quality (19/30)

  • VQ-01: Text Legibility (3/8) — Title and subtitle clipped at right canvas edge; cell percentages slightly small (size=2.8mm) but legible; axis labels and tick labels explicitly sized and readable in both themes
  • VQ-02: No Overlap (5/6) — Legend placed cleverly in empty triangle area; minimal overlap with data tiles
  • VQ-03: Element Visibility (5/6) — Tiles visible and well-sized; cell text readable at canvas resolution
  • VQ-04: Color Accessibility (2/2) — Blue-to-green Imprint gradient is CVD-safe; luminance contrast adequate
  • VQ-05: Layout & Canvas (0/4) — AR-09: title and subtitle clipped at right canvas edge
  • VQ-06: Axis Labels & Title (2/2) — "Months Since Signup" on x-axis; cohort names + sizes on y-axis serve as self-documenting labels
  • VQ-07: Palette Compliance (2/2) — Gradient uses Imprint colors (#4467A3 low → #009E73 high; reversed for semantic retention context); backgrounds #FAF8F1 / #1A1A17 correct in both renders; chrome theme-adaptive

Design Excellence (14/20)

  • DE-01: Aesthetic Sophistication (5/8) — Thoughtful Imprint gradient; highlighted comparison cells with thick borders; annotation adds narrative context; clearly above library defaults
  • DE-02: Visual Refinement (4/6) — Grid removed; panel border kept (appropriate for enclosed heatmap); legend styled with border; good use of empty triangle space
  • DE-03: Data Storytelling (5/6) — Strong: subtitle states the trend, highlighted M4 cells with annotation quantify it, product-maturity arc is visible in the color gradient

Spec Compliance (14/15)

  • SC-01: Plot Type (5/5) — Correct triangular cohort retention heatmap
  • SC-02: Required Features (4/4) — Triangular shape ✓, period 0 = 100% ✓, sequential colormap ✓, cell text ✓, cohort sizes on y-axis ✓, colorbar ✓
  • SC-03: Data Mapping (3/3) — X = periods, Y = cohorts, fill = retention rate; all correct
  • SC-04: Title & Legend (2/3) — Title format "heatmap-cohort-retention · python · plotnine · anyplot.ai" is correct but clipped; legend labels correct

Data Quality (15/15)

  • DQ-01: Feature Coverage (6/6) — Triangular structure, period-0 anchoring, multi-cohort variation, trend across cohorts — all features visible
  • DQ-02: Realistic Context (5/5) — Realistic SaaS product scenario with named monthly cohorts Jan–Oct 2024 and plausible cohort sizes
  • DQ-03: Appropriate Scale (4/4) — Exponential decay from 100% is realistic; cohort sizes 980–1450 plausible; trend bonus for newer cohorts believable

Code Quality (10/10)

  • CQ-01: KISS Structure (3/3) — Clean sequential imports → data → plot → save
  • CQ-02: Reproducibility (2/2) — np.random.seed(42)
  • CQ-03: Clean Imports (2/2) — All imports used
  • CQ-04: Code Elegance (2/2) — Clean, appropriate complexity; no fake UI; annotate rect for highlighting is a legitimate technique
  • CQ-05: Output & API (1/1) — Saves as plot-{THEME}.png

Library Mastery (7/10)

  • LM-01: Idiomatic Usage (4/5) — ggplot grammar used correctly: geom_tile + geom_text + scale_fill_gradient + annotate layered composition
  • LM-02: Distinctive Features (3/5) — annotate("rect", ...fill="none") for cell outlining is a plotnine/ggplot2-specific emphasis technique; scale_color_identity() for per-cell text colors is idiomatic

Score Caps Applied

  • AR-09 auto-reject: Score forced to 0 regardless of quality scores above

Strengths

  • Strong data storytelling: highlighted comparison cells + annotation quantify the product-maturity narrative
  • Smart use of empty triangle space for legend placement
  • All spec features correctly implemented: triangular shape, period-0 anchoring, cohort sizes, colorbar
  • Excellent data quality: realistic SaaS scenario with plausible retention decay
  • Clean code with idiomatic plotnine grammar-of-graphics composition
  • Both themes correctly implement adaptive chrome tokens

Weaknesses

  • AR-09 CRITICAL: Title "heatmap-cohort-retention · python · plotnine · anyplot.ai" is clipped at the right canvas edge in both renders — reduce plot_title fontsize from 12pt to approximately 9pt to fit within the 6-inch (2400px) canvas width. Similarly reduce subtitle fontsize from 9pt to ~7pt.
  • Cell percentage text at size=2.8 (mm) is on the small side — consider increasing to 3.2–3.5 for better mobile readability.

Issues Found

  1. AR-09 CRITICAL: Title and subtitle clipped at right canvas edge in both light and dark renders
    • Root cause: plot_title=element_text(size=12) + plot_subtitle=element_text(size=9) at ha="center" on a 6-inch-wide canvas; the long mandated title overflows horizontally
    • Fix: reduce plot_title size to ~9pt and plot_subtitle size to ~7pt. The formula: round(12 × 67 / 57) ≈ 14 applies to landscape (8 in); for square (6 in) the effective width is smaller so 9pt is the safe target.

AI Feedback for Next Attempt

The implementation is otherwise strong (79/100 pre-reject). The single fix needed is the AR-09 title overflow: set plot_title=element_text(size=9, ...) and plot_subtitle=element_text(size=7, ...) so both title and subtitle fit within the 2400px canvas width. While at it, bump geom_text(size=2.8) to size=3.2 for slightly better mobile readability of the cell labels.

Verdict: REJECTED

@github-actions

Copy link
Copy Markdown
Contributor Author

❌ AI Review Failed (auto-retry exhausted)

The AI review action completed but did not produce valid output files. Auto-retry already tried once.

What happened:

  • The Claude Code Action ran
  • No quality_score.txt file was created

Manual rerun:

gh workflow run impl-review.yml -f pr_number=9079

🤖 impl-review

@github-actions github-actions Bot added the ai-review-failed AI review action failed or timed out label Jun 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai-attempt-1 First repair attempt ai-review-failed AI review action failed or timed out ai-review-rescued Review re-dispatched once after ai-review-failed quality:89 Quality score 89/100

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants