Add LA-level council tax calibration targets (Band D + band distribution)#374
Add LA-level council tax calibration targets (Band D + band distribution)#374vahid-ahmadi wants to merge 3 commits intomainfrom
Conversation
Two families of LA-level targets, covering all 360 LAs in
local_authorities_2021.csv, built from four public sources:
- `ons/council_tax_band_d/{code}` (350 targets): average Band D
council tax inclusive of all precepts per billing authority.
Sources: MHCLG *Council Tax levels set by local authorities in
England 2026-27*, Welsh Government *Council Tax levels April 2026
to March 2027*, Scottish Government *Council Tax Assumptions 2025*.
All 296 English + 22 Welsh + 32 Scottish LAs covered.
- `ons/council_tax_band_count/{code}/{band}` (2,541 targets): number
of dwellings per band A-H per LA. Source: VOA *Council Tax: Stock
of Properties, 2025*. Covers England + Wales (318 LAs × ~8 bands,
minus City of London Band A which is VOA-suppressed).
NI is excluded: domestic rates, not council tax. Scotland band
counts are not in VOA; Scottish Assessors publishes them separately
and is a follow-up.
Files
-----
- `storage/la_council_tax.csv` (31 KB, 360 rows): canonical CSV
joining DLUHC Table 10 column 17, Welsh Table 1 "Overall average
band D", Scottish Gov "CT by Band 2025-26" Band D column, and VOA
CTSOP1.0 bands A-H onto the reference LA list.
- Post-2023 South Yorkshire E-codes (E08000038/39) re-mapped to
pre-2023 codes (E08000016/19) to match the reference list.
- Scottish ampersand/double-space naming normalised
("Argyll & Bute" → "Argyll and Bute", etc.).
- `targets/sources/la_council_tax.py`: reads the CSV, emits Target
objects at geographic_level=LOCAL_AUTHORITY with per-country year
tagging and per-country reference URL.
Testing
-------
22 hermetic tests (no network access, no baseline fixture needed):
Structure
- Row count matches local_authorities_2021.csv.
- Every expected column present.
- Four UK country codes represented.
- Every LA code matches the reference list.
Value plausibility (the #371 lesson)
- Band D amount in [£900, £3,500] for every row with a value.
- Total dwellings in [200, 800,000] for every row with a value.
- Explicit Isles of Scilly regression test: total dwellings in
[500, 5,000], not the 2.49M outlier that slipped into #371.
- Band A-H counts sum to total dwellings within 20-property slack
(VOA 10-property suppression allowance).
- Every band-count target value ≤ 500k (largest LA stock).
Coverage expectations
- Every English, Welsh and Scottish LA has a Band D value.
- Northern Ireland has no council tax flagged (has_council_tax=False).
Spot-checks of published facts
- Wandsworth (E09000032) and Westminster (E09000033) are the two
lowest-Band-D English LAs (catches row-swap bugs).
- Scottish average Band D is £500+ below English average.
Target-API invariants
- get_targets() returns a non-empty list without network access.
- Band D target count matches the CSV's non-null Band D count.
- Band count target count matches Σ non-null band columns.
- Every target carries geographic_level=LOCAL_AUTHORITY and a
geo_code.
- Band D targets use Unit.GBP; band count targets use Unit.COUNT
with is_count=True.
- Every target has at least one year of values.
Sources
-------
- MHCLG (England 2026-27):
https://www.gov.uk/government/statistics/council-tax-levels-set-by-local-authorities-in-england-2026-to-2027
- Welsh Government (Wales 2026-27):
https://www.gov.wales/council-tax-levels-april-2026-march-2027-html
- Scottish Government (Scotland 2025-26):
https://www.gov.scot/publications/council-tax-datasets/
- VOA (England + Wales 2025):
https://www.gov.uk/government/statistics/council-tax-stock-of-properties-2025
Out of scope for this PR (follow-ups)
-------------------------------------
- Wiring these targets into
datasets/local_areas/local_authorities/loss.py so the LA
reweighting actually calibrates on them. Planned follow-up PR.
- Scottish Assessors per-LA chargeable-dwellings to fill the Scotland
band-count gap.
- Council Tax Support caseload per LA (DWP StatXplore).
- Single Person Discount rate per LA (CIPFA).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Review — items to address before mergeBlocking1. Welsh Band I is dropped. Wales has had 9 council tax bands (A–I) since its 2005 revaluation. Fix: add 2.
Because both sides of Coherence3. 4. CSV column asymmetry. Nits5. 6. 7. Module docstring opens with "band A–H" — update once Band I is added. PR descriptionWith Band I added, the "2,541 band-count targets" figure will rise by the number of non-null Welsh Band I cells — worth regenerating the description from the final CSV before merging. My earlier arithmetic of 318 × 8 = 2,544 minus City of London band A = 2,543 gave 2 more suppressions than the description claimed; Band I coverage may resolve part of that gap too. |
Review points addressed:
- Add count_band_I column to la_council_tax.csv, populated for all 22
Welsh LAs (Wales revalued in 2005 and introduced a 9th band). Cardiff
1480, Monmouthshire 670, Vale of Glamorgan 1060, etc. English rows
keep Band I null; VOA marks it [z] (not applicable).
- Re-source total_dwellings from VOA "All properties" column instead
of deriving it as the sum of A-H. Previously Σ(A..H) was used for
both sides of test_band_counts_sum_to_total, making the test
self-referential; now it validates against the published total with
a 20-property slack for VOA rounding.
- Rename count columns symmetrically: band_A..band_H + band_D_count →
count_band_A..count_band_I. Removes the lopsided band_D_count name
that existed only to avoid clashing with band_d_amount.
- Align band-count target names with voa_council_tax.py:
voa/council_tax/{code}/{band} (was ons/council_tax_band_count/...);
variable="council_tax_band" (was council_tax_band_count, which is
not a real PolicyEngine-UK variable); drop breakdown_variable to
match the regional VOA module.
- Cache the CSV read with @lru_cache(maxsize=1), matching voa_council_tax.
- Update module docstring: "A-H in England/Scotland, A-I in Wales".
Tests:
- New: test_welsh_las_have_band_i (all 22 Welsh LAs populated).
- New: test_english_las_have_no_band_i (guard against spurious fills).
- New: test_cardiff_band_i_matches_published_figure (~1,480 per VOA 2025).
Final target counts:
- 350 Band D amount targets (unchanged).
- 2,563 band-count targets, up from 2,541: +22 Welsh Band I plus two
band-H rows that were null due to the earlier truncation.
|
Pushed d2d2acf addressing the review items: Blocking fixes
Coherence fixes
Nits
Final target counts: 350 Band D amount + 2,563 band counts (up from 2,541: +22 Welsh Band I, +2 resolved |
Summary
Adds 350 LA-level
ons/council_tax_band_d/{code}targets (Band D amount per billing authority) and 2,541ons/council_tax_band_count/{code}/{band}targets (dwellings per band A-H per LA) built from four public sources, covering all 296 English + 22 Welsh + 32 Scottish LAs inlocal_authorities_2021.csv. Follows the pattern of the regional land-value targets, one geography deeper.Like #371 this is plumbing + data only — wiring these into
datasets/local_areas/local_authorities/loss.pyis a deliberate follow-up.What this PR does
Four public sources, one canonical CSV
storage/la_council_tax.csv(360 rows, 31 KB) joins:band_d_amount(England)band_d_amount(Wales)band_d_amount(Scotland)band_A…band_H+total_dwellingsColumn 17 in DLUHC Table 10 is the right one: "Average (Band D 2 adult equivalent) council tax for area of the billing authority including both local and major precepts" — i.e. the full amount households pay, inclusive of county, police, fire and parish precepts.
Code joins reconciled
Argyll & Bute→Argyll and Bute,Dumfries & Galloway→Dumfries and Galloway,Perth & Kinross→Perth and Kinross,Shetland Islands(double space) →Shetland Islands,Edinburgh, City of→City of Edinburgh.New module
targets/sources/la_council_tax.py:get_targets()returns Target objects atgeographic_level=LOCAL_AUTHORITY.valuesrather than emitted as NaN so calibrators cleanly skip them.Documented coverage gaps
has_council_tax=Falseflag set; no targets emitted for NI.[c]) for disclosure control; other bands populated.Testing (the #371 lesson)
22 hermetic tests, all green locally:
get_targets()returns non-empty without network, Band D target count matches CSV, band count target count matches Σ non-null band columns, every target carries LOCAL_AUTHORITY geo level +geo_code, correct units andis_count=Trueon count targets, every target has ≥1 year of values, every band count ≤ 500k.Sanity check — top 10 LAs by Band D amount (2026-27 where available)
Bottom of the table is dominated by the central-London LAs with the lowest precepts: Wandsworth (£1,028), Westminster (£1,050), City of London (£1,330).
Out of scope for this PR (follow-ups)
datasets/local_areas/local_authorities/loss.pyso the LA reweighting actually calibrates on them.Related
targets/sources/voa_council_tax.pyalready provides band targets at the REGION level; this PR adds the LA level alongside it without modifying the regional path.