Add vendored data + maintenance scripts for hansen_singleton lectures#926
Open
mmcky wants to merge 3 commits into
Open
Add vendored data + maintenance scripts for hansen_singleton lectures#926mmcky wants to merge 3 commits into
mmcky wants to merge 3 commits into
Conversation
Adds _static/lecture_specific/hansen_singleton_198{2,3}/, each containing:
- make_data.py downloads the raw FRED + Ken French inputs and builds the
analysis-ready monthly CSV (standard library + pandas only)
- *_data.csv frozen snapshot, monthly 1959-02..1978-12
- README.md sources, sample, column definitions, refresh instructions
This seeds the data on main so the lectures can read it from GitHub instead of
querying FRED / Ken French live at build time, which keeps the build
reproducible and avoids the flaky-network failure class. The lecture changes
that consume these files are in PR #925 (part of the pandas 3.0 work, #924).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds vendored, analysis-ready input datasets and regeneration scripts for the hansen_singleton_1982 and hansen_singleton_1983 lectures under the existing lectures/_static/lecture_specific/<lecture>/ pattern, to avoid live FRED / Ken French downloads during book builds.
Changes:
- Add
make_data.pyscripts that download/construct the monthly series and write a committed CSV snapshot. - Add committed CSV snapshots for both lectures (239 monthly observations, 1959-02 to 1978-12).
- Add per-lecture README files documenting sources, sample, columns, and refresh steps.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| lectures/_static/lecture_specific/hansen_singleton_1982/README.md | Documents the vendored dataset and regeneration procedure for the 1982 lecture. |
| lectures/_static/lecture_specific/hansen_singleton_1982/make_data.py | Script to download source series and build the 1982 analysis-ready CSV. |
| lectures/_static/lecture_specific/hansen_singleton_1982/hansen_singleton_1982_data.csv | Frozen CSV snapshot used by the lecture build. |
| lectures/_static/lecture_specific/hansen_singleton_1983/README.md | Documents the vendored dataset and regeneration procedure for the 1983 lecture. |
| lectures/_static/lecture_specific/hansen_singleton_1983/make_data.py | Script to download source series and build the 1983 analysis-ready CSV. |
| lectures/_static/lecture_specific/hansen_singleton_1983/hansen_singleton_1983_data.csv | Frozen CSV snapshot used by the lecture build. |
mmcky
added a commit
that referenced
this pull request
Jun 21, 2026
…line
Switch both lectures to read the pre-built monthly CSV from
_static/lecture_specific/hansen_singleton_198{2,3}/ (added in PR #926) via its
raw GitHub URL, replacing the inline FRED / Fama-French download helpers from
the previous commit. The data construction now lives in the per-lecture
make_data.py maintenance scripts; the lectures just read the frozen snapshot.
This keeps the build reproducible and off the live data endpoints, and still
removes the pandas-datareader dependency that breaks under pandas 3.0.
Depends on PR #926 (must land on main first so the raw URL resolves).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…n_cons The README column formulas divided by `gross_inflation`, but the script computes (and the 1983 CSV exposes) `gross_inflation_cons`. Rename to match, and add a one-line note in the 1982 README since it is an intermediate there, not a CSV column. Addresses Copilot review on PR #926. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
📖 Netlify Preview Ready!Preview URL: https://pr-926--sunny-cactus-210e3e.netlify.app Commit: Build Info
|
Wrap zipfile.ZipFile(...) in a `with` block so the archive is explicitly closed, instead of leaving it to garbage collection. Pure refactor: both scripts still reproduce byte-identical CSVs. Addresses Copilot review (raised on PR #925, where this code previously lived). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds vendored input data plus a maintenance script and README for the two Hansen–Singleton lectures, under the existing
_static/lecture_specific/<lecture>/convention:hansen_singleton_1982/—make_data.py,hansen_singleton_1982_data.csv,README.mdhansen_singleton_1983/—make_data.py,hansen_singleton_1983_data.csv,README.mdEach
make_data.pydownloads the raw inputs (FRED seriesCNP16OV,DNDGRA3M086SBEA,DNDGRG3M086SBEA; the Ken FrenchF-F_Research_Data_Factorsmonthly file), builds the analysis-ready monthly series, and writes the CSV next to it. The committed CSVs are the script output: 239 monthly observations, 1959-02 to 1978-12. The scripts use only the standard library plus pandas.Why
This seeds the data on
mainso the Hansen–Singleton lectures can read it from a stable GitHub URL instead of querying FRED / Ken French live during the build — the same pattern already used bymle,ols, andpandas_panel. Benefits:olsdata download).python make_data.py).Notes
hansen_singleton_198{2,3}to read these files (and removes thepandas-datareaderdependency that breaks under pandas 3.0) is in PR [hansen_singleton] Replace pandas-datareader with direct FRED / Fama-French downloads #925, part of the pandas 3.0 work tracked in pandas 3.0 (anaconda 2026.06) breaks hansen_singleton lectures via pandas-datareader #924. Merge this PR first, then [hansen_singleton] Replace pandas-datareader with direct FRED / Fama-French downloads #925.