Skip to content

Add sparse matrix builder for local area calibration - SNAP targets #456

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
MaxGhenis merged 7 commits into from
Dec 8, 2025
Merged

Add sparse matrix builder for local area calibration - SNAP targets #456

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
821414e
Add sparse matrix builder for local area calibration
baogorek Dec 5, 2025
0400066
Add changelog entry and format code
baogorek Dec 5, 2025
06098a5
Refactor tests and fix enum encoding, minimize PR scope
baogorek Dec 5, 2025
8269168
Rename GEO_STACKING to LOCAL_AREA_CALIBRATION and restore tracer func…
baogorek Dec 5, 2025
95f21ca
Clear notebook outputs for Myst compatibility
baogorek Dec 5, 2025
07852a1
Pin mystmd>=1.7.0 to fix notebook rendering in docs
baogorek Dec 5, 2025
7f6ea43
Add logging for constraint evaluation failures and document CD GEOID …
MaxGhenis Dec 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions .github/workflows/reusable_test.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,18 @@ jobs:
TEST_LITE: ${{ !inputs.upload_data }}
PYTHON_LOG_LEVEL: INFO

- name: Build datasets for local area calibration
if: inputs.full_suite
run: |
LOCAL_AREA_CALIBRATION=true python policyengine_us_data/datasets/cps/cps.py
LOCAL_AREA_CALIBRATION=true python policyengine_us_data/datasets/puf/puf.py
LOCAL_AREA_CALIBRATION=true python policyengine_us_data/datasets/cps/extended_cps.py
python policyengine_us_data/datasets/cps/local_area_calibration/create_stratified_cps.py 10500

- name: Run local area calibration tests
if: inputs.full_suite
run: pytest policyengine_us_data/tests/test_local_area_calibration/ -v

- name: Save calibration log
if: inputs.full_suite
uses: actions/upload-artifact@v4
Expand Down
6 changes: 6 additions & 0 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,12 @@ data:
mv policyengine_us_data/storage/enhanced_cps_2024.h5 policyengine_us_data/storage/dense_enhanced_cps_2024.h5
cp policyengine_us_data/storage/sparse_enhanced_cps_2024.h5 policyengine_us_data/storage/enhanced_cps_2024.h5

data-local-area: data
LOCAL_AREA_CALIBRATION=true python policyengine_us_data/datasets/cps/cps.py
LOCAL_AREA_CALIBRATION=true python policyengine_us_data/datasets/puf/puf.py
LOCAL_AREA_CALIBRATION=true python policyengine_us_data/datasets/cps/extended_cps.py
python policyengine_us_data/datasets/cps/local_area_calibration/create_stratified_cps.py 10500

clean:
rm -f policyengine_us_data/storage/*.h5
rm -f policyengine_us_data/storage/*.db
Expand Down
8 changes: 8 additions & 0 deletions changelog_entry.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
- bump: minor
changes:
added:
- Sparse matrix builder for local area calibration with database-driven constraints
- Local area calibration data pipeline (make data-local-area)
- ExtendedCPS_2023 and PUF_2023 dataset classes
- Stratified CPS sampling to preserve high-income households
- Matrix verification tests for local area calibration
Loading