Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .changelog/add-subtraction-flooring-pattern.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
- bump: patch
changes:
added:
- Add Pattern 5 to policyengine-vectorization-skill documenting correct use of max_() when subtracting values and flooring at zero
- Add floor subtraction pattern to Quick Reference Card
- Document the "phantom income" bug pattern found in MT income tax implementation
4 changes: 4 additions & 0 deletions .changelog/state-income-tax-conformity.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
- bump: patch
changes:
added:
- Add guidance on state income tax conformity to federal rules in policyengine-implementation-patterns-skill
25 changes: 25 additions & 0 deletions agents/country-models/document-collector.md
Original file line number Diff line number Diff line change
Expand Up @@ -252,6 +252,31 @@ reference = "https://www.law.cornell.edu/..." # Full clickable URL
- "[Program] categorical eligibility"
- "[Program] benefit formula"

**CRITICAL: Pay Special Attention to Income Definitions**

When collecting documentation for tax credits or income-based programs, carefully note the **exact income definition** used in the statute. Many programs use modified versions of standard measures:

- Look for phrases like "AGI **plus**..." or "AGI **minus**..."
- Common patterns:
- "adjusted gross income plus exemptions"
- "modified adjusted gross income"
- "income as defined in section [X]"
- "gross income less certain deductions"

**Example:** Arizona Family Tax Credit (ARS 43-1073) specifies:
> "Arizona adjusted gross income, plus the amount subtracted for exemptions under section 43-1023"

This means the eligibility threshold uses `az_agi + az_exemptions`, NOT just `az_agi`.

**Documentation template for modified income:**
```markdown
### Income Definition
**Statute Citation**: [Exact citation]
**Income Measure**: [Standard measure] PLUS/MINUS [adjustments]
**Exact Statutory Language**: "[Quote the statute]"
**Implementation**: variable_name + adjustment_variable_name
```

### 3. Verify Currency
- Check "effective date" on all documents
- Search for "final rule" to find recent changes
Expand Down
10 changes: 9 additions & 1 deletion agents/country-models/edge-case-generator.md
Original file line number Diff line number Diff line change
Expand Up @@ -176,6 +176,7 @@ For each detected pattern:
### Income-Based Programs
- Zero income
- Negative income (self-employment losses)
- **CRITICAL**: Negative income combined with zero expenses/deductions (catches incorrect eligibility/benefit bugs)
- Income exactly at each threshold
- Maximum possible income

Expand All @@ -197,10 +198,17 @@ For each detected pattern:

### Benefit Calculations
- Minimum benefit scenarios
- Maximum benefit scenarios
- Maximum benefit scenarios
- Zero benefit (just above cutoff)
- Rounding edge cases

### Tax Credit Programs
- Negative income with zero deductible expenses (property tax, rent, etc.)
- Negative income with positive deductible expenses
- Zero income scenarios
- Income exactly at phase-out thresholds
- Maximum credit scenarios with minimum qualifying expenses

## Output Format

Generate test files with clear documentation:
Expand Down
24 changes: 24 additions & 0 deletions agents/country-models/implementation-validator.md
Original file line number Diff line number Diff line change
Expand Up @@ -304,6 +304,30 @@ Implementation passes when:
- Proper federal/state separation
- Include effective dates

**CRITICAL: Watch for Modified Income Definitions**

Many state tax credits and benefits use **modified versions** of standard income measures. The statute will specify:
- "AGI **plus** [certain additions]" (e.g., "AGI plus exemptions")
- "AGI **minus** [certain subtractions]" (e.g., "AGI less student loan interest")
- "Income as defined in [other statute section]"

**Example from Arizona Family Tax Credit (ARS 43-1073):**
```python
# ❌ WRONG - Uses only AGI
income = tax_unit("az_agi", period)
eligible = income <= threshold

# ✅ CORRECT - Uses AGI plus exemptions per statute
income = tax_unit("az_agi", period) + tax_unit("az_exemptions", period)
eligible = income <= threshold
```

**Validation checklist:**
- [ ] Read the statute's exact income definition language
- [ ] Check if it references AGI "plus" or "minus" any adjustments
- [ ] Verify the variable uses the correct modified income measure
- [ ] Document the specific statute citation that defines the income measure

### Benefit Calculations
- All rates from parameters
- Min/max thresholds parameterized
Expand Down
2 changes: 2 additions & 0 deletions changelog_entry.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
- bump: patch
changes:
added:
- Document bracket parameter pattern for handling negative values (-.inf vs 0 thresholds) in parameter patterns skill
changed:
- Add critical warning against mixing adds/subtracts with custom formulas in implementation patterns skill
Original file line number Diff line number Diff line change
Expand Up @@ -352,6 +352,83 @@ first_person: 14_580
fpg_multiplier: 2.0 # 200% of FPG
```

### State Income Tax Conformity to Federal Rules

**CRITICAL: State income taxes should reference federal income sources and limits, not redefine them**

Most state income taxes start with federal definitions and then make specific adjustments. When implementing state income tax:

**✅ CORRECT - Reference federal income sources:**
```python
class ms_agi(Variable):
"""Mississippi adjusted gross income"""
value_type = float
entity = TaxUnit
definition_period = YEAR
label = "Mississippi adjusted gross income"
unit = USD

def formula(tax_unit, period, parameters):
# Start with federal AGI, which already includes
# federal capital loss limits and other federal rules
federal_agi = tax_unit("adjusted_gross_income", period)

# Apply Mississippi-specific additions/subtractions
ms_additions = tax_unit("ms_additions_to_agi", period)
ms_subtractions = tax_unit("ms_subtractions_from_agi", period)

return federal_agi + ms_additions - ms_subtractions
```

**❌ WRONG - Redefining income sources:**
```python
# DON'T create state-specific parameters like:
# parameters/gov/states/ms/tax/income/income_sources.yaml
# containing:
# - capital_gains
# - long_term_capital_gains
# - short_term_capital_gains

# This bypasses federal limits like the $3,000 capital loss deduction limit
```

**Why this matters:**
- Federal income tax applies capital loss limits before reporting AGI
- State income taxes that start from federal AGI automatically inherit these limits
- Creating separate state income source parameters bypasses federal rules
- Results in incorrect calculations (e.g., unlimited capital loss deductions)

**Common state conformity patterns:**
1. **Full conformity** - State AGI = Federal AGI (rare)
2. **Rolling conformity** - State follows current federal rules
3. **Static conformity** - State follows federal rules as of a specific date
4. **Selective conformity** - State follows federal but with specific modifications

**Implementation approach:**
- Always start with federal income sources/AGI/taxable income as the base
- Use state parameters only for state-specific additions, subtractions, or modifications
- Reference federal variables: `adjusted_gross_income`, `taxable_income`, etc.
- Don't recreate federal income aggregation logic at the state level

**Example - Mississippi specifics:**
```python
class ms_additions_to_agi(Variable):
"""Mississippi additions to federal AGI"""
# Add state-specific income items not in federal AGI
adds = [
"ms_state_bond_interest",
"ms_other_additions"
]

class ms_subtractions_from_agi(Variable):
"""Mississippi subtractions from federal AGI"""
# Subtract state-specific deductions
adds = [
"ms_retirement_income_exclusion",
"ms_other_subtractions"
]
```

---

## Code Reuse Patterns
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -314,6 +314,52 @@ metadata:
label: State PROGRAM earned income disregard percentage
```

### Bracket-Based Parameters

**CRITICAL: Handling Negative Values**

When creating bracket-based parameters (e.g., tax credits based on AGI), the first bracket threshold MUST be `-.inf` if negative values are possible, NOT `0`.

**❌ WRONG - Excludes negative AGI:**
```yaml
# threshold.yaml (for single filers)
brackets:
- threshold:
2023-01-01: 0 # ❌ Bug: negative AGI excluded!
amount:
2023-01-01: 300
- threshold:
2023-01-01: 30_000
amount:
2023-01-01: 110
```

**✅ CORRECT - Includes all possible values:**
```yaml
# threshold.yaml (for single filers)
brackets:
- threshold:
2023-01-01: -.inf # ✅ Covers negative AGI
amount:
2023-01-01: 300
- threshold:
2023-01-01: 30_000
amount:
2023-01-01: 110
```

**When to use `-.inf`:**
- Income-based calculations (AGI can be negative)
- Any parameter where negative input values are valid
- Tax credits, deductions, or benefits based on earnings

**When `0` is appropriate:**
- Age thresholds (always non-negative)
- Count-based parameters (household size, number of dependents)
- Resource limits (assets can't be negative)

**Real-world example:** Hawaii Food/Excise Tax Credit uses AGI brackets. The first threshold must be `-.inf` to correctly handle taxpayers with negative AGI (e.g., business losses).

---

## 7. Validation Checklist
Expand Down
125 changes: 124 additions & 1 deletion skills/technical-patterns/policyengine-vectorization-skill/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,46 @@ elif amount > maximum:
# Or: amount = max_(0, min_(amount, maximum))
```

### Pattern 5: Flooring Subtraction Results (CRITICAL)

When subtracting values and wanting to floor at zero, you must wrap the **entire subtraction** in `max_()`:

```python
# Common scenario: income after deductions/losses
❌ WRONG - Creates phantom negative values:
income = max_(income, 0) - capital_loss # If capital_loss > income, result is negative!

✅ CORRECT - Properly floors at zero:
income = max_(income - capital_loss, 0) # Entire subtraction floored

# Real example from MT income tax bug:
❌ WRONG - Tax on phantom negative income:
def formula(tax_unit, period, parameters):
income = tax_unit("adjusted_gross_income", period)
capital_gains = tax_unit("capital_gains", period)

# BUG: If capital_gains is negative (loss), this creates negative income
# But max_() only floors income, not the result
regular_income = max_(income, 0) - capital_gains
return calculate_tax(regular_income) # Tax on negative number!

✅ CORRECT - No phantom income:
def formula(tax_unit, period, parameters):
income = tax_unit("adjusted_gross_income", period)
capital_gains = tax_unit("capital_gains", period)

# Properly floors the entire result
regular_income = max_(income - capital_gains, 0)
return calculate_tax(regular_income) # Never negative
```

**Why this matters:**
- If `capital_gains = -3000` (loss), then `income - capital_gains = income + 3000`
- The wrong pattern `max_(income, 0) - capital_gains` allows the subtraction to make the result negative
- This creates "phantom income" where none exists, leading to incorrect tax calculations

**Rule:** When the formula is `A - B` and you want the result floored at zero, use `max_(A - B, 0)`, NOT `max_(A, 0) - B`.

---

## 3. When if-else IS Acceptable
Expand Down Expand Up @@ -287,10 +327,92 @@ def test_vectorization():
| Boolean OR | `or` | `\|` |
| Boolean NOT | `not` | `~` |
| Bounds checking | `if x < 0: x = 0` | `max_(0, x)` |
| Floor subtraction | `max_(x, 0) - y` ❌ | `max_(x - y, 0)` ✅ |
| Complex logic | Nested if | Nested where/select |

---

## 8. Debugging Phantom Values in Tax Calculations

### Problem: Non-Zero Tax Despite Zero Taxable Income

When state tax calculations produce small non-zero values (e.g., $277) even though taxable income is zero, check for:

#### Root Cause 1: Implicit Type Conversion in min/max Operations

```python
# Example from Montana income tax bug
❌ WRONG - Creates phantom values:
def formula(tax_unit, period, parameters):
regular_tax_before_credits = tax_unit("mt_income_tax_before_credits", period)
credits = tax_unit("mt_income_tax_refundable_credits", period)

# BUG: min() with int 0 converts float array to int, losing precision
# When regular_tax_before_credits = 0.0, this can produce non-zero results
return max_(regular_tax_before_credits - credits, 0)

✅ CORRECT - Preserves array types:
def formula(tax_unit, period, parameters):
regular_tax_before_credits = tax_unit("mt_income_tax_before_credits", period)
credits = tax_unit("mt_income_tax_refundable_credits", period)

# Use max_() which handles arrays correctly
return max_(regular_tax_before_credits - credits, 0)
```

#### Root Cause 2: Phantom Intermediate Values in Calculation Chains

When taxable income is zero but tax is non-zero, trace the calculation chain:

```python
# Tax calculation chain (Montana example)
taxable_income: 0 # ✓ Correct
rate: 0.0475 # Used despite zero income
brackets: [15_600] # Used despite zero income
tax_before_credits: 277.41 # ❌ PHANTOM VALUE

# The bug: Brackets calculated regular tax even when taxable income was zero
# due to missing zero-check in bracket calculation
```

#### Debugging Pattern

When you see phantom tax values:

1. **Check the calculation chain** - Run test with verbose output to see intermediate values:
```bash
pytest tests/file.py -vv
```

2. **Verify zero-income handling** - Look for formulas that don't short-circuit on zero income:
```python
✅ GOOD:
def formula(entity, period, parameters):
taxable_income = entity("taxable_income", period)
# Short-circuit when income is zero
return where(taxable_income == 0, 0, calculate_tax(...))

❌ BAD:
def formula(entity, period, parameters):
# Always calculates, even when income is zero
return calculate_brackets(taxable_income, rates, brackets)
```

3. **Check type consistency** - Ensure operations preserve NumPy array dtypes:
```python
✅ Use: max_(value, 0) or clip(value, 0, None)
❌ Avoid: max(value, 0) - Python's max can cause type issues
```

#### Common Symptoms

- Tax calculated despite zero taxable income
- Small non-zero values when expecting exactly zero
- Tax values that don't match manual calculations
- Capital gains deductions not properly reducing taxable income

---

## For Agents

When implementing formulas:
Expand All @@ -300,4 +422,5 @@ When implementing formulas:
4. **Use NumPy operators** (&, |, ~) not Python (and, or, not)
5. **Test with arrays** to ensure vectorization
6. **Parameter conditions** can use if-else (scalars)
7. **Entity data** must use vectorized operations
7. **Entity data** must use vectorized operations
8. **Debug phantom values** by tracing calculation chains and checking type preservation