Skip to content

fix: fetch(attrs, as_dict=True) excludes unrequested primary key#1403

Closed
dimitri-yatsenko wants to merge 1 commit intomasterfrom
fix/fetch-as-dict-extra-keys
Closed

fix: fetch(attrs, as_dict=True) excludes unrequested primary key#1403
dimitri-yatsenko wants to merge 1 commit intomasterfrom
fix/fetch-as-dict-extra-keys

Conversation

@dimitri-yatsenko
Copy link
Member

Summary

Restores 0.14.x behavior where fetch('col1', 'col2', as_dict=True) returned dicts containing only the requested attributes.

In 2.x, proj() always includes primary key attributes, so the returned dicts contained extra keys the caller didn't ask for. This breaks downstream code that concats the result with an existing DataFrame that already has the primary key column — the concat creates duplicate columns and subsequent groupby fails.

The fix filters each dict after to_dicts() to include only the requested attributes. This is safe regardless of row order because we are removing keys from individual dicts, not rearranging rows.

Reproducing the issue

# Table with primary key (dataset) and secondary attrs (heading_dir, head_angle)

# 0.14.x — only requested attrs:
Table.fetch('heading_dir', 'head_angle', as_dict=True)
# [{'heading_dir': ..., 'head_angle': ...}, ...]

# 2.1.0 — primary key leaks in:
Table.fetch('heading_dir', 'head_angle', as_dict=True)
# [{'dataset': ..., 'heading_dir': ..., 'head_angle': ...}, ...]
# pd.concat with a df that already has 'dataset' creates duplicates

Test plan

  • fetch('attr', as_dict=True) returns dicts without primary key
  • fetch('KEY', 'attr', as_dict=True) still includes primary key (explicitly requested)
  • Integration tests pass

…y key

In 0.14.x, fetch('col1', 'col2', as_dict=True) returned dicts containing
only the requested attributes. In 2.x, proj() always includes primary key
attributes, so the returned dicts contained extra keys the caller didn't ask
for. Filter the dicts to only include requested attributes, restoring
backward compatibility.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@dimitri-yatsenko
Copy link
Member Author

Won't fix — the 2.x behavior of always including primary key in fetch(attrs, as_dict=True) is intentional. It preserves row identity. Downstream code should handle column deduplication when concatenating.

@dimitri-yatsenko dimitri-yatsenko deleted the fix/fetch-as-dict-extra-keys branch February 17, 2026 21:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant