Skip to content

Conversation

@pydanny
Copy link
Contributor

@pydanny pydanny commented Nov 28, 2024

Why does this exist?

  1. insert_chunk was still dependant on last_rowid in non-upsert scenarios
  2. Primary keys were not added per the Minidata API spec, we were still learning on last_rowid: https://docs.fastht.ml/explains/minidataapi.html#creating-tables
  3. Tests still relied on last_rowid to function
  4. Makes this rowsafe

What this does:

  1. Removes use of last_rowid table from sqlite-minutils
  2. Address last_rowid in tests
  3. If no primary key set, forces int:id to serve in this role
  4. Adds tests for multiple creates on same db, previous tests only tested creation and minimal transforms

Tests to be done:

  • all pytest tests in repo
  • Fastlite
  • FastHTML
  • adv_app.py
  • Solveit
  • Vacuum existing sample DB and investigate for corruption

@pydanny pydanny marked this pull request as ready for review November 28, 2024 17:03
@pydanny pydanny marked this pull request as draft November 28, 2024 17:04
@pydanny pydanny marked this pull request as ready for review November 28, 2024 21:57
@pydanny pydanny marked this pull request as draft November 29, 2024 15:03
@pydanny pydanny changed the title Cleanup rowid WIP: Cleanup rowid Nov 29, 2024
@pydanny pydanny marked this pull request as ready for review December 2, 2024 12:21
@pydanny pydanny changed the title WIP: Cleanup rowid Cleanup rowid Dec 2, 2024
Copy link
Contributor

@jph00 jph00 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Danny! My only question is about whether we really want all that code to auto-select 'id' for users. My guess is that we don't, but I might be missing something.

)

column_defs = []
# All minidata tables get a primary key: https://docs.fastht.ml/explains/minidataapi.html#creating-tables
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minutils does not support the spec - that's fastlite. We shouldn't add a pk here automatically IMO, unless I'm missing something.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll work to get tests to pass without it.

columns_to_drop = [
column for column in existing_columns if column not in columns
]
# If no primary key was specified and id was added automatically, we prevent
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess we can remove this then?

# Has the primary key changed?
current_pks = table.pks
desired_pk = None
if isinstance(pk, str):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You know, with a ternary op, these 4 lines would be just one line... ;)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh n/m I see this was just a move of some lines Simon wrote.

if transform and self[name].exists():
table = cast(Table, self[name])
should_transform = False
# Has the primary key changed?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought changing the pk already worked? IIRC I used that feature during development of solveit...

defaults=defaults,
pk=pk,
)
# There has been set a primary key that isn't ['id'].It is now safe to drop the ID column.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and remove this.

"Primary key columns for this table."
names = [column.name for column in self.columns if int(column.is_pk)]
if not names:
names = ["rowid"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we want to force people to not use rowid do we?

ignore,
)

if isinstance(pk, Union[str, None]):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK so this one could definitely be a ternary op!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants