Skip to content

Keep select_related column aliases within identifier length limits - #2395

Open
sinnercode228 wants to merge 2 commits into
tortoise:developfrom
sinnercode228:fix/select-related-long-alias
Open

sinnercode228 wants to merge 2 commits into
tortoise:developfrom
sinnercode228:fix/select-related-long-alias

Conversation

@sinnercode228

Copy link
Copy Markdown

Description

select_related() aliases every column of a joined table as "<table name>.<column>", and BaseExecutor.execute_select takes the field name from the part after the first .. PostgreSQL truncates identifiers to 63 bytes. With a long table name the alias comes back cut short, the field name no longer matches any field, and that field is silently dropped. The related instance ends up partial, and save() raises IncompleteInstanceError.

With the models from the issue on PostgreSQL 16:

SELECT ...,"verylongnamefortestingpurposesonlyuserinfo"."not_too_long_user_age" "verylongnamefortestingpurposesonlyuserinfo.not_too_long_user_age", ...

The alias is 64 bytes. It comes back as ...not_too_long_user_ag, so the loaded instance is {'_partial': True, 'user_id': 1, 'id': 1}, without not_too_long_user_age.

This PR uses the index of the joined table in _joined_tables as the alias prefix ("0.not_too_long_user_age"), in both _join_select_related and _resolve_only, through one helper. The executor only needs the prefix to be unique per joined table, and it is. When the same table is selected again (e.g. select_related("left", "left__left")), it gets the same index and the same aliases, as it did with the table name. The executor is unchanged.

The prefix no longer depends on the table name, so the alias is only too long if the column name alone is close to the limit.

Not addressed here: table aliases of deep nested joins (table__field__field...) are truncated as well. That does no harm unless two of them share the first 63 bytes, which needs a far deeper chain than the column case.

Motivation and Context

Fixes #2018. It only shows up on PostgreSQL (SQLite has no identifier limit), which is why it could not be reproduced on SQLite in the issue. The alias format predates #1923, so the cause is the 63-byte limit rather than that PR.

How Has This Been Tested?

  • New test_select_related_with_long_table_name in tests/test_relations.py, with a self-referencing model ModelWithATableNameLongEnoughToHitIdentifierLimits added to tests/testmodels.py ("<table>__parent.title" is 64 bytes). It checks the related instance's fields after select_related("parent") and that it can be saved. On develop it fails on PostgreSQL with both asyncpg and psycopg (AttributeError: ... object has no attribute 'title') and passes on SQLite. With the change it passes on all three.
  • The full suite on SQLite, SQLite with regexp functions, and PostgreSQL 16 with asyncpg and psycopg, plus make check.
  • MySQL, MSSQL and Oracle were not run locally.

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have added the changelog accordingly.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

select_related() aliased each column of a joined table as
"<table name>.<column>", and the executor took the field name from the part
after the first ".". PostgreSQL truncates identifiers to 63 bytes, so for a
long table name the alias came back cut short, the field name no longer
matched, and the field was silently dropped. The related instance was then
partial and save() raised IncompleteInstanceError.

Use the index of the joined table as the prefix instead. The executor only
needs the prefix to be unique per joined table, and the same table still
gets the same alias when it is selected twice.

Fixes tortoise#2018
@codspeed

codspeed Bot commented Nov 3, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

1 warning

⚠️ 21 benchmarks are dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

✅ 24 untouched benchmarks


Comparing sinnercode228:fix/select-related-long-alias (543282a) with develop (cdef122)

Open in CodSpeed

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Misbehaving model save() when names are too long

1 participant