release: typedal 6.0

.github/workflows/checks.yml

@@ -0,0 +1,44 @@
+name: run checks
+on:
+  push:
+    branches-ignore:
+      - master
+jobs:
+  check_min:
+    name: Lint, format and test on lowest Python
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      - uses: actions/setup-python@v4
+        with:
+          python-version: '3.12'
+      - uses: yezz123/setup-uv@v4
+        with:
+          uv-venv: ".venv"
+      - run: uv pip install .[dev,all]
+      # ruff + ty
+      - run: edwh lint --output ci
+      # what `edwh fmt` would change: import order and formatting
+      - run: ruff check --select I . && ruff format --check .
+      # pytest with coverage over src/; [tool.coverage.report] fail_under enforces 100%
+      - run: edwh test.run
+
+  check_max:
+    name: Lint, format and test on highest Python
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      - uses: actions/setup-python@v4
+        with:
+          python-version: '3.15'
+          allow-prereleases: true
+      - uses: yezz123/setup-uv@v4
+        with:
+          uv-venv: ".venv"
+      - run: uv pip install .[dev,all]
+      # ruff + ty
+      - run: edwh lint --output ci
+      # what `edwh fmt` would change: import order and formatting
+      - run: ruff check --select I . && ruff format --check .
+      # pytest with coverage over src/; [tool.coverage.report] fail_under enforces 100%
+      - run: edwh test.run

README.md

@@ -2,10 +2,9 @@
 
 [![PyPI - Version](https://img.shields.io/pypi/v/TypeDAL.svg)](https://pypi.org/project/TypeDAL)
 [![PyPI - Python Version](https://img.shields.io/pypi/pyversions/TypeDAL.svg)](https://pypi.org/project/TypeDAL)  
-[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
+[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)  
-[![su6 checks](https://github.com/trialandsuccess/TypeDAL/actions/workflows/su6.yml/badge.svg?branch=development)](https://github.com/trialandsuccess/TypeDAL/actions)
-![coverage.svg](coverage.svg)
+[![checks](https://github.com/trialandsuccess/TypeDAL/actions/workflows/checks.yml/badge.svg?branch=development)](https://github.com/trialandsuccess/TypeDAL/actions)
 
 Typing support for [PyDAL](http://web2py.com/books/default/chapter/29/6).
 This package aims to improve the typing support for PyDAL. By using classes instead of the define_table method,

docs/10_advanced_apis.md

@@ -40,6 +40,106 @@ migrate that table to `TypedTable`.
 
 ## Upsert and validation helpers
 
+### Unique-key upsert in v6
+
+`upsert(key, **values)` inserts or updates by a unique key and returns the resulting
+typed instance. It is a table-class operation; query builders and row instances do
+not expose it. `upsert_async` has the same contract.
+
+```python
+user = User.upsert({"email": "a@example.com"}, name="Alice")
+user = await User.upsert_async({"email": "a@example.com"}, name="Bob")
+```
+
+The key must be nonempty, contain known fields, exclude `id`, and have no `None`
+values; its values must be plain scalars (`str`, `int`, `float`, `bool`, `bytes`, `Decimal`, `UUID`,
+dates and times). Key fields cannot also occur in the values, and `id` can't be a value either
+(it would re-key the row). All of these raise `UpsertKeyError` (a `ValueError`) before any SQL runs.
+The upsert exceptions live in `typedal.exceptions` (all subclass `typedal.TypeDALError`),
+`UpsertHooksWarning` in `typedal.warnings`, and the `UpsertKey` types in `typedal.types`.
+To change a key field, use
+`update_or_insert({"email": "old@example.com"}, email="new@example.com")`.
+A call with only a key returns an existing row unchanged, without after-hooks,
+or inserts a new row using defaults.
+
+#### Native and fallback paths
+
+Only PostgreSQL has a native path: an atomic `INSERT ... ON CONFLICT ... RETURNING`. A key without a
+matching unique constraint or index raises `UpsertKeyError`. PostgreSQL checks the proposed insert's
+constraints before resolving the conflict; provide required insertion values even when you expect to update
+an existing row. The key-only existing-row path avoids attempting that insertion, and skips the
+conflict-target validation because it executes no write.
+
+SQLite and MySQL always use the fallback: a lookup by key followed by an insert or update. PostgreSQL uses it too
+for tables with a common filter or a multi-tenant (`request_tenant`) field, because `ON CONFLICT` would bypass those
+filters, and when the values replace an upload field with `autodelete=True`. The fallback respects the filters,
+does not inspect indexes, and raises `UpsertAmbiguityError` if the key matches multiple visible rows. It is not
+atomic under concurrent writes, so database unique constraints remain recommended. On SQLite 3.35+ the update
+branch reads the row back with `UPDATE ... RETURNING`; MySQL and older SQLite need one extra `SELECT`.
+Conflicts on a different unique column raise the database's normal integrity error on every backend.
+
+#### Hooks
+
+Before-hooks never run on either path because the branch is unknown beforehand.
+After-hooks run for the branch that occurred: `after_insert(row, id)` or
+`after_update(affected_set, row)`. The hook row is PyDAL's operation row, and the
+update set is restricted to the affected ID, ignoring common filters so the hook
+can still access a row moved outside its filter.
+
+Register a before-hook's upsert policy explicitly (also accepted by `before_insert_once`/`before_update_once`):
+
+```python
+User.before_insert(validate_user, upsert="error")  # Require update_or_insert instead.
+User.before_update(normalize_user, upsert="ignore")  # Skip silently during upsert.
+```
+
+An unmarked before-hook you registered emits `UpsertHooksWarning` (pointing at the `upsert()` call) and is skipped.
+An `"error"` registration raises `UpsertHookError` before executing SQL. Policies belong to each model registration;
+they do not change ordinary inserts, updates, or `update_or_insert`.
+PyDAL's own upload hooks, present on every table, don't warn: upsert stores uploaded files itself and, on the update
+branch, removes the replaced file for `autodelete` fields just like a normal update.
+
+The built-in `SlugMixin` and `TimestampsMixin` register their hooks with `upsert="error"`: skipping them would insert
+rows without a slug or leave `updated_at` stale, so `upsert()` on those tables raises `UpsertHookError`.
+Use `update_or_insert` there.
+
+Inserts apply field defaults and computations. Upsert updates write only explicitly
+supplied values: omitted `Field(update=...)` and `compute` fields stay unchanged.
+Supply these values explicitly if they need to change. TypeDAL cache invalidation
+runs through the normal after-hooks.
+
+#### Decimal values
+
+PyDAL writes decimal values into SQL unquoted. Since 6.0, TypeDAL converts every value for a `decimal` field to
+`Decimal` first (for inserts, updates, queries and upserts alike) and raises `ValueError` for anything that isn't a
+finite number, so a string from request data can't change the statement.
+
+### Affected-ID update hooks since 6.0
+
+Updates through TypeDAL's query builders, `db(query).update(...)`, and record updates
+pass an `AffectedSet` (`typedal.types.AffectedSet`, matching the updated rows by primary key) to after-update hooks instead of the original
+query, so hooks still find rows whose filtered columns the update changed. Its `affected_ids` lists the primary
+keys already captured by the write; for keyed tables (`primarykey=[...]`) these are the key values, or tuples for
+composite keys. Calling `.where(...)` or `rows(...)` on it returns a narrowed (plain) `UpdateSet`.
+
+After-update hooks only run when at least one row was updated. That's unchanged from PyDAL, which only calls them
+for a nonzero rowcount.
+
+PostgreSQL and SQLite 3.35+ obtain the IDs with `UPDATE ... RETURNING`. MySQL and older SQLite select the IDs first
+(MySQL with `FOR UPDATE`) and then update exactly those rows, so a row that starts matching in between is neither
+updated nor reported; the returned count is the real rowcount. That costs one extra `SELECT` per update on those
+backends whenever IDs are needed: with an after-update hook (including TypeDAL's own cache invalidation) or for a
+QueryBuilder `update()`, which returns the IDs.
+Raw updates with no after-hooks use ordinary rowcount without collecting IDs.
+Large affected-ID lists currently stay in memory; there is no threshold or temporary-table strategy.
+
+PyDAL's reverse-reference LazySets (`row.articles.update(...)`) and `db.smart_query(...)` construct plain Sets,
+which keep PyDAL's original-query after-hook semantics: their hooks receive that plain `Set` without
+`affected_ids`. Because the original query may no longer match the updated rows, TypeDAL's cache invalidation
+drops every cached result that depends on the table in that case.
+
+### Validation and general update-or-insert
+
 `TypedTable` exposes convenience methods for common upsert/validation flows:
 
 ```python

docs/2_defining_tables.md

@@ -118,7 +118,7 @@ see [their docs](http://www.web2py.com/books/default/chapter/29/06/the-database-
 
 ```python
 from typedal import TypedTable
-from typedal.types import OpRow, Reference, Set
+from typedal.types import AffectedSet, OpRow, Reference, Set
 
 
 class MyTable(TypedTable): ...
@@ -144,8 +144,12 @@ def my_before_update(query: Set, changes: OpRow):
     # return True to cancel
 
 
-def my_after_update(query: Set, changes: OpRow):
-    """`changes` that were applied to the row selection Set"""
+def my_after_update(rows: AffectedSet, changes: OpRow):
+    """
+    `changes` that were applied. `rows` selects exactly the updated rows (by id), even if the update
+    changed the columns the original query filtered on. `rows.affected_ids` lists their ids.
+    Only called when at least one row was updated.
+    """
 
 
 MyTable.before_update(my_before_update)
@@ -168,5 +172,10 @@ row.delete_record()  # to trigger
 MyTable.where(...).delete()  # to trigger
 ```
 
+Since 6.0, `after_update` receives an `AffectedSet` instead of a Set with the original update query
+(see [10. Advanced APIs](./10_advanced_apis.md#affected-id-update-hooks-since-60)).
+Before-hooks also accept an `upsert=` policy, which decides what `upsert()` does with them,
+since upsert can't run before-hooks (see [Unique-key upsert](./10_advanced_apis.md#unique-key-upsert-in-v6)).
+
 Now that we have some tables, it's time to actually query them! Let's go
 to [3. Building Queries](./3_building_queries.md) to learn how.

docs/3_building_queries.md

@@ -169,6 +169,61 @@ Person.join("articles", method="inner")  # will only yield persons that have rel
 
 For more details about relationships and joins, see [4. Relationships](./4_relationships.md).
 
+### cross_join
+
+Since 6.0, a query that mentions a table without relating it to the rest of the query raises
+`ImplicitCrossJoinError` instead of silently producing a `CROSS JOIN` (every row combined with every other row).
+This catches a common mistake: filtering on a table that isn't joined.
+
+```python
+from typedal.exceptions import ImplicitCrossJoinError
+
+Person.where(Article.title == "Hello")  # raises ImplicitCrossJoinError when the query is built
+Person.select(Person.ALL, Article.ALL)  # same: nothing relates article to person
+
+# fine, the comparison links the two tables:
+Person.where(Person.id == Article.author)
+# also fine: any comparison mentioning both tables counts as a link
+Person.where((Person.age + Article.word_count) > 1000)
+```
+
+If you really want a cross join, ask for it explicitly with `cross_join()`.
+Tables reached through a cross-joined table are accepted too:
+
+```python
+Person.cross_join(Color)  # every person combined with every color
+Person.cross_join(Article).where(Article.id == Comment.article)  # comment is linked via article
+```
+
+A relationship join puts the related table in the SQL under an alias. Filtering or ordering on that table
+(`where()`, `orderby()`, `groupby()`, `having()`) is pointed at the alias when the query runs, so it doesn't matter
+whether `where()` comes before or after `join()`, or whether the builder is returned and extended somewhere else:
+
+```python
+Person.join("articles", method="inner").where(Article.published == True)
+Person.where(Article.published == True).join("articles", method="inner")  # the same query
+```
+
+Like `condition_and`, this filters the joined rows too: each person comes back with only their published articles.
+On a left join it also drops the persons without a matching article, and `where(Article.id == None)` finds the persons
+without any. Pagination and `count()` take the filter into account.
+
+When the same table is joined more than once, `Article` alone is ambiguous and raises `AliasedTableMismatchError`
+(a subclass of `ImplicitCrossJoinError`). Pick the join with extra lambda arguments, named after the relationship
+(a nested one as `parent__child`, or just `child` when that name is unique). These are resolved when the query runs,
+so they can also come before the `join()`:
+
+```python
+builder = Article.join("writer").join("reviewer")  # both relationships to Author
+builder.where(Author.name == "ann")  # raises: author is joined as 'writer' and as 'reviewer'
+builder.where(lambda article, reviewer: reviewer.name == "ann")
+```
+
+`delete()` and `update()` ignore joins, so they refuse a builder with such a lambda.
+
+If you want a second, independent copy of a joined table, `cross_join(Article)` makes it explicit: the table name
+then means that copy, not the joined one.
+
 ### groupby & having
 
 Group query results by one or more fields, typically used with aggregate functions like `count()`, `sum()`, `avg()`,

docs/9_memoization.md

@@ -83,19 +83,24 @@ db.memoize(func, data, ttl=datetime(2026, 1, 7))
 
 ## Cache Maintenance
 
-The `typedal.caching` module provides utilities for cache management:
+The `typedal.caching` module provides utilities for cache management.
+Every database with caching enabled has its own cache tables, so each function takes the database to act on:
 
 ```python
-from typedal.caching import clear_cache, remove_cache_for_table, clear_expired
+from typedal.caching import cache_models, clear_cache, remove_cache_for_table, clear_expired
 
 # Remove all cache entries
-clear_cache()
+clear_cache(db)
 
 # Invalidate all cache entries related to a specific table
-remove_cache_for_table(User)
+remove_cache_for_table(db, User)
 
 # Clean up expired entries only
-clear_expired()
+clear_expired(db)
+
+# The cache models bound to this database, e.g. to count entries:
+Cache, CacheDependency = cache_models(db)
+Cache.count()
 ```
 
 You can also use the CLI: