[python] Add permission endpoints to the REST mock server

[plusplusjiajia]

Purpose

PyPaimon manages permissions through the REST catalog, and reads of query-auth tables check COLUMN grants through AuthTableQuery, but the Python mock REST server has no permission endpoints, and its AuthTableQuery endpoint only returns the response a test stores in table_query_auth_store. This adds the permission endpoints and the COLUMN grant check, after Java RESTCatalogServer:

• Permissions are listed, granted and revoked, and listings honor maxResults and pageToken.
• Grants are removed or moved with the table, database or function they name, and COLUMN grants follow column renames.
• A COLUMN grant needs query-auth.enabled and existing columns, and stops applying at its expire time, read with the same instant rules as PermissionAssignment. A schema change is refused if it would disable query authorization or drop every allowed column of a COLUMN grant.
• AuthTableQuery checks the selected columns against the COLUMN grants of the principals the caller acts as (set with set_query_principals) before it answers with the response in table_query_auth_store.
• Tables with branches cannot take COLUMN grants, and tables with COLUMN grants cannot get branches, because grants belong to the table while a branch carries its own schema.

rest_permission_server_test.py covers these rules and runs in every job. sql_permission_test.py checks that grants made with the CALL sys.* procedures are listed by PyPaimon and the other way round, and that a COLUMN grant restricts PyPaimon and the DataFusion SQLContext of pypaimon-rust alike. It is marked native_plan, so it runs only in the Native CI job, which builds paimon-rust from main; the procedures test needs pypaimon-rust 0.4 and the read test 0.5, and both are skipped on older versions.

Data policies follow in a separate PR.

Tests

pypaimon/tests/rest passes (408 tests, excluding one that needs the optional lance package), and on Python 3.6 the mock server tests pass. With pypaimon-rust built from apache/paimon-rust main (4ceef6b) and all five PYPAIMON_TEST_NATIVE_* switches set, sql_permission_test.py passes together with the native query auth, REST plan and SQL context tests; with pypaimon-rust 0.4.0 its read test is skipped, and with 0.3.0 both tests are.

[JingsongLi]

The permission/procedure interoperability coverage has clear end-to-end value. I found one mock-server contract issue below.

Validation at 3e1697e: 208 focused permission/query-auth tests passed; the broader REST run had 378 passes and one environment failure because the optional lance package is absent. Changed Python files passed the repository Flake8 configuration, and license validation passed. The exact head's Native CI is green. My local installed Rust binary predates read-time query auth, so its SQL failures are not attributed to this PR.

[JingsongLi]

[P2] Collect name-only FIELD_REF references before validating schema changes

A valid row-filter fieldRef can omit index, e.g. {"kind":"LEAF","transform":{"name":"FIELD_REF","fieldRef":{"name":"id","type":"BIGINT"}},"function":"EQUAL","literals":[1]}. Both the existing Python predicate parser and Java's FieldRef/remapping support this. Requiring index here makes _policy_columns return an empty set. I reproduced through HTTP: create this policy on id, then rename id to user_id; the rename succeeds and the stored policy still references id. This bypasses the new policy-preserving schema guard and leaves subsequent authorized reads with a stale policy. Please recognize field references by their transform structure/name, and cover the name-only form for policy creation and column rename/drop.

[plusplusjiajia]

Thanks for catching this! The mock now reads policy columns from every fieldRef and transform input, with or without index, and the tests cover name-only refs for policy creation and column rename/drop.

[JingsongLi]

Requirement fit: SUPPORTED. The cross-client permission/read coverage has end-to-end value. The previous name-only fieldRef finding is fixed: both direct fieldRef and transform inputs now protect referenced columns before schema changes are committed.

I re-reviewed the full diff and reran the permission/policy/mock tests (49 passed), auth/JSON/validation tests (262 passed), and the broader REST suite (418 passed initially; the sole missing-DuckDB environment failure passed after installing the declared optional dependency). Changed files passed the repository Flake8 configuration. I also verified the exact-head Native CI log: all five SQL permission tests passed with Rust 0.5.0 at 3bc53b4d5f2fcd54d4f4bcbc2dec2aea1c4357a6; I did not rerun that native SQL lane locally.

One remaining mock/API contract mismatch is reproduced below. Two added JDK 8 Java reference probes passed without fast-build, while the equivalent Python HTTP rejection probe failed because the mock accepts the invalid mask.

[JingsongLi]

[P2] Validate mask output types as well as referenced column names

This validation accepts a mask for STRING email with {'name':'YEAR','fieldRef':{'name':'joined','type':'DATE'}}, because both column names exist. I reproduced through catalog.policy_management().create_policy: it succeeds, while an assertion expecting BadRequestException fails. YEAR produces INT; the documented management contract requires a mask's output type to match its protected column, and Java RESTCatalogServerUtils.parseColumnMask rejects this mismatch. Direct Java probes confirmed the rejection.

The new successful fixtures also depend on this looser mock: NULL_MASK is used on STRING email, whereas Java's NullTransform.outputType() is BOOLEAN and its reference server rejects that policy too. Thus the cross-client pass can exercise policies the reference service cannot create. Validate schema-bound mask transforms before storing them and when validating schema changes; add a wrong-output-type negative case. Use a valid STRING mask for email, and a nullable BOOLEAN column for NULL-mask coverage (or separately establish and synchronize a broader NULL policy contract). This finding concerns the new mock/test validity, not an unchanged Java implementation regression.

[plusplusjiajia]

Thanks for the careful check! To keep this PR focused, it now covers only permissions and the COLUMN grant check of AuthTableQuery, and the data policy endpoints move to a follow-up PR. There a column mask must return its column's type, your YEAR-on-email case is a rejection test, and the SQL tests use a STRING mask on email and NULL on a BOOLEAN column.

[JingsongLi]

Requirement fit: SUPPORTED. Implementation: FINDINGS (one P2 mock/API contract mismatch). The permission and query-auth endpoints provide useful HTTP integration coverage. The earlier field-reference and mask-type concerns are addressed.

I reviewed the full diff, grant/principal/expiry composition, rename/drop isolation, paging, policy typing and schema evolution against Java. All 421 local REST tests passed; changed files passed Flake8. An additional real HTTP probe and a JDK 8 Java parser reference reproduce the remaining mismatch described inline. This finding concerns test-server fidelity, not a production authorization bypass.

[JingsongLi]

[P2] Reject unknown row-filter predicate functions

A row filter with a valid FIELD_REF but an unknown predicate function is accepted here because only its transforms are validated. I reproduced this through the public REST management client on this head: creating a LEAF with function="DOES_NOT_EXIST", literals=[1], and FIELD_REF(id BIGINT) returns success, stores the policy, and returns it from query auth. Java RESTCatalogServerUtils.parseRowFilter deserializes Predicate; the same JDK 8 probe fails with "Could not resolve leaf function", which validatePolicy maps to HTTP 400. A typo in a policy can therefore pass these new integration endpoints even though the Java server rejects it. Validate the leaf/compound predicate functions before storing the policy and add an HTTP rejection regression.

[plusplusjiajia]

Thanks! This moves to the same follow-up PR, where predicate functions are checked against Java's leaf and compound functions and an unknown function is a rejection test.