Conversation
Merge feature to main
…#586) * Closes #268: Add display_expression for composite custom object names Adds a Jinja2 display_expression field to CustomObjectType. When set, CustomObject.__str__() renders it with all field values as context instead of falling back to the single primary-field display name. - models.py: display_expression CharField(max_length=500, blank=True) on CustomObjectType; CustomObject._render_display_expression() helper renders it via a SandboxedEnvironment (security); any rendering error silently falls through to the existing primary-field fallback - migration 0015 - forms.py: Display fieldset with display_expression - serializers.py: display_expression in CustomObjectTypeSerializer - customobjecttype.html: shows expression in code block when set - 5 tests: composite render, missing-field fallback, empty expression, rendering error, empty-result fallback Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Move display-related fields into Display fieldset Group verbose_name, verbose_name_plural, group_name, and display_expression into a Display fieldset positioned between Name and URL path/slug. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Clarify help text * Add performance note to _render_display_expression docstring Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Address agent review: caching, error handling, validation, import form, trailing separator docs - Move _compile_display_template() after imports (not in the middle of them) - Cache compiled Jinja2 templates via functools.lru_cache(maxsize=256) keyed by expression string; avoids N recompilations on list views - Move custom_object_type access inside the try block so RelatedObjectDoesNotExist and deserialization errors also fall through silently - Shorten _render_display_expression docstring to one line; move the performance/caching note to inline comments - Add clean_display_expression() to CustomObjectTypeForm: parses the expression via SandboxedEnvironment().parse() and surfaces a ValidationError with the Jinja2 error message on syntax errors - Add display_expression (plus verbose_name, verbose_name_plural, group_name) to CustomObjectTypeImportForm - Extend help_text to warn about trailing separators when referenced fields are blank, with the recommended {% if %} pattern Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Add trailing-separator and form-validation tests; coerce None field values to '' - _render_display_expression: coerce None return from get_display_value() to '' so unset nullable fields render as empty string rather than 'None' - test_trailing_separator_with_blank_optional_field: documents and tests the dangling-separator behaviour vs the {% if %} guard pattern - DisplayExpressionFormValidationTestCase: 4 tests covering valid expression, blank expression, invalid Jinja2 syntax, and unclosed block tag Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Fix ruff E501 long-line violations in test_models.py Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Use core render_jinja2 util instead of rolling our own * migrations: renumber display_expression migration to 0017 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: use RequestContext in CustomObjectLink.left_page() so render_table can access context.request Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: revert render_jinja2 to hand-rolled SandboxedEnvironment; fix migration newline Avoids footgun where JINJA2_FILTERS custom filters pass form validation (plain SandboxedEnvironment) but fail silently at render time (render_jinja2 picks up settings filters). Both paths now use the same environment. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Add branching-aware test configuration and CI coverage for Custom Objects, including a dedicated branching test suite for sync, merge, revert, and cross-COT lifecycle scenarios. Fix branch merge handling for dynamically generated Custom Object models by preserving dependency graph edges when ContentType.model_class() is not available, including self-referential multi-object fields. Create direct M2M target foreign keys as DEFERRABLE INITIALLY DEFERRED and re-defer them after creation so iterative branch merges can insert through table rows before all target objects exist in the main schema. Stabilize the branching test environment by cleaning up branch schemas before flush, closing/terminating client branch connections, tuning PostgreSQL for schema-heavy tests, and limiting the branching CI job to the branching test suite. Also update CI to test against NetBox's pinned requirements without upgrading dependencies, split branching and non-branching test steps, and add compatibility stubs for inherited NetBox view tests.
#620 - avoid loading entire table into memory when opening bulk import/edit/delete pages
…l docs (#623) * Closes #566: Add missing collectstatic step to install instructions README.md and docs/installation.md omitted collectstatic, unlike NetBox core's generic plugin install instructions, which always include it alongside migrate. * Build in-app model documentation for CustomObjectType Even with collectstatic wired into the install instructions, the "Help" link on CustomObjectType's edit page (docs_url, served at static/docs/models/netbox_custom_objects/customobjecttype/) 404'd, since the plugin shipped no source content and no build step for it, unlike NetBox core's own model docs (built via mkdocs directly into its Django static tree). Add docs/models/netbox_custom_objects/customobjecttype.md and a dedicated mkdocs.models.yml (site_dir points into netbox_custom_objects/static/docs/models, mirroring NetBox core's docs_url convention) so this page builds and collects correctly. Wire the build into the release workflow before packaging, since the package-data glob in pyproject.toml already includes any static/ files present at build time. Verified end-to-end: built the wheel locally and confirmed netbox_custom_objects/static/docs/models/netbox_custom_objects/customobjecttype/index.html lands at the exact path Django's AppDirectoriesFinder would collect under STATIC_ROOT. * Drop redundant leading slash in .gitignore entry * Address review findings on model-docs build - Move model doc source from docs/models/ to docs_models/, since the former overlapped with the primary mkdocs.yml's default docs_dir, causing an unreferenced-page warning on every primary docs build. - Switch mkdocs.models.yml to the lightweight built-in "mkdocs" theme and disable plugins, cutting shipped package size from 2.5MB to 1.9MB by dropping mkdocs-material's 40-language search bundle entirely (not needed for a single-page site). - Pin mkdocs/mkdocs-material version ranges in release.yaml, guarding against the upstream-announced breaking mkdocs-material 2.0 rewrite silently breaking a future release build. - Add a PR-time check to lint-tests.yaml that builds mkdocs.models.yml and asserts the expected output file exists, so a broken nav entry or config typo is caught on every PR instead of only at release time. - Add trailing newlines to the two new files, matching the rest of the docs tree. Re-verified end-to-end after these changes: wheel build still places the file at the correct path, and building the primary mkdocs.yml no longer emits the unreferenced-page warning. * Address automated review findings on mkdocs pins and release build - Pin mkdocs/mkdocs-material in pyproject.toml's dev extras to match the ranges in release.yaml, so the lint job (pip install .[dev]) can't silently pass on a mkdocs 2.x or other future release that the actual release build would never see. - Add the same output-verification assertion used in lint-tests.yaml to release.yaml, so a build producing output at the wrong location fails the release instead of silently publishing a package missing the model docs. The hardcoded blob/main GitHub links in the model doc were also flagged; left as-is, matching the existing project-wide pattern (README.md, docs/installation.md already link the same way).
dd a custom_objects namespace and filter for resolving Custom Object Types by name in device configuration and export templates. Support dot, bracket, and filter syntax, with per-render caching and an empty read-only queryset fallback for unknown type names. Log unresolved names once per process and prevent the filter from querying the database during Jinja template compilation. Register the integration through NetBox's jinja_filters resource and get_jinja_context hook. Older NetBox versions continue to load the plugin without exposing the new template helpers. Add unit and integration coverage for ConfigTemplate rendering, unknown types, repeated lookups, leading-digit names, and compile-time query prevention.
) The dynamically generated filterset only registered fields backed by CustomObjectTypeField records. Since the model primary key is not a user-defined field, no filter was registered for `id`, causing `?id=<pk>` to be silently ignored. Include `id` in `Meta.fields` so django-filter generates NetBox's standard multi-value numeric filter for the primary key without affecting the explicitly generated custom field filters. Add regression coverage to the existing filterset and API test classes. The filterset test runs across all supported scalar field types, while the API test covers a Custom Object Type with no fields and verifies that only the matching object is returned.
…lated field edits (#631) * Closes #625: Fix schema_id nulled by unrelated edits to a field CustomObjectTypeFieldForm.Meta.fields = '__all__' pulled schema_id in as a real, writable form field, but it's never rendered in any fieldset. The browser therefore never submits a value for it; Django reads the missing key as None, and ModelForm._post_clean() overwrites the existing schema_id with that None on every save -- even when the user only changed an unrelated attribute like label. Set editable=False on schema_id, matching its existing read_only_fields treatment in the REST API serializer (CustomObjectTypeFieldSerializer) and NetBox core's own convention for internal, system-managed model fields. Django's ModelForm machinery automatically drops non-editable fields from Meta.fields = '__all__', so it's simply absent from the form and never touched. Also excludes deprecated, deprecated_since, and scheduled_removal from CustomObjectTypeFieldForm: same root cause (never rendered in a fieldset, silently reset by an unrelated edit) and same fix shape, but via Meta.exclude on the form rather than editable=False on the model, since these three ARE meant to be writable through the REST API (set via the portable-schema import executor) and are not declared as explicit serializer fields there -- editable=False would have made DRF silently force them read-only in the API too. Verified via git-stash-based regression testing: the new test fails against the pre-fix code with the exact reported symptom (schema_id becomes None after an unrelated edit), and passes after the fix. * Add regression test: deprecation fields remain writable via API Per code review on #631: confirm deprecated/deprecated_since/ scheduled_removal can still be set via the REST API after excluding them from CustomObjectTypeFieldForm, since editable=False (used for schema_id) would have made DRF silently force them read-only too. Verified the test catches this failure mode: temporarily added editable=False to deprecated and confirmed the test fails with the exact symptom (PATCH succeeds but the value isn't persisted). * Hoist import json
…against concurrent readers
create_polymorphic_m2m_table() built and registered a fresh through-model
class and only afterward repointed its "source" FK at the caller's model,
all without holding CustomObjectType._global_lock. A concurrent
get_model(no_cache=True) call -- lock-protected only on its own side --
could land in that window, find the through model already registered, and
repoint "source" at its own (different) model instance instead, leaving the
through's FK and whatever get_model() subsequently caches pointing at two
different classes for the same table. That produced the intermittent
ValueError ("Cannot query 'X': Must be 'TableYModel' instance.") and
RecursionError reported here (recurrence of #477).
Wrapping the build+register+repoint sequence in the same global lock closes
the gap. Added a deterministic regression test that forces a writer thread
(create_polymorphic_m2m_table) and a reader thread (get_model) into the
exact interleaving via a mocked apps.register_model(), rather than relying
on real thread-scheduling luck to land inside the race window.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ments PolymorphicMultiObjectConcurrencyTestCase exercises through-model registration during polymorphic multiobject field creation (a schema operation), not deletion logic -- it only lived in test_deletion.py because the investigation started from the bug's delete-time symptom. Moved it next to the other schema-creation/registry tests it actually belongs with. Also trimmed the docstrings and inline comments, which had grown into multi-paragraph explanations restating the same points -- cut to the essential why (what's already locked, what isn't, and why the fixed case times out rather than deadlocking) without re-deriving the whole investigation inline. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
tests (main) was failing on 4 list-view query-count assertions, expecting 2 more queries than NetBox's current main branch actually issues -- an upstream prefetch/permission-check optimization shaved 2 queries off customobject-simple, customobject-objectfields, customobject-complex, and customobjecttype's list_objects_with_permission checks since these baselines were last recorded. Regenerated via UPDATE_QUERY_COUNTS=1 against a clean checkout of NetBox main (not the locally cached checkout, which was 33 commits behind and had a handful of divergent dependency pins). All four keys dropped by exactly 2 queries each, matching the CI failure precisely; no other keys changed. Re-ran the full plugin suite (1126 tests) against the same environment to confirm nothing else regressed. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ines Fixes #649: Refresh query_counts.json baseline for current NetBox main
reader_done.wait(timeout=2) inside ordered_register_model always times out with the fix applied (R is blocked on _global_lock and can never signal it), taxing every CI run by a flat 2 seconds. The duration only bounds an unavoidable wait; correctness doesn't depend on it, since R's ability to run concurrently is decided by lock state, not by wall-clock timing. Confirmed via 5 runs each way: cutting it to 0.5s still passes reliably with the fix and still fails reliably without it. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* test_forced_registration_interleaving_stays_consistent: assert both threads actually completed after join(), rather than letting a join() timeout silently leave the result dicts empty and the assertions below vacuously pass. * Cover the delete-confirmation GET (issue #640, step 4): obj.delete() realigns each through's "source" FK to type(self) before Django's collector runs, which would silently paper over a lingering registry mismatch that a plain GET -- the actual reported UI path -- does not repair. * Add a regression through the public field-save path (CustomObjectTypeField.objects.create()) with the reported two-type Custom Object setup, instead of only ever starting from an already-persisted field and calling create_polymorphic_m2m_table() directly. A deterministic (mocked apps.register_model()) version of this specific scenario was attempted and abandoned after it produced a genuine deadlock in testing: two threads targeting the identical through table can block each other at the Postgres DDL level while also contending for CustomObjectType._global_lock. Real thread-scheduling concurrency, exercised via 12 looping readers (mirroring the existing single-type test), reaches the same code path safely.
…ultiobject-delete-race # Conflicts: # netbox_custom_objects/tests/test_schema_operations.py
create_polymorphic_m2m_table() held _global_lock across both the build+register+repoint step AND the table-existence probe/DDL. A concurrent CustomObjectTypeField.save() for the same field also calls CustomObjectType.clear_model_cache(), which acquires this same lock: if the lock stayed held across schema_editor.create_model() (an uncommitted CREATE TABLE inside this save()'s own transaction), a second thread blocked on the lock -- itself stuck at the Postgres level waiting on the first thread's uncommitted transaction for the same physical table -- would prevent the first thread from ever reaching clear_model_cache() to commit. Neither side could then make progress. Scope the lock to just the build+register+repoint step; release it before the table-existence probe/DDL runs. Confirmed via a new regression test (two threads double-submitting field creation for the identical (custom_object_type, name)): hangs against the previous, wider-scoped lock (reproduced the exact deadlock signature in pg_stat_activity -- one thread idle-in-transaction waiting on the lock, the other actively blocked on Postgres waiting for the first's uncommitted CREATE TABLE), completes in ~1.5s with the fix.
Bump the maximum supported NetBox version to 4.7.x and record it in the compatibility matrix. No plugin code changes are required: the deferred search-cache flush error against Custom Object dynamic tables is fixed in NetBox core, so 4.7 support depends on a NetBox release carrying that fix.
…st docstring The registration-before-repoint race and its regression test comments were still citing #640 (the unreproduced report this PR doesn't fix) instead of #658 (the actual bug this PR fixes and closes). Left the one reference to #640 that correctly attributes the delete-confirmation-GET test coverage to that issue's own numbered reproduction steps, which #658 doesn't have. Also corrected test_forced_registration_interleaving_stays_consistent's docstring: it described _global_lock as held "for that whole call," which was true before the lock was narrowed to stop before the table-existence probe/DDL. Shortened to describe only what the test itself asserts, with a pointer to #658 for the full analysis.
…ct-delete-race Closes #658: Harden polymorphic multiobject through-model registration against concurrent creation/reads
Confirm compatibility with NetBox v4.7
…lk edit forms (#663) Fields configured with ui_editable=hidden were disabled but still rendered because the form templates render all grouped fields unconditionally. Exclude hidden fields and their backing columns from create, edit, and bulk edit forms while keeping ui_editable=no fields visible and read-only. Reuse the fetched field definitions and coordinate helpers to avoid duplicate queries and naming logic. Preserve hidden non-polymorphic MultiObject fields for internal bookkeeping so configured defaults are applied when creating an object and existing relations remain unchanged when editing one. Add regression coverage for regular, coordinate, read-only, and MultiObject fields.
Merged
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Merge 'main' into 'feature'.