Skip to content

Closes: #665 - Support YAML for the portable schema export/preview/apply feature - #668

Open
bctiemann wants to merge 3 commits into
featurefrom
665-yaml-schema-interchange
Open

Closes: #665 - Support YAML for the portable schema export/preview/apply feature#668
bctiemann wants to merge 3 commits into
featurefrom
665-yaml-schema-interchange

Conversation

@bctiemann

@bctiemann bctiemann commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Closes: #665

Adds YAML support for the portable schema export/preview/apply endpoints (SchemaPreviewView, SchemaApplyView), per #665. JSON remains fully supported and is still the default when no Accept/Content-Type header is given, so this is backward compatible.

  • api/parsers.py: YAMLParser(BaseParser), media_type = 'application/yaml', parses with yaml.safe_load.
  • api/renderers.py: YAMLRenderer(BaseRenderer), media_type = 'application/yaml', format = 'yaml', renders with yaml.safe_dump(data, sort_keys=False, default_flow_style=False).
  • api/views.py: _SCHEMA_PARSER_CLASSES = [YAMLParser, JSONParser] and _SCHEMA_RENDERER_CLASSES = [JSONRenderer, YAMLRenderer, FormlessBrowsableAPIRenderer] (JSON first so a request with no Accept header still gets JSON).
  • pyproject.toml: adds PyYAML as a dependency.
  • docs/portable-schema.md: documents both formats.
  • Tests: tests/schema/test_yaml_codec.py (parser/renderer round-trip), plus new cases in tests/schema/test_schema_api.py covering content-negotiated preview/apply requests.

Rebased onto the now-merged feature (post #666/#667), which brought in the Django 6.1 from_db() fix. Full suite verified in a clean venv (no stray netbox_branching package on the path): 1218 tests, 0 failures/errors, 8 skipped.

Adds YAML as an accepted and documented-preferred interchange format for
Custom Object Type schema documents, alongside JSON.

- Add PyYAML as an explicit dependency (previously only an incidental
  transitive one via NetBox core).
- Add YAMLParser/YAMLRenderer (netbox_custom_objects/api/{parsers,renderers}.py),
  built directly on PyYAML rather than pulling in djangorestframework-yaml.
- SchemaPreviewView and SchemaApplyView now accept either
  Content-Type: application/yaml or application/json, and render YAML when
  Accept: application/yaml is sent. A request with no Accept header still
  gets JSON, so existing callers are unaffected.
- export_cot()/export_cots() are unchanged -- they already returned a plain
  dict and left serialization to the caller; only the documented examples
  and the two API endpoints change.
- Update docs/portable-schema.md examples to YAML, and clarify that
  cot_schema_v1.json is a format-agnostic JSON Schema validator (it
  validates the parsed document, not a specific wire encoding).

Closes: #665
…oints

- SchemaApplyView.post() called request.data.get(...) before checking
  request.data was a dict, so a non-dict top-level body (e.g. a YAML/JSON
  list) crashed with an unhandled AttributeError (500) instead of the
  documented 400. Now validated up front.

- YAMLParser silently turned an empty body into {}, while JSONParser (used
  on the same endpoints) raises ParseError (400) for an empty body. YAMLParser
  now raises ParseError for an empty or null document too, so both formats
  fail the same way for the same input.

- yaml.safe_load uses PyYAML's default YAML 1.1 boolean resolver, which
  coerces unquoted no/yes/on/off (any case) to booleans -- the "Norway
  problem" -- so a schema field named/valued "no" would silently become
  False. YAMLParser now uses a _StrictBoolLoader that only resolves
  true/false as booleans.

Added tests for all three in test_yaml_codec.py and test_schema_api.py.
@bctiemann

Copy link
Copy Markdown
Contributor Author

@claude /review and limit response to actionable issues.

@github-actions

github-actions Bot commented Aug 19, 2026

Copy link
Copy Markdown

Claude finished @bctiemann's task in 3m 13s —— View job


PR Review: Support YAML for the portable schema export/preview/apply feature

  • Gather context and read changed files
  • Analyze code for issues
  • Post review findings

The implementation is clean and well-structured. Three actionable issues:


1. YAMLRenderer.render() returns str instead of bytesapi/renderers.py:16

yaml.safe_dump() returns a str. DRF's BaseRenderer contract says renderers should return bytestringsJSONRenderer does. DRF does handle this case in Response.render() via ret.encode(renderer.charset), so it works in practice, but it's against convention and could surface with middleware that inspects the raw render output before encoding.

# renderers.py:16 — current
return yaml.safe_dump(data, sort_keys=False, default_flow_style=False)

# fix
return yaml.safe_dump(data, sort_keys=False, default_flow_style=False).encode(self.charset)

Fix this →


2. Misleading error message for a valid null YAML document — api/parsers.py:44-45

Sending null as the request body returns "YAML parse error: request body is empty." — but the body isn't empty, it's a valid null document. The yaml.load("null") returns None and the code conflates that with an empty body. Someone debugging a rejected request would be misled.

# parsers.py:42-45 — current
if data is None:
    raise ParseError("YAML parse error: request body is empty.")

Fix: distinguish empty from null, or use a more accurate message:

if data is None:
    raise ParseError("YAML parse error: request body must be a mapping (object), not null or empty.")

Or read the raw bytes first to distinguish the two cases.

Fix this →


3. SchemaPreviewView missing non-dict body guard — api/views.py:358-362

SchemaApplyView (line 434) explicitly guards against a non-dict body before doing anything else:

if not isinstance(request.data, dict):
    raise ValidationError({"non_field_errors": [_("Request body must be a JSON/YAML object.")]})

SchemaPreviewView.post() has no such guard — a YAML list or scalar body goes straight to _validate_schema_doc(), which passes it to jsonschema and produces {"schema_errors": [...]} rather than the consistent non_field_errors format. Functionally a 400 either way, but the error shapes differ.

Fix this →


Everything else looks solid — the _StrictBoolLoader Norway-problem fix is correct, content negotiation ordering (JSON first for backwards compat) is right, test coverage is thorough, and the copy.deepcopy approach for mutating the resolver table doesn't affect the base SafeLoader.

- YAMLRenderer.render() now returns bytes (encoded with self.charset)
  instead of str, matching DRF's renderer convention (JSONRenderer does
  the same). DRF's Response.render() happened to paper over the str case,
  but middleware inspecting the raw render output before encoding would
  not have.

- YAMLParser's ParseError for a None-parsed document no longer claims the
  body is "empty" -- yaml.load() can't distinguish an empty body from an
  explicit `null` document, and the old message was misleading for the
  latter. Message now says the body must be a mapping, not null or empty.

- SchemaPreviewView.post() now guards against a non-dict top-level body
  the same way SchemaApplyView already did, so both endpoints return the
  same non_field_errors shape for a non-dict body instead of preview's
  YAML-list/scalar case falling through to jsonschema's schema_errors
  format.

Updated tests accordingly (renderer output is bytes now; added
non-dict-body coverage for SchemaPreviewView). Full suite verified in a
clean venv: 1227 tests, 0 failures/errors, 8 skipped.
@bctiemann

Copy link
Copy Markdown
Contributor Author

Addressed all three:

  1. YAMLRenderer.render() now returns bytes per DRF convention.
  2. The null/empty-body ParseError message no longer misleadingly claims the body is "empty" when it's actually a valid null document.
  3. SchemaPreviewView now has the same non-dict-body guard as SchemaApplyView.

@bctiemann
bctiemann requested review from a team and jnovinger and removed request for a team August 19, 2026 23:42
@bctiemann bctiemann changed the title Support YAML for the portable schema export/preview/apply feature Closes: #665 - Support YAML for the portable schema export/preview/apply feature Aug 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant