Repository navigation
Solid models with metadata - #328
Conversation
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
7071464 to
3fb234f
Compare
|
I'm still digesting this, but wanted to put down notes on my understanding so far. Requirements are something like:
Q&A: Q1. Why LayerRef rather than SourceLayer itself? Q2. Why LayerRef rather than SemanticMeta (i.e., why drop Q3. Is R1's "one and only one PG" true even if the second group containing the entity is not used in SolidModel configuration? Q4. Can't an entity be in zero groups? Q5. How does this relate to Q6. Why Locator and not extra metadata or style on an ordinary object? Q7. Why keep Q8. Why Q9: Why does thickness of an extruded generated layer live in a different place from source layer thicknesses (in an extrusion operation rather than a layer struct)? The overall shape makes sense to me. I'm not sure it can't be broken up but I'll have to think about it. |
3fb234f to
07c3c47
Compare
|
Read your note -- your understanding is correct on all fronts. Btw, while you review the code, one note I'd add here is that I'd like to put the |
| function SourceLayer( | ||
| material::Material; | ||
| level::Union{Int, Pair{Int, Int}}=1, | ||
| offset::Union{Coordinate, NTuple{2, Coordinate}}=0μm, |
There was a problem hiding this comment.
If level is a Pair, should this default to (0μm, 0μm)?
Also, I think this needs to be Union{Coordinate, NTuple{2, <:Coordinate}}, most ways of constructing the tuple will give it a narrower eltype than Coordinate.
There was a problem hiding this comment.
If level is a Pair, should this default to (0μm, 0μm)
Yeah good call. I think in practice this didn't really matter because I always access offset via first(offset) and last(offset), which are safe regardless of whether it is a scalar or tuple, but I'll change the default anyway.
Also, I think this needs to be Union{Coordinate, NTuple{2, <:Coordinate}}
I tried this, but the following fails to construct: SourceLayer(NULL; level=1 => 2, offset=(10μm, -5.4μm))
There was a problem hiding this comment.
Ah right, both that and (10μm, -5μm) should still be a Tuple{<:Coordinate, <:Coordinate)
There was a problem hiding this comment.
Ok I'll leave it as-is then.
| for (layer_name, state) in registry | ||
| source_layer = get(stack.layers, layer_name, nothing) | ||
| isnothing(source_layer) && continue | ||
| (source_layer.material == METAL && state.dim == 2) || continue |
There was a problem hiding this comment.
Does this miss the boundaries of an extruded metal layer?
There was a problem hiding this comment.
It does, but I think that's a good thing. If users want the boundaries of an extruded metal layer to be tagged as terminals, they should explicitly isolate them into a 2D layer, for example by calling Hollow(:metal) or GetBoundary(:metal, :metal). One of the design principles of this PR is that everything (or at least, as much as possible) should be done via explicit operations, rather than implicitly through conventions that may or may not be well documented.
That said, writing this I realize that right now, this only works if you use in-place operations. Out-of-place operations such as GetBoundary(:boundary, :metal) emit a generated layer which does not have a stack entry and therefore doesn't have METAL as its material. Even if you're not trying to extract boundaries of extruded metals, there may be footguns, e.g. if you call Fuse(:all_metal, [..., :metal, ...]) followed by Remove(:metal; remove_entites=false), the metal entities that are still in the model won't get picked up.
So this makes me wonder if the registry should track materials too, even of generated layers, assuming we can come up with logical rules for deriving them for each operation from their object and tool layers' materials.
There was a problem hiding this comment.
I see, I'm not convinced we need that immediately. If it's OK with you to leave materials for generated layers as a potential breaking change, then we can wait and see what comes up.
There was a problem hiding this comment.
Yeah that's ok with me.
There was a problem hiding this comment.
Maybe one thing to add here, if someone leaves out their Hollow(:metal) and only has extruded 3D metal groups, they get the early return Selection[] and don't see any of the errors below.
|
One thing I'd consider in light of #344 is whether/how we want to support material priority. Seems really useful, I think we've been underrating how much we can do with post-fragment set differences where it's easy to think in terms of priority (as opposed to pre-fragment geometry differences). It's not really a per-render setting but a consequence of what |
|
Let me see if I understand the point you brought up about material precedence. I see what Layla's PR is doing but the only thing I don't get yet is where those operations (the ones generated by @laylagi for vis, in case you want to explain/clarify some of my questions. |
2167da9 to
c186468
Compare
In that pipeline they'd go into The idea is useful in 2D as well, but the 3D is what's caused multiple OCC/mesh issues, so special-casing 3D booleans into post-fragment set differences might be useful. It probably doesn't have to be decided now. Using only explicit operations works for me. I think the main concerns are more chances for the users to do the wrong thing and repetition across targets but those are both manageable. Also I suppose the conformal pipeline will never actually extrude or do 3D booleans so the interpretation of those operations will be a stretch, but the information is there. |
|
Ok then if it's ok with you, let's just defer the post-fragment set differences for 3D booleans to a future PR? |
|
Thanks for the ping @parrangoiz , Greg covered the |
|
In terms of timing and which one will merge first, Greg knows best since he's reviewing everything. I'll take a look at |
|
Yeah I expect I'll merge #331 first. So if we wanted to use both together I think it looks something like
It's not too late to change the design of #331 but I don't think there are fundamental obstacles in that series of PRs. The stuff above doesn't necessarily have to happen in this PR but I would like to get to a consensus on how it will be handled before merging. |
|
I think the problem with circularity in Edit: on closer look at #331, it actually doesn't seem like it'd be that bad of a change. First we'd need to remove |
| for (layer_name, state) in registry | ||
| source_layer = get(stack.layers, layer_name, nothing) | ||
| isnothing(source_layer) && continue | ||
| (source_layer.material == METAL && state.dim == 2) || continue |
There was a problem hiding this comment.
Maybe one thing to add here, if someone leaves out their Hollow(:metal) and only has extruded 3D metal groups, they get the early return Selection[] and don't see any of the errors below.
8cd5fb2 to
c787e81
Compare
Yeah good point. Indeed I wouldn't want that case to result in a silent return with no selections. OTOH, maybe the user never intended to have 2D metal in their model, and having this produce an error would be annoying at best. Instead how about we add the flags |
That seems reasonable, I'm happy with ddfe088. |
ddfe088 to
d344a8b
Compare
|
Looks good to me, could you just add a CHANGELOG entry for the new module and the #287 fix? |
Julia 1.10 and 1.11 reject assigning a testitem-level variable whose name was already resolved as an import, which failed CI when a local `layer` followed calls to DeviceLayout's `layer`. Rename test variables that shadow exported functions (`layer`, `geometry`, `graph`, `component`, `cell`) throughout the file, and update the unused `inspect_registry` helper for the string-only layer registry.
Replace SourceLayer's `level::Union{Int, Pair{Int, Int}}` and
`offset::Union{T, NTuple{2, T}}` with `level::Int`, `offset::T`,
`destination_level::Union{Int, Nothing}`, and `destination_offset::T`, and
add `_spans_levels` to test for a destination level. The keyword constructor
still accepts `level=a => b` and an offset pair, promoting all lengths to a
common type that the new inner constructor converts to.
The inner constructor rejects a destination level combined with a nonzero
thickness. The keyword constructor rejects an offset pair without a level
pair, and a scalar offset with a level pair displaces both ends equally.
SourceStack validation, layer_z, thickness, artwork, and serialization read
the new fields directly.
Change SourceStack{L, T} to SourceStack{T} with
`layers::Dict{Symbol, SourceLayer{T}}`, so every layer in a stack shares the
stack's coordinate type. The constructor promotes all layer and level
coordinates to a common concrete T after its existing validation and
converts layers and levels to it, via a new Base.convert method for
SourceLayer. SolidModelTarget{L, T} becomes SolidModelTarget{T} to match.
WavePort now carries only a layer and name and is constructed as WavePort(layer, name); LumpedPort keeps its index. layerindex reads the index only for LumpedPort, so WavePort falls back to the default of 1. Locator name qualification gets separate WavePort and LumpedPort methods, and serialized wave ports are keyed by name alone with no "index" entry.
Widen the JSON compat bound from "0.21" to "0.21, 1". JSON is a test-only dependency, and the SolidModel end-to-end and Aqua test items pass under both JSON 0.21.4 and 1.10.0 with no deprecation warnings.
Narrow the JSON compat bound from "0.21, 1" to "1", dropping support for JSON 0.21. JSON is a test-only dependency; the SolidModel end-to-end and Aqua test items pass under JSON 1.10.0.
Raise the two connected-component diagnostics in _select_ccs! from @warn to @error: a Terminal or Ground locator that matches no metal connected component, and a metal connected component with no Terminal or Ground locator. Under the default strict=:error, render! now fails on either, while strict=:no still allows rendering to continue.
Add tests that offset tuples whose elements have different numeric types or units, such as (10μm, -5.4μm) and (10μm, -5.4nm), are accepted and promoted to a common type, and that non-coordinate, wrong-length, and mixed unitful/unitless offset tuples throw TypeError.
Wave ports are vertical patches on the model boundary, built by placing a boundary LineSegment on a source layer below the port and extruding it up through the cross section. WavePort locators now select every vertical piece of such a patch through the locator's (x, y), found with the same R-tree prefilter as planar locators; other locators still select the single coplanar entity at their z. Only lumped ports carry a direction: wave ports need no WithDirection entity and serialize without "direction" or "index", and the metadata schema requires both for lumped ports and forbids them for wave ports. The initial compiler registry registers layers made only of LineSegments as 1D so that Extrude builds their patches. This relies on the LineSegment placement, conversion, and bounds fixes from arrangop/linesegment-fixes. The end-to-end test adds a Paths.CPW drive line per transmon fed by a wave port on the chip boundary, rotates q2 by fusing the east coupler hooks so both drive lines reach the boundary, and checks the wave-port selections, the drive-line terminals, and schema validation.
Component geometry resolution recovers from failures while a renderer's resolution context is active, but it also caught InterruptException, so Ctrl-C during a render was logged and skipped instead of stopping it. Now interrupts are always rethrown. Add tests of the recovery path: failures propagate without a context; within one they are logged once, return an empty cached geometry without rerunning the component, clear its partially built cache, and leave no context behind; interrupts propagate; and solid-model renders recover under strict=:no and fail under strict=:error, including the experimental render.
A model whose metal was only extruded 3D (no Hollow) had no 2D METAL layers, so terminal selection returned nothing and its Terminal and Ground locators were silently dropped. Add select_terminals, select_tagged_surfaces, and select_ports keywords (all true by default) to the experimental render! so users can skip selections that don't apply to their model. Terminal selection without 2D METAL layers, and tag or port locators on an unrealized layer or matching no entity, now log errors naming the flag that disables the selection, so the strict mode decides whether the render fails. Document the flags, pass select_terminals=false in the example and the compiler integration fixtures (which have no metal), and test the opt-outs and the logged errors.
The compiler absorbed a Remove that immediately followed a Cut, Fuse, or Intersect into the boolean's OCC removal flags. Absorption is needed for correctness, since geometry that passes through a boolean unchanged keeps its entities and a separate Remove of an input also deletes them from the result, but it only covered adjacent removals. Expose the flags directly instead: Cut and Intersect take remove_object and remove_tool, and Fuse takes remove_sources, all false by default like the PG-level boolean functions. A flag naming an input that the destination replaces is ignored with a warning, and Cut now rejects an object that is also one of its tools. Remove the lowered operation types and the absorption pass, document the flags and the Remove pitfall, and update the tests and the end-to-end operation list to use the flags.
render!(cs, sch, target::LayoutTarget) documented that strict=:no renders a failing component as an empty geometry, but it installed no geometry resolution context, so the component's exception escaped even under strict=:no and left the log file open. Run the render inside with_geometry_resolution_context and a try/finally that closes the log file, as the solid-model render does, and apply the strictness check afterwards. Move check_render_strict next to it and give it a stage argument so all three schematic renders share it. Test that strict=:no renders the remaining components and logs the failure, and that strict=:error throws.
A Cut whose destination was its object or one of its tools passed remove_object/remove_tool = false to difference_geom!, so the result took over the input's physical group while the input's original entities stayed in the model ungrouped. They still took part in fragmentation and split their neighbours along spurious internal faces, which could produce degenerate elements when meshed. Consume the replaced input in the OCC call, as Intersect already does. OCC consumes all tools (or all fused sources) or none, so a Cut whose destination is one of several tools now requires remove_tool=true, and a Fuse whose destination is one of several sources requires remove_sources=true; both constructors throw otherwise. Update the docstrings and the concepts page. Test the compiled flags and the new constructor errors, and render in-place Cut and Fuse cases end to end, checking that no entity is ungrouped when the model is fragmented and that the resulting areas are right.
Translate compiled to translate! without a dimension, so it always looked up the 2D group and failed for 1D and 3D layers; it now passes the layer's dimension. Extrude and Translate passed unitless stack lengths straight to extrude_z! and translate!, which require lengths and threw a DimensionError; unitless lengths are now converted from STP units (µm) with a new _stp_length helper next to _stp_float. A Cut whose tools had a different dimension than its object silently skipped them; it now throws an ArgumentError, as Fuse already does for mixed dimensions. Revolve took only unitless origins and angles. It now stores its origin as the given coordinates, like Translate, converting them to STP lengths when compiled, and converts its angle into a Float64 field as StructureReference does for rot: a unitful angle is converted to radians, and a unitless one is taken as radians. Add GetBoundary(source; kwargs...), which extracts a layer's boundary in place, matching the one-layer forms of the other operations. Serialized terminal locator names are de-duplicated, so a component carrying the same terminal name twice produces schema-valid metadata. The experimental render! no longer rejects an output_dir keyword; like other keywords, it is passed on to SolidModels.render!. Test translating a solid and a unitless stack, the Cut dimension check, unitful Revolve origins and angles in compilation and rendering, and the in-place GetBoundary constructor.
…n blocks Blue style reserves short-form definitions for ones that fit on a single line. The tuple Cut constructor's signature and the SourceLayer convert method's body each span several lines, so define them with function ... end.
Rewrite the concepts page, retitled "Solid Models (Experimental)", to state what the pipeline does and the constraints a user must meet: LayerRef geometry and locators, source stacks and their materials and units, extrusion of declared thicknesses, the operations and their in-place forms, terminal and ground selection, locator name prefixing, the metadata sections and keys, and artwork levels. Its examples form one runnable sequence. This corrects statements about flattening, artwork levels, and Hollow, and removes implementation details and advice. Add an API reference page for the experimental module, linked from the reference and concepts indexes. Rewrite the module's docstrings to match: keywords listed under a Keywords section with the signature ending in `; kwargs...`, simple return types in the signature, and the unitful Revolve origin and angle and the in-place GetBoundary documented. Add a Material docstring, document SolidModels.render!'s preflattened keyword, and correct the in-place Intersect comment. Update the example to use a METAL layer and a Ground locator, and add changelog entries.
State what the SolidModelsExperimental module does without listing every operation, shorten the preflattened entry, and reduce the geometry-failure fix to the behavior it fixes, with a reference to #287.
The strictness and selection tests printed their expected render errors and gmsh warnings, which hid real failures. Each scenario now plans and renders its schematic under with_test_logger, requiring the errors it expects and failing on any other log; the renders that should log nothing allow no logs. Expected errors are identified by level, logging module, and keywords rather than message text. The items use QuietGmshSetup instead of raising gmsh's verbosity, and the end-to-end test no longer prints its artifact directory.
The example rendered a single rectangle and showed little beyond what the concepts page already does. Remove it and the concepts page's reference to it.
Retitle the page "Solid Models with Metadata (Experimental)" and introduce the module as a way to attach locator-selected terminals, ports, and surfaces to a solid model and export them as JSON. Explain SourceLayer, SourceStack, and LayerRef as replacements for SemanticMeta and ProcessTechnology, and walk through an Imon-style transmon: its stack, its CAD kernel operations, its placements and locator selections, and rendering the model, with an image of it, and saving its metadata, with an abridged example of the metadata file. Explain that selection names are prefixed with their node's id, and that the new pipeline ignores SemanticMeta geometry while the legacy targets render LayerRef geometry and fail on locators.
ea78106 to
eacc30b
Compare
|
Ok done, all my tests are silent when passing. Also done with the docs pass. I'll go ahead and merge. |
This PR introduces a new solid model rendering pipeline that enables marking special objects in a model -- such as terminals or ports -- and annotating them with metadata. The marked objects are isolated into dedicated physical groups (PGs) and the metadata is returned by
render!in a structured format that can later be used to programmatically generate Palace configurations. By design, the metadata is purely semantic and geometric, and does not include "physics configuration" such as material properties or port impedances. This enables a clean split between CAD+mesh generation and downstream configuration of any Palace simulation from a single source mesh.To enable this, I've added a few new concepts to the public API (all encapsulated in an opt-in module):
SourceLayer. This is a natural extension ofSemanticMeta: it has a name and a level, but also includes a material class (metal, dielectric, or null), a level, and a thickness. It corresponds naturally to a grouping of objects that can be added or removed by a single fabrication step, e.g. when depositing a film or etching pockets. Each source layer can be configured to appear or not appear in its eventual solid model or GDS destination (for the latter, an explicitGDSMetacan be passed). The "source" qualifier refers to the fact that these are the layers into which we'llplace!entities -- subsequent CAD kernel operations can create new ones, which we refer to as "generated" layers.SourceStack. The set of allSourceLayers in a model, keyed by their name, plus a map of level indices to z-coordinates.LayerRef. A simpleSymbolwrapper (and subtype ofDeviceLayout.Meta) used to signal whichSourceLayerwe're placing an entity into.Locator. A point-likeGeometryEntitythat can beplace!d to mark special objects for selection. The entity or set of entities selected is that which contains the point.LocatorMeta. An abstract type for all locator metadata. For now, its concrete types areTerminal,Ground,LumpedPort,WavePort, andTag. The type ofLocatorMetaused toplace!aLocatordetermines the type of selection that is created. In the case ofTerminalandGround, the full connected component of metallic entities is selected.Extrude,Cut,Fuse,Intersect,GetBoundary,GetInterface,Translate,Revolve,Hollow,Remove,RestrictTo, andSetPeriodic. These operate on layers (source or generated) and are lowered to PG-level operations by a simple compiler, which keeps track of layer dimensions, absorbs adjacentRemoveoperations ( simple peephole optimization to make the OCC kernel work more efficiently), and tracks operations that can only be executed after fragmentation (GetInterfaceandHollow). While simple, down the line I expect this compiler to prove useful for simplifying the interface to the OCC kernel and perform more optimizations.SolidModelTarget. ASourceStack+ a set of operations, providing the complete recipe for turning a schematic into a solid model. Fed directly to the renderer viarender!(solid_model, schematic, target). I greedily used the same name for the type that we use in the existing pipeline, as it's encapsulated in the module and not exported, but open to changing it.Under the hood, the thing that makes it possible to create and track selections while offering a simple layer-based interface is a deduplication pass that ensures that every entity in the model belongs to one and only one PG, as this is a critical requirement by Palace and the MFEM mesh data structures. The pass partitions the model into as many PGs as needed while keeping track of their layer memberships.
Changes to existing code
In addition to this new functionality, this PR makes a few minor (and non-breaking) modifications to existing code:
src/schematics/components/components.jl—geometry(comp)can now access a task-local context, providing a cache and an error-recovery callback so that rendering withstrict=:nocan continue past a failing component. Fixes #287.src/schematics/solidmodels.jl—render!now runs inside that context and atry/finallyaround the logfilesrc/schematics/schematics.jl—close_logfiletolerates an already-closed stream so cleanup doesn't mask an earlier failure.src/solidmodels/render.jl—_render_orchestrator!has a new keywordpreflattened=false; the newpipeline flattens the coordinate system once itself (after qualifying locator names by graph node) and passes the result through.
Project.toml— addsJSONandJSONSchemaas test-only dependencies (schema validation of thereturned metadata).
docs/make.jlregisters the module and a concept page.Testing
test/test_solidmodels_experimental.jl: compiler unit tests per operation (lowering,lifetime absorption, rejection cases), Gmsh-backed integration tests with measured
areas/volumes for each operation family including hollowing and interfaces,
locator/port/tag resolution, deduplication invariants (dims 1–3), registry/model
consistency, strictness modes, and a schematic end-to-end test that validates the metadata
against the schema.
Limitations / follow-ups
paths are not yet encoded.
ustrip(STP_UNIT, ::Float64)limitation inextrude_z!).Miscellaneous
AGENTS.mdwith a few style conventions. Feel free to reject it or modify it, although it'd probably be useful to have such a file for the project anyway, even if we end up using a different version.TODO
P.S. sorry for the messy commit history... I took a meandering path to get here. Maybe we just squash when we merge?