feat: support optional flag in from_ extractors#1552
Open
mgdemers wants to merge 1 commit intoapache:mainfrom
Open
feat: support optional flag in from_ extractors#1552mgdemers wants to merge 1 commit intoapache:mainfrom
mgdemers wants to merge 1 commit intoapache:mainfrom
Conversation
17d214d to
df408f2
Compare
skrawcz
reviewed
Apr 19, 2026
|
|
||
| :param target: Parameter, into which we're loading the data | ||
| :param loaders: A list of data loaders that are viable candidates, given the key after `from_` | ||
| :param optional: Whether missing targets should raise exceptions or skip being loaded. |
Contributor
There was a problem hiding this comment.
I would add a bit more here.
Suggested change
| :param optional: Whether missing targets should raise exceptions or skip being loaded. | |
| :param optional: Whether missing targets should raise exceptions or skip being loaded. Optional=True will mean that if the target does not exist in your graph, we skip try to inject this into the graph. |
Author
There was a problem hiding this comment.
Added - I also added a note on the materialization docs about specifying optional data loaders.
skrawcz
reviewed
Apr 19, 2026
Contributor
skrawcz
left a comment
There was a problem hiding this comment.
looks good. can you update the materialization docs too? A comment or example snippet should suffice.
df408f2 to
f8c0a51
Compare
* Add optional flag to ExtractorFactory to skip data loader when target was not found rather than raise exceptions. * Add unit tests for optional and non-optional extractors. * Add more detailed comments and documentation note.
f8c0a51 to
00d444d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Support optional
from_extractors when DAG modules' targets do not reference an extractor.Changes
optionalflag toExtractorFactorygenerate_nodes, return an empty list instead of raisingValueErrorwhen flag is set.How I tested this
Notes
Discussion in #1551
Checklist