Improve PREPARED_INGREDIENT accuracy for irreversible-prep verbs at volumetric amounts

[MaximusCub]

Summary

Fixes PREPARED_INGREDIENT returning the wrong value for sentences where an irreversible action verb (chopped, sliced, diced, etc.) trails a comma at a volumetric amount.

>>> parse_ingredient("1 cup carrots, diced").amount[0].PREPARED_INGREDIENT
False  # on develop today

>>> parse_ingredient("1 cup diced carrots").amount[0].PREPARED_INGREDIENT
True

Both sentences describe the same thing -- 1 cup of diced carrots -- but the flag flips on word order alone. That behavior is correct for genuinely two-way verbs like sifted (where 1 cup flour, sifted and 1 cup sifted flour are physically different quantities), but produces the wrong answer for verbs where pre-prep volumetric measurement is physically impossible.

Root cause

PREPARED_INGREDIENT is set by syntactic position in ParsedIngredient.set_prepared_flag: prep-between-amount-and-name -> True, prep-after-name -> False, plus the to yield/to make heuristic. The flour/sifted contrast that motivates this rule is genuinely two-way, but it's a corner case in real recipes. For most prep verbs (chopped, sliced, diced, minced, grated, shredded, etc.) you cannot cup-measure the unprepped form -- there is no way to put unchopped onion or unshredded cheese into a cup measure -- so the post-comma form must measure the prepared form regardless of word order.

Approach

A postprocessor fix, no CRF retrain. The CRF correctly tags the verbs as PREP; only the post-CRF flag interpretation needs to change.

• A closed list of 24 irreversible action verbs (IRREVERSIBLE_PREP_VERBS)
• Fires only when the amount unit is volumetric AND not strictly liquid-only -- LIQUID_ONLY_UNIT_NAMES = {centiliter, deciliter, fluid_ounce, liter, milliliter} excluded because pre-prep liquid measurement is physically possible and volume is preserved through prep
• Only flips False to True; never overrides an existing True (positional or yield/make)
• Reversible-order verbs (sifted, packed) and state adjectives (frozen, dried) remain excluded

Changes

• ingredient_parser/en/_constants.py -- IRREVERSIBLE_PREP_VERBS (24) and LIQUID_ONLY_UNIT_NAMES (5)
• ingredient_parser/en/_utils.py -- public predicates is_volumetric_unit(unit) and is_liquid_only_unit(unit) delegated to pint dimensionality, plus a shared _coerce_unit_to_pint helper
• ingredient_parser/en/postprocess.py -- _has_irreversible_prep_verb() method, override sites in _fallback_pattern and _composite_amounts_pattern. Runs before _distribute_related_flags so paired metric/imperial amounts (1 cup (140 g) onion, finely chopped) propagate correctly.
• tests/parser/test_prepared_ingredient.py -- 9 new tests covering canonical positive, paired propagation, count/mass/strict-liquid gates, hyphenated PREP token, composite path, and a negative case (1 cup rice, cooked stays False) pinning the exclusion of adjacent verbs
• docs/source/explanation/postprocessing.rst -- replaces the existing 1 tbsp chopped nuts / 1 tbsp nuts, chopped worked example with prose covering both the syntactic rule and the verb-class override
• docs/source/tutorials/examples.rst -- adds a paragraph after the flour/sifted archetype contrasting volumetric/count/mass behavior under the override
• CHANGELOG.md -- entry under ### develop

7 files, 339 insertions / 2 deletions.

Validation

Full test suite green (466 passed). The 5 existing prepared-ingredient tests pass unchanged.

Corpus measurement: parsed every sentence in train/data/training.sqlite3 matching the SQL pattern below across all 24 verbs (1086 unique sentences after dedup; 1130 raw matches), once with the override and once with the override reverted to origin/develop:

SELECT sentence FROM en
WHERE LOWER(sentence) GLOB '*[0-9]* {unit}*, {verb}*'
   OR LOWER(sentence) GLOB '*[0-9]* {unit}*, {verb}'
-- where {unit} in {cup, tablespoon, tbsp, tbs, teaspoon, tsp, pint, quart, gallon}
-- and {verb} in IRREVERSIBLE_PREP_VERBS

	PREPARED_INGREDIENT=True count
origin/develop (no override)	192
this PR (with override)	1042
flipped False -> True by the override	850

10 random rows sampled from the 850 flipped sentences:

1 tablespoon sliced pimientos, chopped
1/2 cup artichoke hearts, drained and chopped (optional)
1/3 cup onions, chopped
1/2 cup oil-packed sun-dried tomatoes, drained and diced
2 tablespoons drained bottled capers, chopped
1 tablespoon thyme leaves, chopped
2 cups canned Italian plum tomatoes, with their juice, chopped
1 cup fresh basil leaves, chopped
1/2 cup porcini mushrooms, chopped (if dried, 1/4 cup)
1/2 cup sweet pickles, diced with some juice from the jar

In most sampled cases the natural reading appears to be form-at-measurement (1/3 cup of chopped onions, 1 cup of chopped fresh basil leaves, etc.). The hypothetical "measure first, then prep as a workflow step" reading isn't expressed in this syntactic shape -- when writers want that they use a different form entirely.

A subset is genuinely borderline: herbs in particular (1 cup fresh basil leaves, chopped, 1 tablespoon thyme leaves, chopped) can reasonably mean either "measure whole leaves, then chop" or "chop, then measure" -- whole leaves are themselves cup-measurable, unlike whole onions or unshredded cheese, so the workflow reading isn't physically excluded the way it is for the canonical cases. This is accepted as a minor cost. For a use case that may commonly be served by the flag -- volumetric-to-mass density conversion for nutrition / calorie mapping -- a misclassification on a tablespoon of herbs has small mass and calorie consequences relative to correctly resolving the bulk-ingredient cases (chopped onions, diced carrots, shredded cheese, halved cherry tomatoes, etc.) where the unprepped and prepped forms differ in mass by 25-100%.

Why the verb list is shaped this way

Pre-comma vs post-comma frequency at volumetric units across the 24 verbs (same SQL pattern, swapping '*[0-9]* {unit}*, {verb}*' for '*[0-9]* {unit}* {verb} *' for the pre-comma count):

verb	pre-comma	post-comma
chopped	3,011	290
ground	1,151	32
grated	827	21
minced	780	79
sliced	451	61
shredded	364	34
diced	329	62
crushed	245	88
drained	161	179
pitted	120	45
crumbled	77	32
halved	33	32
seeded	57	42
cored	29	35
(10 others, smaller)

For pre-comma-dominant verbs (chopped, ground, grated, minced), writers prefer the explicit cup VERB X syntax when meaning form-at-measurement; the post-comma form is rarer but means the same thing. For near-equal pre/post ratios (halved, quartered, cored, drained, seeded), writers use both forms interchangeably.

Deliberately excluded from the verb list: state adjectives (frozen, dried, chilled, refrigerated), reversible-order preps (sifted, packed, melted, softened), and time-process verbs (cooked, boiled, roasted, toasted) where the syntactic rule remains correct or where corpus inspection shows the post-comma form is genuinely ambiguous.

Notes

A CRF-shaped solution is possible -- splitting PREP into sub-labels for irreversible vs reversible-order verbs in the corpus, then keying the flag off the new label -- but it felt like meaningful added complexity (re-labelling and retraining for a property that's a flat per-verb attribute) for little practical gain over the postprocessor approach. The postprocessor route seemed the most elegant fit: the CRF is already correctly identifying these tokens as PREP, the bug is purely in how that signal is interpreted into the flag, and a closed verb-class set captures it directly. Open to redoing as a CRF change if you prefer; the verb list and corpus survey above would feed the labelling pass.

[strangetom]

Hi @MaximusCub

I need to have a think about this one because I'm not sure I agree.

As I understand it, the problem you are proposing to fix is as follows:

• There are ingredients that you cannot reasonably measure by volume
• Recipe ingredient sentences can refer to such ingredient by volume
• If such a sentence includes preparation instructions (specifically, irreversibly preparation), then the intent of the sentence is always that the amount refers to the prepared ingredient, even though the structure of the sentence would imply that the amount refers to the ingredient before preparation.

The thing that makes me hesitant about this is I can think of counter examples where this isn't true. In the 1 cup carrots, diced example, I agree that the 1 cup likely refers to the diced carrots. However there are other examples where it's ambiguous:

• 2 tablespoons drained bottled capers, chopped
  Capers are small enough that measuring them in tablespoons is not unreasonable.
• 2 cups canned Italian plum tomatoes, with their juice, chopped
  In a can of tomatoes, the tomatoes are typically very soft and with the juice could reasonably be measured by volume.
• 1 cup fresh basil leaves, chopped
  As you mention, measuring herbs by volume is possible.
• 1/4 teaspoon cloves, ground
  A tsp of ground cloves is a very different amount than a tsp of cloves that are then ground.
• 1/4 cup garlic cloves, minced
  Similarly here, a minced garlic clove occupies a smaller volume than a whole one.

[MaximusCub]

Hey @strangetom -- fair point on those examples. Spent some time digging in and I'm less confident this is "the best" approach than I started. It's an improvement, but like the existing code it carries its own trade-offs.

A few high-level thoughts before the verb list. The fundamental issue is that whether the amount refers to the prepared form depends on (verb, ingredient, unit_size) jointly, and sometimes there's no canonical answer even with all three -- the "right" answer often requires context the parser doesn't have. Honestly, a confidence/probability score is probably the better long-term shape than a bool, but that's an interface change and would need either CRF modifications or a separate classifier feeding it. The classifier route adds architectural complexity and inference latency. The CRF route avoids the latency but needs relabeling effort. Both are bigger than this PR's scope. A middle-ground I considered is an ingredient-class lookup table -- could capture the missing ingredient dimension without ML.

I'd be happy to look into implementation approaches for any of these if you have the appetite for it.

If you'd still like to ship something low-tech in the meantime, here's a tighter verb list. One framing note first: "non-reversible" was a poor naming choice on my part -- it's not that the action is non-reversible, it's that the heuristic odds (not certainty) favor prep-before-measurement for certain verbs. LIKELY_PREPARED_BEFORE_MEASUREMENT_VERBS or similar would be clearer. Happy to rename.

Drop:

• ground -- you're right, I should have caught this. Mostly applies to seeds/peppercorns/spices where measure-then-grind is the natural reading.
• drained -- safer left out. For solid items the liquid flows around so flag doesn't matter; for soft things in juice it can go either way. The (tomato) juice isn't worth the squeeze ;)
• minced -- 55% herbs / 45% alliums/ginger/peppers/zest. The non-herb cases are clean prep-first but cup-level herbs lean workflow and tbsp/tsp herbs are genuinely mixed. No clean heuristic without more complexity. Leave it as-is.
• crushed -- ~47% whole spices and ~65% workflow overall. Same shape as ground.
• cubed -- ~72% butter in the post-comma volumetric corpus; butter is measure-first (by stick equivalence) so override should not fire on those. Only ~28% (cheese, firm vegetables like potatoes / carrots) is prep-first. Wrong-fires dominate; safer to leave out.
• halved, pitted, quartered -- doesn't confidently infer prep-first; mostly small cup-measurable items.
• deveined -- almost no override-relevant volumetric cases in the corpus (the few that appear are pre-comma forms already handled by the positional rule). Effectively dead code; leave it.
• pureed -- ~1 corpus row at a volumetric unit. Effectively dead code; leave it.

Keep:

• chopped -- bulk vegetables (~25%: onions, peppers, celery, carrots) drive the win with substantial volume-change impact. Herbs, nuts, capers, dried fruit are technically mixed but their volume change through chopping is smaller in comparison so wrong-flag has low practical impact.
• sliced -- ~50 post-comma volumetric rows. Firm whole vegetables (~25%: onions, cabbage, asparagus, zucchini, rhubarb) and irregular cup-awkward items (~30%: mushrooms, strawberries, peaches) are prep-first. Butter (~10%) is measure-first (by stick equivalence) so override should not fire on these. Olives / dates / cup-measurable small items (~15%) can go either way. Net is roughly 55% prep-first, 10% override-wrong, 35% mixed.
• slivered -- the dominant "slivered almonds" form is pre-comma and already handled by the positional rule. The 7 post-comma volumetric cases are mostly onions/alliums where prep-first is correct.
• diced -- bulk vegetables and alliums (~46%) plus cooked proteins (~9%, always prepped then diced) drive the win. Butter (~26%) is measure-first (by stick equivalence) so override should not fire on those. Net ~55% prep-first, 26% override-wrong, rest mixed.
• grated -- small corpus (~5 rows): cheese, ginger, citrus zest. Whole forms not cup-measurable.
• shredded -- cheese dominant (~47%, blocks not cup-measurable); cooked chicken (~22%) is prep-first. Cabbage / lettuce / coconut round it out.
• julienned -- firm cup-awkward produce (carrots, melons, turnip).
• seeded -- tomatoes dominant, plus peppers, cucumbers, other firm produce. Whole forms not cup-measurable.
• cored -- apples dominant (~50%), peppers and tomatoes minor. Whole forms not cup-measurable.
• mashed -- potatoes, bananas -- mash then measure.
• crumbled -- dried herbs (~50%) are technically workflow but volume mostly conserved through crumbling so wrong-flag has near-zero practical impact. Cheeses (~20%, feta/Gorgonzola/goat) have substantial volume change where override correctly fires.
• snipped -- leafy/grass-like herbs (chives, basil, rosemary) at tbsp-style volumes. Mostly doesn't fit prior to prep.

Let me know if you'd want to move ahead with something like this, or if a different approach (CRF, classifier, ingredient table) would be a better use of effort. Happy to adjust the naming/doc conventions either way.

Thanks again for the thoughtful review.

[strangetom]

A few high-level thoughts before the verb list. The fundamental issue is that whether the amount refers to the prepared form depends on (verb, ingredient, unit_size) jointly, and sometimes there's no canonical answer even with all three -- the "right" answer often requires context the parser doesn't have. Honestly, a confidence/probability score is probably the better long-term shape than a bool, but that's an interface change and would need either CRF modifications or a separate classifier feeding it. The classifier route adds architectural complexity and inference latency. The CRF route avoids the latency but needs relabeling effort. Both are bigger than this PR's scope.

Modification of the CRF sounds like it could be worth investigating but, as you say, it would be a lot of effort. I'm imagining changes the current label set to split the PREP label into two variants: one for where for the prep happens after measuring the amount, one where it happens before. In theory, training the model like should result in it learning that certain verbs and ingredient names and combinations thereof imply the prep happens before/after measurement. One issue with introducing new labels is that it tends to reduce model accuracy, but if that happens it might be offset by including new features based on your verb list.

You are welcome to investigate this further, and if you do I would suggest doing so on a subset of the training data to avoid spending ages relabelling the data only to find it doesn't work well enough. I'll add this to my backlog of things to investigate but I doubt I'll get to it any time soon.

A middle-ground I considered is an ingredient-class lookup table -- could capture the missing ingredient dimension without ML.

I can see the challenge with this being generating a sufficiently large table that covers the majority of cases.