Split LiteCoder terminal system prompts from tasks

[neubig]

Closes #283.
Refs #296.

Summary

• keep raw_to_atif faithful to the source terminal transcript
• move terminal prompt splitting, empty-turn cleanup, and completion JSON conversion into atif_to_std.py using shared terminal helpers
• convert terminal completion JSON with task_complete=true and no commands into structured finish tool calls only when the message is non-empty
• regenerate sample_atif.json, sample_std.json, and existing sample_sft outputs

Design decisions

• Split marker: use the shared terminal Task Description marker helper so LiteCoder follows the same behavior as other terminal datasets.
• Conversion stage: perform prompt splitting and completion structuring in atif_to_std.py so raw_to_atif.py remains source-faithful.
• Empty completion messages: skip finish-call structuring when analysis/plan are empty so the converter does not create semantically empty finish calls.
• Shared helper reuse: keep terminal prompt splitting and terminal JSON completion logic in scripts/atif_to_std_common.py to avoid dataset-local duplication.

Validation

• regenerated sample_atif.json and sample_std.json from sample_raw.json
• regenerated sample_sft/openhands_v0.json and sample_sft/openhands_sdk.json from sample_std.json
• confirmed sample_atif.json preserves the raw combined terminal prompt
• confirmed sample_std.json starts with system instructions followed by the user task
• confirmed finish tool calls include non-empty message arguments
• uv run --no-project --python /usr/bin/python3.12 --with-requirements requirements.txt python -m pytest tests/test_standardized_schemas.py tests/test_std_to_sft_conversion.py -k "litecoder_terminal_sft or openthoughts_agent_sft" -q

Known limitations

• Terminal completion JSON with task_complete=true but empty analysis/plan is left as the original message rather than forced into a finish call.
• The prompt split intentionally applies only to the initial combined terminal prompt pattern handled by the shared helper.

[github-actions]

🟡 Acceptable — The split logic and empty-turn filtering in raw_to_atif.py are sound and the updated sample_atif.json/sample_std.json look correct. However, the two SFT sample files were not regenerated after sample_std.json changed, leaving them out of sync with the rest of the pipeline.

Stale SFT files (blocking):

Regenerating via the standard pipeline produces files that differ substantially from what is committed — 500-line diff for openhands_v0.json and 1579-line diff for openhands_sdk.json. The committed SFT files still carry the old single combined user message (system instructions + task), while the updated sample_std.json now correctly separates them. Per AGENTS.md: "Keep the same records and order across sample_raw.json, sample_atif.json, sample_std.json, and each sample_sft/<agent_name>.json; the samples should represent the same tasks at each stage."

Please regenerate both SFT files using the commands from AGENTS.md (cat sample_std.json | ... | python agents/openhands_v0/std_to_sft.py ...).

PR description gap (non-blocking): A design-decision entry explaining the choice of TASK_DESCRIPTION_MARKER and the empty-turn filtering rationale would align with the AGENTS.md design-decision catalog requirement.

---

Was this automated review useful? React with 👍 or 👎 to this review to help us measure review quality.
Workflow run: https://github.com/neulab/agent-data-protocol/actions/runs/28747969931

[github-actions]

🟡 Acceptable — The primary blocking issue from the previous review (stale SFT files) is resolved: both openhands_v0.json and openhands_sdk.json have been regenerated and their trajectory IDs align with sample_std.json. The old unresolved thread about a redundant renumber_steps call in raw_to_atif.py is now moot — that file was replaced entirely with a clean main(__file__) delegation. Schema validation passes for all three sample stages.

One latent data quality risk noted below.

This review was generated by an AI agent (OpenHands) on behalf of the repository maintainers.

---

Was this automated review useful? React with 👍 or 👎 to this review to help us measure review quality.
Workflow run: https://github.com/neulab/agent-data-protocol/actions/runs/28751013595

[github-actions]

🟡 Acceptable overall — the pipeline split (keeping raw_to_atif faithful, moving normalization to atif_to_std) is correct, and the sample artifacts look good: IDs align across all four stages, trailing newlines are present, and finish-call messages are non-empty. However, the new atif_to_std.py duplicates two functions that already exist in scripts/atif_to_std_common.py, and the bug that motivated the duplication should instead be fixed in the shared module.

This review was generated by an AI agent (OpenHands) on behalf of the user.

---

Was this automated review useful? React with 👍 or 👎 to this review to help us measure review quality.
Workflow run: https://github.com/neulab/agent-data-protocol/actions/runs/28754028365

[github-actions]

🟡 Acceptable — the code changes are correct and the primary pipeline stages are reproducible. All previous blocking issues (code duplication, stale SFT files, empty-message guard) are resolved.

sample_sft/openhands_sdk.json has pre-existing converter drift — regenerating via agents/openhands_sdk/std_to_sft.py produces output that differs from the committed file in two places: (1) system prompt content (converter pulls from the live codebase which changes over time), and (2) a cache_control field the current converter emits but the committed sample lacks. This drift is not caused by this PR's changes — sample_std.json and sample_sft/openhands_v0.json are both byte-for-byte reproducible. ✅

The openhands_sdk.json should be regenerated before merge, or the openhands_sdk converter's system-prompt templating and cache_control behavior should be stabilized so samples stay reproducible across converter updates.

This review was generated by an AI agent (OpenHands) on behalf of the reviewer.

---

Was this automated review useful? React with 👍 or 👎 to this review to help us measure review quality.
Workflow run: https://github.com/neulab/agent-data-protocol/actions/runs/28754212621

[github-actions]

🟡 Acceptable — all previous blocking issues are resolved and the new implementation is correct. Verified: IDs align across all four stages, trailing newlines are present, schema version is ATIF-v1.7, from fields are correct, sample_atif.json preserves the combined terminal prompt, and sample_std.json correctly produces a system step followed by a user Task Description step. Finish-call messages are non-empty where present.

One minor observation noted inline.

Review generated by an AI agent (OpenHands) on behalf of the user.

---

Was this automated review useful? React with 👍 or 👎 to this review to help us measure review quality.
Workflow run: https://github.com/neulab/agent-data-protocol/actions/runs/28754444136

[github-actions]

🟡 Acceptable — the implementation is correct and all previously blocking issues are resolved. The pipeline is fully reproducible: sample_std.json regenerates cleanly from sample_atif.json, and sample_sft/openhands_v0.json regenerates cleanly from sample_std.json. IDs align across all four stages, trailing newlines are present, schema version is ATIF-v1.7, step sources are valid, and finish-call messages are non-empty.

One minor observation noted inline.

This review was generated by an AI agent (OpenHands) on behalf of the repository maintainers.

---

Was this automated review useful? React with 👍 or 👎 to this review to help us measure review quality.
Workflow run: https://github.com/neulab/agent-data-protocol/actions/runs/28754640015

[github-actions]

🟡 Acceptable — all previous blocking issues are resolved. IDs align across all four pipeline stages (sample_atif.json, sample_std.json, openhands_v0.json, openhands_sdk.json), trailing newlines are present, schema version is ATIF-v1.7, and the prompt split is correctly staged in atif_to_std.py. One minor duplication noted inline.

[RISK ASSESSMENT]

• [Overall PR] ⚠️ Risk Assessment: 🟢 LOW — isolated normalization refactor with regenerated sample artifacts; no schema changes, no shared API modifications, and no new dependencies.

This review was generated by an AI agent (OpenHands) on behalf of the reviewer.

---

Was this automated review useful? React with 👍 or 👎 to this review to help us measure review quality.
Workflow run: https://github.com/neulab/agent-data-protocol/actions/runs/28754844958

[github-actions]

🟡 Acceptable — all prior blocking issues are resolved and the refactor is correct. IDs align across all four pipeline stages, trailing newlines are present, schema version is ATIF-v1.7, and sample_atif.json correctly preserves the combined terminal prompt while sample_std.json correctly produces a system step followed by a user Task Description step.

One minor ordering issue flagged inline; the nearly-identical atif_to_std.py files are a pre-existing structural concern worth noting for a future follow-up (e.g., adding a main_terminal export to the shared module so both datasets can delegate with a single import).

[RISK ASSESSMENT]

• [Overall PR] ⚠️ Risk Assessment: 🟢 LOW — isolated normalization refactor; no schema changes, no new dependencies, and the only behavioural change for litecoder is applying normalize_terminal_trajectory instead of the generic standardize_tools-only path, which is the intended fix.

VERDICT:
✅ Worth merging — correct implementation, all previous blockers addressed.

This review was generated by an AI agent (OpenHands) on behalf of the repository maintainer.

---

Was this automated review useful? React with 👍 or 👎 to this review to help us measure review quality.
Workflow run: https://github.com/neulab/agent-data-protocol/actions/runs/28754997904

[github-actions]

🟡 Acceptable — the implementation is correct and all previously blocking issues are resolved. normalize_terminal_trajectory is cleanly extracted into the shared module; the empty-message guard lives in the right place; IDs align across all four pipeline stages; trailing newlines are present; schema version is ATIF-v1.7; and nemotron sample files are unchanged, confirming output parity with the old normalize_nemotron.

One new observation noted inline.

This review was generated by an AI agent (OpenHands) on behalf of the repository maintainers.

---

Was this automated review useful? React with 👍 or 👎 to this review to help us measure review quality.
Workflow run: https://github.com/neulab/agent-data-protocol/actions/runs/28755131806

[github-actions]

🟢 Good taste — the refactor is correct and complete. All previously blocking issues are resolved.

Verified:

• IDs align across all four pipeline stages (sample_atif.json → sample_std.json → openhands_v0.json → openhands_sdk.json)
• sample_std.json opens with system → user Task Description → agent, confirming the prompt split is applied in atif_to_std
• sample_atif.json preserves the combined terminal prompt (split not applied at raw stage)
• Schema version is ATIF-v1.7; trailing newlines present on all JSON files
• Function ordering in atif_to_std_common.py is correct: all callees precede normalize_terminal_trajectory
• is_empty_message_only_step and normalize_terminal_trajectory live in the shared module; both LiteCoder and Nemotron are down to 6-line delegates
• Focused tests: 3 passed for litecoder, 3 for openthoughts, 3 for nemotron

[RISK ASSESSMENT]

• [Overall PR] ⚠️ Risk Assessment: 🟢 LOW — isolated normalization refactor with regenerated sample artifacts; no schema changes, no new dependencies, parity with the pre-refactor Nemotron output confirmed by passing tests.

VERDICT:
✅ Worth merging

---

Was this automated review useful? React with 👍 or 👎 to this review to help us measure review quality.
Workflow run: https://github.com/neulab/agent-data-protocol/actions/runs/28755286590

This review was generated by an AI agent (OpenHands) on behalf of the user.