Related work
The Gap QuoteBench Targets
Existing terminal and command-generation benchmarks exercise shell quoting incidentally; QuoteBench isolates it as the measured skill.
Direct Competitors
None found. Terminal benchmarks and NL-to-command datasets either do not isolate quoting, or use metrics that erase the argument values where quoting failures occur.
Near Misses
| Work | Relation |
|---|---|
| Terminal-Bench / TB2.0 | State-based terminal benchmark; quoting appears incidentally, not as the isolated target. |
| NL2SH-ALFA | Closest one-shot command synthesis methodology; motivates machine-constructed, execution-verified oracles. |
| InterCode-Bash | Useful verification precedent and gold-data cautionary tale. |
| NLC2CMD / NL2Bash | Template metrics can erase argument-level quoting mistakes. |
| tau-bench | Motivates reliability-style reporting for tail-event failures. |
Why the Failure Mode Is Real
- Public issue trackers for coding agents repeatedly document heredoc, apostrophe, nested-shell, and cross-shell wrapping failures.
- Agent frameworks often steer users away from shell editing paths and toward structured editor tools, which avoids rather than measures the skill.
- QuoteBench removes common escape hatches so quoting itself becomes visible in the score.
Design Sources
Task hazards are grounded in ShellCheck rule families, Wheeler's hostile filename taxonomy, and real shell edge cases such as sed replacement escaping, heredoc delimiter collision, and nested ssh-style evaluation.