Header Ads

Show HN: Terminal-Wrench, a dataset of 331 realistic hackable environments https://ift.tt/BSwxE2v

Show HN: Terminal-Wrench, a dataset of 331 realistic hackable environments I want to share a new dataset of 331 reward-hackable environments. These are real environments used in Terminal Bench and adjacent benchmarks. I first got interested in this because, as a reviewer of Terminal Bench, I noticed a lot of our tasks were hackable. I also noticed that many contributors to the benchmark do so because it provides credibility when selling environments to labs. Hence, TBench tasks are, in my opinion, held to a higher quality standard than those being used today for RL. No one is spending hours manually reviewing the $1B in tasks being purchased by major labs. As far as I understand, while everyone knows environments are hackable, nobody has released hundreds of "realistic" environments. https://ift.tt/THNJGMF April 15, 2026 at 01:42AM

No comments

Powered by Blogger.