UseDesktop Research
DesktopBench: Benchmarking Agents on Economically Valuable Professional Software Work
UseDesktop
Paper, code, data, and public ranking artifacts will be attached as DesktopBench moves from evaluation preview to public release.
Abstract
DesktopBench is a benchmark for agents working in professional software workflows across commerce, accounting, support, sales, and office tools. It is particularly designed to test whether an agent can complete economically valuable work end-to-end.
Each workflow includes a resettable RL environment, tasks, and a programmatic verifier. This makes the benchmark useful for evaluation, reproducible rollout testing, and future training runs where success can be checked from the final state rather than only from a transcript.
Demo videos
Agent runs on the same workflow contract
These cards are reserved for public videos once trace bundles and scoring artifacts are ready.
Run video pending
Opus 4.8
Commercial frontier model
A full desktop workflow run with screen trace, action log, final state diff, and verifier result.
Run video pending
OpenAI 5.4 mini
Small frontier model
A lower-cost agent attempt on the same resettable workflow contract and scoring surface.
Run video pending
RLVR open-weight model
Trained open-weight run
A policy improved against DesktopBench-style rewards, evaluated with the same verifier.
Benchmark
A benchmark for real desktop work
Core tasks
The initial release target across commerce, accounting, support, and office workflows.
Core suites
StoreOps, AccountingOps, SupportOps, and OfficeOps form the first public target.
Total suites
The broader map expands into sales, legal, HR, and public portal workflows.
Task contract
Every workflow is packaged as a resettable RL environment, tasks, and a programmatic verifier.
Resettable environment
Every task starts from a known seed and can be replayed across model attempts.
Executable verifier
Tasks are scored by final state, artifact, and side-effect checks rather than screenshots alone.
Real desktop work
Suites are modeled after store, accounting, support, sales, office, legal, HR, and public portal operations.
Optional trajectory
Human demonstrations and failed attempts can be attached as training or audit evidence.
Results
Benchmark results
| Model | Track | Verified success | Cost | Evidence |
|---|---|---|---|---|
| Opus 4.8 | DesktopBench-Core | TBD | TBD | Awaiting public trace bundle |
| OpenAI 5.4 mini | DesktopBench-Core | TBD | TBD | Awaiting public trace bundle |
| RLVR open-weight model | DesktopBench-Core | TBD | TBD | Awaiting public trace bundle |
Task examples
Economically valuable workflows become executable tasks
StoreOps-50
StoreOps
Fulfill order #1048 using tracking number CJ123456789KR and carrier CJ Logistics.
order.status == fulfilled, carrier == CJ Logistics, tracking_number == CJ123456789KR
AccountingOps-50
AccountingOps
Match the KRW 1,200,000 bank transaction from Acme Korea to invoice INV-1028.
txn_8842.matched_invoice == INV-1028 and txn_8842.reconciled == true
SupportOps-50
SupportOps
Classify ticket #5831 as Billing and set priority to High.
ticket.category == Billing and ticket.priority == High
OfficeOps-50
OfficeOps
Find the latest email from billing@aws.amazon.com and save the attached invoice PDF to Invoices/AWS.
file exists in Invoices/AWS and source email/message id matches latest AWS billing email
Evaluation
Task contract
| task_id | Stable identifier such as StoreOps.OrderFulfill.001. |
|---|---|
| suite / family | Benchmark suite and task family for stratified splits. |
| difficulty | L1-L5 long-horizon difficulty level. |
| instruction | Natural language task objective given to the agent. |
| environment | Mock app or desktop workspace runtime. |
| initial_state_seed | Seed used to reset state before each rollout. |
| allowed_apps | Browser, files, spreadsheet, email, calendar, or multi-app desktop surface. |
| success_verifier | Executable final-state, artifact, or process checks. |
| side_effect_verifier | Checks that unrelated records/files were not changed. |
| limits | Max steps and max minutes for the rollout. |
| trajectory | Optional human demonstration or known-good trace. |
| reward | Success and partial-credit reward contract for RLVR. |
Citation
Cite DesktopBench
@misc{chae2026desktopbench,
title={DesktopBench: Benchmarking Agents on Economically Valuable Professional Software Work},
author={Chae, Seungju},
year={2026},
note={Technical report in progress}
}