BENCHMARK-TARGETED TRAINING

Original engineering tasks for frontier coding agents.

Overview Production
LONG-HORIZON ENGINEERINGFIVE LANGUAGESBEHAVIOR-LEVEL VERIFICATION

01 / DATASET OVERVIEW

EXPERT-LEVEL TASK DATA

SCOPE & PURPOSE

The training task.

  • LONG-HORIZON ENGINEERING
  • FIVE LANGUAGES
  • BEHAVIOR-LEVEL VERIFICATION

Designed for long-horizon software engineering task solving by frontier coding agents, DeepSWE provides original, contamination-free engineering task data. The dataset covers five programming languages, with tasks created from scratch and independently verified based on software behavior, enabling evaluation of real-world engineering capabilities rather than memorization. It can be used for coding agent training, capability evaluation, and failure-mode research.

02 / DATASET PROFILE & TECHNICAL SIGNALS

SCOPE · SCALE · CONSTRUCTION
01LANGUAGE COVERAGE
5 languages
Cross-language software engineering tasks.
02TASK ORIGIN
From scratch
Original, contamination-free engineering tasks.
03DATA SOURCES
Dual source
Open-source and authorized private repositories.
04VERIFICATION
Independent
Software behavior and patch-level verification.

03 / DATA PRODUCTION & CONSTRUCTION

METHOD & WORKFLOW

Data production strengths

We have dual-source data acquisition capabilities covering open-source repositories and authorized private repositories, together with expertise in designing original long-horizon tasks across languages and repositories. Our capabilities cover sandboxed trajectory collection, behavior-level test development, and patch-level independent verification, supporting the full workflow from task creation and execution to quality control and scoring.

  1. 01SOURCE

    Dual-source acquisition

    Acquire data from open-source repositories and authorized private repositories.

  2. 02DESIGN

    Original long-horizon tasks

    Design original tasks across programming languages and repositories.

  3. 03EXECUTE

    Sandboxed trajectories

    Collect execution trajectories in sandboxed environments.

  4. 04VERIFY

    Behavior & patch verification

    Develop behavior-level tests and independently verify patches for quality control and scoring.

04 / COVERAGE & CAPABILITY MATRIX

TASKS · METHODS · APPLICATIONS
DIMENSIONCOVERAGE
01Engineering scope
Long-horizon tasksFive programming languagesCross-repository design
02Repository sources
Open sourceAuthorized private repositories
03Verification layers
Software behaviorBehavior-level testsIndependent patch verification
04Research & training
Coding agent trainingCapability evaluationFailure-mode research
Explore the production method

NEXT STEP