Agent evals¶

Can an AI coding agent, given this linter, actually fix a failing diagram? These results answer that across agent harnesses and model tiers, for the rule fixtures listed below that --fix cannot clear on its own.

Which fixtures are evaluated¶

Each case seeds one file from tests/fixtures/rules/, copied into skills/mermaidjs-lint/evals/fixtures/ without its expect comment.

Case	Fixture
`border_contrast`	`contrast-border-reported.md`
`erd_contrast`	`contrast-text-unfixable-erd.md`
`label_colour`	`contrast-directive-no-color.md`
`large_chain`	`complexity-large-chain.md`
`nesting`	`complexity-nesting.md`
`parse_repair`	`parse-unparseable.md`

Other fixtures that --fix cannot clear are not evaluated. Those are contrast-edge-label.md, and the larger scenario files under complexity/, contrast/ and skill-evals/.

What a cell proves¶

Each cell runs one agent CLI, headless, on one failing fixture, with the mermaidjs-lint agent skill. The task is a single sentence: fix every finding the linter reports, editing the file in place.

A cell passes only when all of these hold:

The session was real. The harness ran, and its own session log was captured and priced.
The right file changed, and nothing else appeared. Linter caches are the only allowed additions.
The linter reports nothing. This package's own CLI runs on the agent's file and finds zero findings.
Nothing was deleted to get there. Every diagram and every node of the original is still present, so removing a diagram never passes.
The structure matches the committed LLM fix. Each fixture's .llm.md sibling is the golden, and only structure is compared, because the colours and split points are the agent's to choose.

The grader is deterministic: the verdict comes from the linter and the structure checks, never from another model.

Results¶

Run on 2026-10-17 with the tier 1, 2 and 3 models of each harness, the project matrix in pyproject.toml. The run used the skill as of commit 9df8f56.

Case	`claude/claude-haiku-5-5`	`claude/claude-sonnet-5-5`	`claude/claude-opus-5-5`	`codex/gpt-6-luna`	`codex/gpt-5.6-terra`	`codex/gpt-6.1-sol`
`border_contrast`	pass, 3 turns	pass, 3 turns	pass, 3 turns	pass, 4 turns	pass, 5 turns	pass, 4 turns
`erd_contrast`	pass, 4 turns	pass, 3 turns	pass, 3 turns	pass, 5 turns	pass, 6 turns	pass, 5 turns
`label_colour`	pass, 4 turns	pass, 3 turns	pass, 3 turns	pass, 7 turns	pass, 6 turns	pass, 5 turns
`large_chain`	pass, 5 turns	pass, 3 turns	pass, 3 turns	pass, 7 turns	pass, 13 turns	pass, 6 turns
`nesting`	pass, 3 turns	pass, 3 turns	pass, 3 turns	pass, 5 turns	pass, 5 turns	pass, 4 turns
`parse_repair`	pass, 4 turns	pass, 3 turns	pass, 3 turns	pass, 5 turns	pass, 4 turns	pass, 4 turns

36 of 36 cells passed. The whole run cost an estimated $1.73.

What each agent wrote¶

Each case shows its original fixture, then the file each harness and model left behind, copied from its kept workspace. A passing output must lint clean, and the site build fails if one does not.

`border_contrast`¶

The original fixture is contrast-border-reported.md. It reports contrast-border-below-ui-minimum.

flowchart LR A --> B classDef c fill:#ffffff,color:#000000,stroke:#fefefe class A c

Original source

# Border below 3 to 1

The label passes, the stroke does not. There is no fixer, because the stroke is a design choice.

```mermaid
flowchart LR
  A --> B
  classDef c fill:#ffffff,color:#000000,stroke:#fefefe
  class A c
```

`claude/claude-haiku-5-5`¶

pass 3 turns, an estimated $0.004

flowchart LR A --> B classDef c fill:#ffffff,color:#000000,stroke:#767676 class A c

Output source

# Border below 3 to 1

The label passes, the stroke does not. There is no fixer, because the stroke is a design choice.

```mermaid
flowchart LR
  A --> B
  classDef c fill:#ffffff,color:#000000,stroke:#767676
  class A c
```

`claude/claude-sonnet-5-5`¶

pass 3 turns, an estimated $0.062

flowchart LR A --> B classDef c fill:#ffffff,color:#000000,stroke:#767676 class A c

Output source

# Border below 3 to 1

The label passes, the stroke does not. There is no fixer, because the stroke is a design choice.

```mermaid
flowchart LR
  A --> B
  classDef c fill:#ffffff,color:#000000,stroke:#767676
  class A c
```

`claude/claude-opus-5-5`¶

pass 3 turns, an estimated $0.117

flowchart LR A --> B classDef c fill:#ffffff,color:#000000,stroke:#595959 class A c

Output source

# Border below 3 to 1

The label passes, the stroke does not. There is no fixer, because the stroke is a design choice.

```mermaid
flowchart LR
  A --> B
  classDef c fill:#ffffff,color:#000000,stroke:#595959
  class A c
```

`codex/gpt-6-luna`¶

pass 4 turns, an estimated $0.002

flowchart LR A --> B classDef c fill:#ffffff,color:#000000,stroke:#767676 class A c

Output source

# Border below 3 to 1

The label passes, the stroke does not. There is no fixer, because the stroke is a design choice.

```mermaid
flowchart LR
  A --> B
  classDef c fill:#ffffff,color:#000000,stroke:#767676
  class A c
```

`codex/gpt-5.6-terra`¶

pass 5 turns, an estimated $0.050

flowchart LR A --> B classDef c fill:#ffffff,color:#000000,stroke:#767676 class A c

Output source

# Border below 3 to 1

The label passes, the stroke does not. There is no fixer, because the stroke is a design choice.

```mermaid
flowchart LR
  A --> B
  classDef c fill:#ffffff,color:#000000,stroke:#767676
  class A c
```

`codex/gpt-6.1-sol`¶

pass 4 turns, an estimated $0.042

flowchart LR A --> B classDef c fill:#ffffff,color:#000000,stroke:#767676 class A c

Output source

# Border below 3 to 1

The label passes, the stroke does not. There is no fixer, because the stroke is a design choice.

```mermaid
flowchart LR
  A --> B
  classDef c fill:#ffffff,color:#000000,stroke:#767676
  class A c
```

`erd_contrast`¶

The original fixture is contrast-text-unfixable-erd.md. It reports contrast-text-below-aa.

erDiagram CAR ||--o{ WHEEL : has classDef c fill:#ffffff,color:#ffffff class CAR c

Original source

# White label on white fill

A directive whose text and fill are both white, in a diagram type where the pair is still audited.

```mermaid
erDiagram
  CAR ||--o{ WHEEL : has
  classDef c fill:#ffffff,color:#ffffff
  class CAR c
```

`claude/claude-haiku-5-5`¶

pass 4 turns, an estimated $0.004

erDiagram CAR ||--o{ WHEEL : has classDef c stroke:#767676 class CAR c

Output source

# White label on white fill

A directive whose text and fill are both white, in a diagram type where the pair is still audited.

```mermaid
erDiagram
  CAR ||--o{ WHEEL : has
  classDef c stroke:#767676
  class CAR c
```

`claude/claude-sonnet-5-5`¶

pass 3 turns, an estimated $0.058

erDiagram CAR ||--o{ WHEEL : has classDef c stroke:#0b5fff,stroke-width:3px class CAR c

Output source

# White label on white fill

A directive whose text and fill are both white, in a diagram type where the pair is still audited.

```mermaid
erDiagram
  CAR ||--o{ WHEEL : has
  classDef c stroke:#0b5fff,stroke-width:3px
  class CAR c
```

`claude/claude-opus-5-5`¶

pass 3 turns, an estimated $0.116

erDiagram CAR ||--o{ WHEEL : has classDef c stroke:#d9480f,stroke-width:2px class CAR c

Output source

# White label on white fill

A directive whose text and fill are both white, in a diagram type where the pair is still audited.

```mermaid
erDiagram
  CAR ||--o{ WHEEL : has
  classDef c stroke:#d9480f,stroke-width:2px
  class CAR c
```

`codex/gpt-6-luna`¶

pass 5 turns, an estimated $0.003

erDiagram CAR ||--o{ WHEEL : has classDef c stroke:#000000 class CAR c

Output source

# White label on white fill

A directive whose text and fill are both white, in a diagram type where the pair is still audited.

```mermaid
erDiagram
  CAR ||--o{ WHEEL : has
  classDef c stroke:#000000
  class CAR c
```

`codex/gpt-5.6-terra`¶

pass 6 turns, an estimated $0.056

erDiagram CAR ||--o{ WHEEL : has classDef c stroke:#000000 class CAR c

Output source

# White label on white fill

A directive whose text and fill are both white, in a diagram type where the pair is still audited.

```mermaid
erDiagram
  CAR ||--o{ WHEEL : has
  classDef c stroke:#000000
  class CAR c
```

`codex/gpt-6.1-sol`¶

pass 5 turns, an estimated $0.032

erDiagram CAR ||--o{ WHEEL : has classDef c stroke:#767676 class CAR c

Output source

# White label on white fill

A directive whose text and fill are both white, in a diagram type where the pair is still audited.

```mermaid
erDiagram
  CAR ||--o{ WHEEL : has
  classDef c stroke:#767676
  class CAR c
```

`label_colour`¶

The original fixture is contrast-directive-no-color.md. It reports contrast-directive-unauditable.

flowchart LR A --> B classDef c fill:#123456 class A c

Original source

# Fill with no label colour

A fill with no `color:` beside it cannot be audited, so it is reported rather than skipped.

```mermaid
flowchart LR
  A --> B
  classDef c fill:#123456
  class A c
```

`claude/claude-haiku-5-5`¶

pass 4 turns, an estimated $0.004

flowchart LR A --> B classDef c fill:#123456,color:#ffffff class A c

Output source

# Fill with no label colour

A fill with no `color:` beside it cannot be audited, so it is reported rather than skipped.

```mermaid
flowchart LR
  A --> B
  classDef c fill:#123456,color:#ffffff
  class A c
```

`claude/claude-sonnet-5-5`¶

pass 3 turns, an estimated $0.053

flowchart LR A --> B classDef c fill:#123456,color:#ffffff class A c

Output source

# Fill with no label colour

A fill with no `color:` beside it cannot be audited, so it is reported rather than skipped.

```mermaid
flowchart LR
  A --> B
  classDef c fill:#123456,color:#ffffff
  class A c
```

`claude/claude-opus-5-5`¶

pass 3 turns, an estimated $0.106

flowchart LR A --> B classDef c fill:#123456,color:#ffffff class A c

Output source

# Fill with no label colour

A fill with no `color:` beside it cannot be audited, so it is reported rather than skipped.

```mermaid
flowchart LR
  A --> B
  classDef c fill:#123456,color:#ffffff
  class A c
```

`codex/gpt-6-luna`¶

pass 7 turns, an estimated $0.003

flowchart LR A --> B classDef c fill:#123456,color:#ffffff class A c

Output source

# Fill with no label colour

A fill with no `color:` beside it cannot be audited, so it is reported rather than skipped.

```mermaid
flowchart LR
  A --> B
  classDef c fill:#123456,color:#ffffff
  class A c
```

`codex/gpt-5.6-terra`¶

pass 6 turns, an estimated $0.056

flowchart LR A --> B classDef c fill:#123456,color:#fff class A c

Output source

# Fill with no label colour

A fill with no `color:` beside it cannot be audited, so it is reported rather than skipped.

```mermaid
flowchart LR
  A --> B
  classDef c fill:#123456,color:#fff
  class A c
```

`codex/gpt-6.1-sol`¶

pass 5 turns, an estimated $0.030

flowchart LR A --> B classDef c fill:#123456,color:#ffffff class A c

Output source

# Fill with no label colour

A fill with no `color:` beside it cannot be audited, so it is reported rather than skipped.

```mermaid
flowchart LR
  A --> B
  classDef c fill:#123456,color:#ffffff
  class A c
```

`large_chain`¶

The original fixture is complexity-large-chain.md. It reports complexity-node-count-hard-limit and complexity-visual-score-critical.

flowchart LR
  N0 --> N1
  N1 --> N2
  N2 --> N3
  N3 --> N4
  N4 --> N5
  N5 --> N6
  N6 --> N7
  N7 --> N8
  N8 --> N9
  N9 --> N10
  N10 --> N11
  N11 --> N12
  N12 --> N13
  N13 --> N14
  N14 --> N15
  N15 --> N16
  N16 --> N17
  N17 --> N18
  N18 --> N19
  N19 --> N20
  N20 --> N21
  N21 --> N22
  N22 --> N23
  N23 --> N24
  N24 --> N25
  N25 --> N26
  N26 --> N27
  N27 --> N28
  N28 --> N29
  N29 --> N30
  N30 --> N31
  N31 --> N32
  N32 --> N33
  N33 --> N34
  N34 --> N35
  N35 --> N36
  N36 --> N37
  N37 --> N38
  N38 --> N39
  N39 --> N40
  N40 --> N41
  N41 --> N42
  N42 --> N43
  N43 --> N44
  N44 --> N45
  N45 --> N46
  N46 --> N47
  N47 --> N48
  N48 --> N49
  N49 --> N50
  N50 --> N51
  N51 --> N52
  N52 --> N53
  N53 --> N54
  N54 --> N55
  N55 --> N56
  N56 --> N57
  N57 --> N58
  N58 --> N59
  N59 --> N60
  N60 --> N61
  N61 --> N62
  N62 --> N63
  N63 --> N64
  N64 --> N65
  N65 --> N66
  N66 --> N67
  N67 --> N68
  N68 --> N69
  N69 --> N70
  N70 --> N71
  N71 --> N72
  N72 --> N73
  N73 --> N74
  N74 --> N75
  N75 --> N76
  N76 --> N77
  N77 --> N78
  N78 --> N79
  N79 --> N80
  N80 --> N81
  N81 --> N82
  N82 --> N83
  N83 --> N84
  N84 --> N85
  N85 --> N86
  N86 --> N87
  N87 --> N88
  N88 --> N89
  N89 --> N90
  N90 --> N91
  N91 --> N92
  N92 --> N93
  N93 --> N94
  N94 --> N95
  N95 --> N96
  N96 --> N97
  N97 --> N98
  N98 --> N99
  N99 --> N100
  N100 --> N101
  N101 --> N102
  N102 --> N103
  N103 --> N104
  N104 --> N105
  N105 --> N106
  N106 --> N107
  N107 --> N108
  N108 --> N109
  N109 --> N110
  N110 --> N111
  N111 --> N112
  N112 --> N113
  N113 --> N114
  N114 --> N115
  N115 --> N116
  N116 --> N117
  N117 --> N118
  N118 --> N119
  N119 --> N120

Original source

# A long chain

```mermaid
flowchart LR
  N0 --> N1
  N1 --> N2
  N2 --> N3
  N3 --> N4
  N4 --> N5
  N5 --> N6
  N6 --> N7
  N7 --> N8
  N8 --> N9
  N9 --> N10
  N10 --> N11
  N11 --> N12
  N12 --> N13
  N13 --> N14
  N14 --> N15
  N15 --> N16
  N16 --> N17
  N17 --> N18
  N18 --> N19
  N19 --> N20
  N20 --> N21
  N21 --> N22
  N22 --> N23
  N23 --> N24
  N24 --> N25
  N25 --> N26
  N26 --> N27
  N27 --> N28
  N28 --> N29
  N29 --> N30
  N30 --> N31
  N31 --> N32
  N32 --> N33
  N33 --> N34
  N34 --> N35
  N35 --> N36
  N36 --> N37
  N37 --> N38
  N38 --> N39
  N39 --> N40
  N40 --> N41
  N41 --> N42
  N42 --> N43
  N43 --> N44
  N44 --> N45
  N45 --> N46
  N46 --> N47
  N47 --> N48
  N48 --> N49
  N49 --> N50
  N50 --> N51
  N51 --> N52
  N52 --> N53
  N53 --> N54
  N54 --> N55
  N55 --> N56
  N56 --> N57
  N57 --> N58
  N58 --> N59
  N59 --> N60
  N60 --> N61
  N61 --> N62
  N62 --> N63
  N63 --> N64
  N64 --> N65
  N65 --> N66
  N66 --> N67
  N67 --> N68
  N68 --> N69
  N69 --> N70
  N70 --> N71
  N71 --> N72
  N72 --> N73
  N73 --> N74
  N74 --> N75
  N75 --> N76
  N76 --> N77
  N77 --> N78
  N78 --> N79
  N79 --> N80
  N80 --> N81
  N81 --> N82
  N82 --> N83
  N83 --> N84
  N84 --> N85
  N85 --> N86
  N86 --> N87
  N87 --> N88
  N88 --> N89
  N89 --> N90
  N90 --> N91
  N91 --> N92
  N92 --> N93
  N93 --> N94
  N94 --> N95
  N95 --> N96
  N96 --> N97
  N97 --> N98
  N98 --> N99
  N99 --> N100
  N100 --> N101
  N101 --> N102
  N102 --> N103
  N103 --> N104
  N104 --> N105
  N105 --> N106
  N106 --> N107
  N107 --> N108
  N108 --> N109
  N109 --> N110
  N110 --> N111
  N111 --> N112
  N112 --> N113
  N113 --> N114
  N114 --> N115
  N115 --> N116
  N116 --> N117
  N117 --> N118
  N118 --> N119
  N119 --> N120
```

`claude/claude-haiku-5-5`¶

pass 5 turns, an estimated $0.006

flowchart LR N24 --> N25 N49 --> N50 N74 --> N75 N99 --> N100

flowchart LR N0 --> N1 N1 --> N2 N2 --> N3 N3 --> N4 N4 --> N5 N5 --> N6 N6 --> N7 N7 --> N8 N8 --> N9 N9 --> N10 N10 --> N11 N11 --> N12 N12 --> N13 N13 --> N14 N14 --> N15 N15 --> N16 N16 --> N17 N17 --> N18 N18 --> N19 N19 --> N20 N20 --> N21 N21 --> N22 N22 --> N23 N23 --> N24

flowchart LR N25 --> N26 N26 --> N27 N27 --> N28 N28 --> N29 N29 --> N30 N30 --> N31 N31 --> N32 N32 --> N33 N33 --> N34 N34 --> N35 N35 --> N36 N36 --> N37 N37 --> N38 N38 --> N39 N39 --> N40 N40 --> N41 N41 --> N42 N42 --> N43 N43 --> N44 N44 --> N45 N45 --> N46 N46 --> N47 N47 --> N48 N48 --> N49

flowchart LR N50 --> N51 N51 --> N52 N52 --> N53 N53 --> N54 N54 --> N55 N55 --> N56 N56 --> N57 N57 --> N58 N58 --> N59 N59 --> N60 N60 --> N61 N61 --> N62 N62 --> N63 N63 --> N64 N64 --> N65 N65 --> N66 N66 --> N67 N67 --> N68 N68 --> N69 N69 --> N70 N70 --> N71 N71 --> N72 N72 --> N73 N73 --> N74

flowchart LR N75 --> N76 N76 --> N77 N77 --> N78 N78 --> N79 N79 --> N80 N80 --> N81 N81 --> N82 N82 --> N83 N83 --> N84 N84 --> N85 N85 --> N86 N86 --> N87 N87 --> N88 N88 --> N89 N89 --> N90 N90 --> N91 N91 --> N92 N92 --> N93 N93 --> N94 N94 --> N95 N95 --> N96 N96 --> N97 N97 --> N98 N98 --> N99

flowchart LR N100 --> N101 N101 --> N102 N102 --> N103 N103 --> N104 N104 --> N105 N105 --> N106 N106 --> N107 N107 --> N108 N108 --> N109 N109 --> N110 N110 --> N111 N111 --> N112 N112 --> N113 N113 --> N114 N114 --> N115 N115 --> N116 N116 --> N117 N117 --> N118 N118 --> N119 N119 --> N120

Output source

# A long chain

### Overview

The chain is split into five parts. Each edge that crosses a part boundary is drawn here.

```mermaid
flowchart LR
  N24 --> N25
  N49 --> N50
  N74 --> N75
  N99 --> N100
```

### Part 1: N0 to N24

```mermaid
flowchart LR
  N0 --> N1
  N1 --> N2
  N2 --> N3
  N3 --> N4
  N4 --> N5
  N5 --> N6
  N6 --> N7
  N7 --> N8
  N8 --> N9
  N9 --> N10
  N10 --> N11
  N11 --> N12
  N12 --> N13
  N13 --> N14
  N14 --> N15
  N15 --> N16
  N16 --> N17
  N17 --> N18
  N18 --> N19
  N19 --> N20
  N20 --> N21
  N21 --> N22
  N22 --> N23
  N23 --> N24
```

### Part 2: N25 to N49

```mermaid
flowchart LR
  N25 --> N26
  N26 --> N27
  N27 --> N28
  N28 --> N29
  N29 --> N30
  N30 --> N31
  N31 --> N32
  N32 --> N33
  N33 --> N34
  N34 --> N35
  N35 --> N36
  N36 --> N37
  N37 --> N38
  N38 --> N39
  N39 --> N40
  N40 --> N41
  N41 --> N42
  N42 --> N43
  N43 --> N44
  N44 --> N45
  N45 --> N46
  N46 --> N47
  N47 --> N48
  N48 --> N49
```

### Part 3: N50 to N74

```mermaid
flowchart LR
  N50 --> N51
  N51 --> N52
  N52 --> N53
  N53 --> N54
  N54 --> N55
  N55 --> N56
  N56 --> N57
  N57 --> N58
  N58 --> N59
  N59 --> N60
  N60 --> N61
  N61 --> N62
  N62 --> N63
  N63 --> N64
  N64 --> N65
  N65 --> N66
  N66 --> N67
  N67 --> N68
  N68 --> N69
  N69 --> N70
  N70 --> N71
  N71 --> N72
  N72 --> N73
  N73 --> N74
```

### Part 4: N75 to N99

```mermaid
flowchart LR
  N75 --> N76
  N76 --> N77
  N77 --> N78
  N78 --> N79
  N79 --> N80
  N80 --> N81
  N81 --> N82
  N82 --> N83
  N83 --> N84
  N84 --> N85
  N85 --> N86
  N86 --> N87
  N87 --> N88
  N88 --> N89
  N89 --> N90
  N90 --> N91
  N91 --> N92
  N92 --> N93
  N93 --> N94
  N94 --> N95
  N95 --> N96
  N96 --> N97
  N97 --> N98
  N98 --> N99
```

### Part 5: N100 to N120

```mermaid
flowchart LR
  N100 --> N101
  N101 --> N102
  N102 --> N103
  N103 --> N104
  N104 --> N105
  N105 --> N106
  N106 --> N107
  N107 --> N108
  N108 --> N109
  N109 --> N110
  N110 --> N111
  N111 --> N112
  N112 --> N113
  N113 --> N114
  N114 --> N115
  N115 --> N116
  N116 --> N117
  N117 --> N118
  N118 --> N119
  N119 --> N120
```

`claude/claude-sonnet-5-5`¶

pass 3 turns, an estimated $0.070

flowchart LR subgraph P1["Part 1: N0 to N24"] N0 N24 end subgraph P2["Part 2: N25 to N49"] N25 N49 end subgraph P3["Part 3: N50 to N74"] N50 N74 end subgraph P4["Part 4: N75 to N99"] N75 N99 end subgraph P5["Part 5: N100 to N120"] N100 N120 end N24 --> N25 N49 --> N50 N74 --> N75 N99 --> N100

flowchart LR N0 --> N1 N1 --> N2 N2 --> N3 N3 --> N4 N4 --> N5 N5 --> N6 N6 --> N7 N7 --> N8 N8 --> N9 N9 --> N10 N10 --> N11 N11 --> N12 N12 --> N13 N13 --> N14 N14 --> N15 N15 --> N16 N16 --> N17 N17 --> N18 N18 --> N19 N19 --> N20 N20 --> N21 N21 --> N22 N22 --> N23 N23 --> N24

flowchart LR N25 --> N26 N26 --> N27 N27 --> N28 N28 --> N29 N29 --> N30 N30 --> N31 N31 --> N32 N32 --> N33 N33 --> N34 N34 --> N35 N35 --> N36 N36 --> N37 N37 --> N38 N38 --> N39 N39 --> N40 N40 --> N41 N41 --> N42 N42 --> N43 N43 --> N44 N44 --> N45 N45 --> N46 N46 --> N47 N47 --> N48 N48 --> N49

flowchart LR N50 --> N51 N51 --> N52 N52 --> N53 N53 --> N54 N54 --> N55 N55 --> N56 N56 --> N57 N57 --> N58 N58 --> N59 N59 --> N60 N60 --> N61 N61 --> N62 N62 --> N63 N63 --> N64 N64 --> N65 N65 --> N66 N66 --> N67 N67 --> N68 N68 --> N69 N69 --> N70 N70 --> N71 N71 --> N72 N72 --> N73 N73 --> N74

flowchart LR N75 --> N76 N76 --> N77 N77 --> N78 N78 --> N79 N79 --> N80 N80 --> N81 N81 --> N82 N82 --> N83 N83 --> N84 N84 --> N85 N85 --> N86 N86 --> N87 N87 --> N88 N88 --> N89 N89 --> N90 N90 --> N91 N91 --> N92 N92 --> N93 N93 --> N94 N94 --> N95 N95 --> N96 N96 --> N97 N97 --> N98 N98 --> N99

flowchart LR N100 --> N101 N101 --> N102 N102 --> N103 N103 --> N104 N104 --> N105 N105 --> N106 N106 --> N107 N107 --> N108 N108 --> N109 N109 --> N110 N110 --> N111 N111 --> N112 N112 --> N113 N113 --> N114 N114 --> N115 N115 --> N116 N116 --> N117 N117 --> N118 N118 --> N119 N119 --> N120

Output source

# A long chain

The chain is split into an overview and five parts of at most 25 nodes each.

## Overview

```mermaid
flowchart LR
  subgraph P1["Part 1: N0 to N24"]
    N0
    N24
  end
  subgraph P2["Part 2: N25 to N49"]
    N25
    N49
  end
  subgraph P3["Part 3: N50 to N74"]
    N50
    N74
  end
  subgraph P4["Part 4: N75 to N99"]
    N75
    N99
  end
  subgraph P5["Part 5: N100 to N120"]
    N100
    N120
  end
  N24 --> N25
  N49 --> N50
  N74 --> N75
  N99 --> N100
```

## Part 1

```mermaid
flowchart LR
  N0 --> N1
  N1 --> N2
  N2 --> N3
  N3 --> N4
  N4 --> N5
  N5 --> N6
  N6 --> N7
  N7 --> N8
  N8 --> N9
  N9 --> N10
  N10 --> N11
  N11 --> N12
  N12 --> N13
  N13 --> N14
  N14 --> N15
  N15 --> N16
  N16 --> N17
  N17 --> N18
  N18 --> N19
  N19 --> N20
  N20 --> N21
  N21 --> N22
  N22 --> N23
  N23 --> N24
```

## Part 2

```mermaid
flowchart LR
  N25 --> N26
  N26 --> N27
  N27 --> N28
  N28 --> N29
  N29 --> N30
  N30 --> N31
  N31 --> N32
  N32 --> N33
  N33 --> N34
  N34 --> N35
  N35 --> N36
  N36 --> N37
  N37 --> N38
  N38 --> N39
  N39 --> N40
  N40 --> N41
  N41 --> N42
  N42 --> N43
  N43 --> N44
  N44 --> N45
  N45 --> N46
  N46 --> N47
  N47 --> N48
  N48 --> N49
```

## Part 3

```mermaid
flowchart LR
  N50 --> N51
  N51 --> N52
  N52 --> N53
  N53 --> N54
  N54 --> N55
  N55 --> N56
  N56 --> N57
  N57 --> N58
  N58 --> N59
  N59 --> N60
  N60 --> N61
  N61 --> N62
  N62 --> N63
  N63 --> N64
  N64 --> N65
  N65 --> N66
  N66 --> N67
  N67 --> N68
  N68 --> N69
  N69 --> N70
  N70 --> N71
  N71 --> N72
  N72 --> N73
  N73 --> N74
```

## Part 4

```mermaid
flowchart LR
  N75 --> N76
  N76 --> N77
  N77 --> N78
  N78 --> N79
  N79 --> N80
  N80 --> N81
  N81 --> N82
  N82 --> N83
  N83 --> N84
  N84 --> N85
  N85 --> N86
  N86 --> N87
  N87 --> N88
  N88 --> N89
  N89 --> N90
  N90 --> N91
  N91 --> N92
  N92 --> N93
  N93 --> N94
  N94 --> N95
  N95 --> N96
  N96 --> N97
  N97 --> N98
  N98 --> N99
```

## Part 5

```mermaid
flowchart LR
  N100 --> N101
  N101 --> N102
  N102 --> N103
  N103 --> N104
  N104 --> N105
  N105 --> N106
  N106 --> N107
  N107 --> N108
  N108 --> N109
  N109 --> N110
  N110 --> N111
  N111 --> N112
  N112 --> N113
  N113 --> N114
  N114 --> N115
  N115 --> N116
  N116 --> N117
  N117 --> N118
  N118 --> N119
  N119 --> N120
```

`claude/claude-opus-5-5`¶

pass 3 turns, an estimated $0.145

flowchart LR subgraph P1["Part 1: N0 to N19"] N0 N19 end subgraph P2["Part 2: N20 to N39"] N20 N39 end subgraph P3["Part 3: N40 to N59"] N40 N59 end subgraph P4["Part 4: N60 to N79"] N60 N79 end subgraph P5["Part 5: N80 to N99"] N80 N99 end subgraph P6["Part 6: N100 to N120"] N100 N120 end N19 --> N20 N39 --> N40 N59 --> N60 N79 --> N80 N99 --> N100

flowchart LR N0 --> N1 N1 --> N2 N2 --> N3 N3 --> N4 N4 --> N5 N5 --> N6 N6 --> N7 N7 --> N8 N8 --> N9 N9 --> N10 N10 --> N11 N11 --> N12 N12 --> N13 N13 --> N14 N14 --> N15 N15 --> N16 N16 --> N17 N17 --> N18 N18 --> N19

flowchart LR N20 --> N21 N21 --> N22 N22 --> N23 N23 --> N24 N24 --> N25 N25 --> N26 N26 --> N27 N27 --> N28 N28 --> N29 N29 --> N30 N30 --> N31 N31 --> N32 N32 --> N33 N33 --> N34 N34 --> N35 N35 --> N36 N36 --> N37 N37 --> N38 N38 --> N39

flowchart LR N40 --> N41 N41 --> N42 N42 --> N43 N43 --> N44 N44 --> N45 N45 --> N46 N46 --> N47 N47 --> N48 N48 --> N49 N49 --> N50 N50 --> N51 N51 --> N52 N52 --> N53 N53 --> N54 N54 --> N55 N55 --> N56 N56 --> N57 N57 --> N58 N58 --> N59

flowchart LR N60 --> N61 N61 --> N62 N62 --> N63 N63 --> N64 N64 --> N65 N65 --> N66 N66 --> N67 N67 --> N68 N68 --> N69 N69 --> N70 N70 --> N71 N71 --> N72 N72 --> N73 N73 --> N74 N74 --> N75 N75 --> N76 N76 --> N77 N77 --> N78 N78 --> N79

flowchart LR N80 --> N81 N81 --> N82 N82 --> N83 N83 --> N84 N84 --> N85 N85 --> N86 N86 --> N87 N87 --> N88 N88 --> N89 N89 --> N90 N90 --> N91 N91 --> N92 N92 --> N93 N93 --> N94 N94 --> N95 N95 --> N96 N96 --> N97 N97 --> N98 N98 --> N99

flowchart LR N100 --> N101 N101 --> N102 N102 --> N103 N103 --> N104 N104 --> N105 N105 --> N106 N106 --> N107 N107 --> N108 N108 --> N109 N109 --> N110 N110 --> N111 N111 --> N112 N112 --> N113 N113 --> N114 N114 --> N115 N115 --> N116 N116 --> N117 N117 --> N118 N118 --> N119 N119 --> N120

Output source

# A long chain

```mermaid
flowchart LR
  subgraph P1["Part 1: N0 to N19"]
    N0
    N19
  end
  subgraph P2["Part 2: N20 to N39"]
    N20
    N39
  end
  subgraph P3["Part 3: N40 to N59"]
    N40
    N59
  end
  subgraph P4["Part 4: N60 to N79"]
    N60
    N79
  end
  subgraph P5["Part 5: N80 to N99"]
    N80
    N99
  end
  subgraph P6["Part 6: N100 to N120"]
    N100
    N120
  end
  N19 --> N20
  N39 --> N40
  N59 --> N60
  N79 --> N80
  N99 --> N100
```

```mermaid
flowchart LR
  N0 --> N1
  N1 --> N2
  N2 --> N3
  N3 --> N4
  N4 --> N5
  N5 --> N6
  N6 --> N7
  N7 --> N8
  N8 --> N9
  N9 --> N10
  N10 --> N11
  N11 --> N12
  N12 --> N13
  N13 --> N14
  N14 --> N15
  N15 --> N16
  N16 --> N17
  N17 --> N18
  N18 --> N19
```

```mermaid
flowchart LR
  N20 --> N21
  N21 --> N22
  N22 --> N23
  N23 --> N24
  N24 --> N25
  N25 --> N26
  N26 --> N27
  N27 --> N28
  N28 --> N29
  N29 --> N30
  N30 --> N31
  N31 --> N32
  N32 --> N33
  N33 --> N34
  N34 --> N35
  N35 --> N36
  N36 --> N37
  N37 --> N38
  N38 --> N39
```

```mermaid
flowchart LR
  N40 --> N41
  N41 --> N42
  N42 --> N43
  N43 --> N44
  N44 --> N45
  N45 --> N46
  N46 --> N47
  N47 --> N48
  N48 --> N49
  N49 --> N50
  N50 --> N51
  N51 --> N52
  N52 --> N53
  N53 --> N54
  N54 --> N55
  N55 --> N56
  N56 --> N57
  N57 --> N58
  N58 --> N59
```

```mermaid
flowchart LR
  N60 --> N61
  N61 --> N62
  N62 --> N63
  N63 --> N64
  N64 --> N65
  N65 --> N66
  N66 --> N67
  N67 --> N68
  N68 --> N69
  N69 --> N70
  N70 --> N71
  N71 --> N72
  N72 --> N73
  N73 --> N74
  N74 --> N75
  N75 --> N76
  N76 --> N77
  N77 --> N78
  N78 --> N79
```

```mermaid
flowchart LR
  N80 --> N81
  N81 --> N82
  N82 --> N83
  N83 --> N84
  N84 --> N85
  N85 --> N86
  N86 --> N87
  N87 --> N88
  N88 --> N89
  N89 --> N90
  N90 --> N91
  N91 --> N92
  N92 --> N93
  N93 --> N94
  N94 --> N95
  N95 --> N96
  N96 --> N97
  N97 --> N98
  N98 --> N99
```

```mermaid
flowchart LR
  N100 --> N101
  N101 --> N102
  N102 --> N103
  N103 --> N104
  N104 --> N105
  N105 --> N106
  N106 --> N107
  N107 --> N108
  N108 --> N109
  N109 --> N110
  N110 --> N111
  N111 --> N112
  N112 --> N113
  N113 --> N114
  N114 --> N115
  N115 --> N116
  N116 --> N117
  N117 --> N118
  N118 --> N119
  N119 --> N120
```

`codex/gpt-6-luna`¶

pass 7 turns, an estimated $0.005

flowchart LR Part1["N0–N24"] --> Part2["N25–N49"] Part2 --> Part3["N50–N74"] Part3 --> Part4["N75–N99"] Part4 --> Part5["N100–N120"]

flowchart LR N0 --> N1 N1 --> N2 N2 --> N3 N3 --> N4 N4 --> N5 N5 --> N6 N6 --> N7 N7 --> N8 N8 --> N9 N9 --> N10 N10 --> N11 N11 --> N12 N12 --> N13 N13 --> N14 N14 --> N15 N15 --> N16 N16 --> N17 N17 --> N18 N18 --> N19 N19 --> N20 N20 --> N21 N21 --> N22 N22 --> N23 N23 --> N24

flowchart LR N25 --> N26 N26 --> N27 N27 --> N28 N28 --> N29 N29 --> N30 N30 --> N31 N31 --> N32 N32 --> N33 N33 --> N34 N34 --> N35 N35 --> N36 N36 --> N37 N37 --> N38 N38 --> N39 N39 --> N40 N40 --> N41 N41 --> N42 N42 --> N43 N43 --> N44 N44 --> N45 N45 --> N46 N46 --> N47 N47 --> N48 N48 --> N49

flowchart LR N50 --> N51 N51 --> N52 N52 --> N53 N53 --> N54 N54 --> N55 N55 --> N56 N56 --> N57 N57 --> N58 N58 --> N59 N59 --> N60 N60 --> N61 N61 --> N62 N62 --> N63 N63 --> N64 N64 --> N65 N65 --> N66 N66 --> N67 N67 --> N68 N68 --> N69 N69 --> N70 N70 --> N71 N71 --> N72 N72 --> N73 N73 --> N74

flowchart LR N75 --> N76 N76 --> N77 N77 --> N78 N78 --> N79 N79 --> N80 N80 --> N81 N81 --> N82 N82 --> N83 N83 --> N84 N84 --> N85 N85 --> N86 N86 --> N87 N87 --> N88 N88 --> N89 N89 --> N90 N90 --> N91 N91 --> N92 N92 --> N93 N93 --> N94 N94 --> N95 N95 --> N96 N96 --> N97 N97 --> N98 N98 --> N99

flowchart LR N100 --> N101 N101 --> N102 N102 --> N103 N103 --> N104 N104 --> N105 N105 --> N106 N106 --> N107 N107 --> N108 N108 --> N109 N109 --> N110 N110 --> N111 N111 --> N112 N112 --> N113 N113 --> N114 N114 --> N115 N115 --> N116 N116 --> N117 N117 --> N118 N118 --> N119 N119 --> N120

Output source

# A long chain

```mermaid
flowchart LR
  Part1["N0–N24"] --> Part2["N25–N49"]
  Part2 --> Part3["N50–N74"]
  Part3 --> Part4["N75–N99"]
  Part4 --> Part5["N100–N120"]
```

```mermaid
flowchart LR
  N0 --> N1
  N1 --> N2
  N2 --> N3
  N3 --> N4
  N4 --> N5
  N5 --> N6
  N6 --> N7
  N7 --> N8
  N8 --> N9
  N9 --> N10
  N10 --> N11
  N11 --> N12
  N12 --> N13
  N13 --> N14
  N14 --> N15
  N15 --> N16
  N16 --> N17
  N17 --> N18
  N18 --> N19
  N19 --> N20
  N20 --> N21
  N21 --> N22
  N22 --> N23
  N23 --> N24
```

```mermaid
flowchart LR
  N25 --> N26
  N26 --> N27
  N27 --> N28
  N28 --> N29
  N29 --> N30
  N30 --> N31
  N31 --> N32
  N32 --> N33
  N33 --> N34
  N34 --> N35
  N35 --> N36
  N36 --> N37
  N37 --> N38
  N38 --> N39
  N39 --> N40
  N40 --> N41
  N41 --> N42
  N42 --> N43
  N43 --> N44
  N44 --> N45
  N45 --> N46
  N46 --> N47
  N47 --> N48
  N48 --> N49
```

```mermaid
flowchart LR
  N50 --> N51
  N51 --> N52
  N52 --> N53
  N53 --> N54
  N54 --> N55
  N55 --> N56
  N56 --> N57
  N57 --> N58
  N58 --> N59
  N59 --> N60
  N60 --> N61
  N61 --> N62
  N62 --> N63
  N63 --> N64
  N64 --> N65
  N65 --> N66
  N66 --> N67
  N67 --> N68
  N68 --> N69
  N69 --> N70
  N70 --> N71
  N71 --> N72
  N72 --> N73
  N73 --> N74
```

```mermaid
flowchart LR
  N75 --> N76
  N76 --> N77
  N77 --> N78
  N78 --> N79
  N79 --> N80
  N80 --> N81
  N81 --> N82
  N82 --> N83
  N83 --> N84
  N84 --> N85
  N85 --> N86
  N86 --> N87
  N87 --> N88
  N88 --> N89
  N89 --> N90
  N90 --> N91
  N91 --> N92
  N92 --> N93
  N93 --> N94
  N94 --> N95
  N95 --> N96
  N96 --> N97
  N97 --> N98
  N98 --> N99
```

```mermaid
flowchart LR
  N100 --> N101
  N101 --> N102
  N102 --> N103
  N103 --> N104
  N104 --> N105
  N105 --> N106
  N106 --> N107
  N107 --> N108
  N108 --> N109
  N109 --> N110
  N110 --> N111
  N111 --> N112
  N112 --> N113
  N113 --> N114
  N114 --> N115
  N115 --> N116
  N116 --> N117
  N117 --> N118
  N118 --> N119
  N119 --> N120
```

`codex/gpt-5.6-terra`¶

pass 13 turns, an estimated $0.150

flowchart LR Stage1[Nodes N0–N24] --> Stage2[Nodes N24–N48] Stage2 --> Stage3[Nodes N48–N72] Stage3 --> Stage4[Nodes N72–N96] Stage4 --> Stage5[Nodes N96–N120]

flowchart LR N0 --> N1 N1 --> N2 N2 --> N3 N3 --> N4 N4 --> N5 N5 --> N6 N6 --> N7 N7 --> N8 N8 --> N9 N9 --> N10 N10 --> N11 N11 --> N12 N12 --> N13 N13 --> N14 N14 --> N15 N15 --> N16 N16 --> N17 N17 --> N18 N18 --> N19 N19 --> N20 N20 --> N21 N21 --> N22 N22 --> N23 N23 --> N24

flowchart LR N24 --> N25 N25 --> N26 N26 --> N27 N27 --> N28 N28 --> N29 N29 --> N30 N30 --> N31 N31 --> N32 N32 --> N33 N33 --> N34 N34 --> N35 N35 --> N36 N36 --> N37 N37 --> N38 N38 --> N39 N39 --> N40 N40 --> N41 N41 --> N42 N42 --> N43 N43 --> N44 N44 --> N45 N45 --> N46 N46 --> N47 N47 --> N48

flowchart LR N48 --> N49 N49 --> N50 N50 --> N51 N51 --> N52 N52 --> N53 N53 --> N54 N54 --> N55 N55 --> N56 N56 --> N57 N57 --> N58 N58 --> N59 N59 --> N60 N60 --> N61 N61 --> N62 N62 --> N63 N63 --> N64 N64 --> N65 N65 --> N66 N66 --> N67 N67 --> N68 N68 --> N69 N69 --> N70 N70 --> N71 N71 --> N72

flowchart LR N72 --> N73 N73 --> N74 N74 --> N75 N75 --> N76 N76 --> N77 N77 --> N78 N78 --> N79 N79 --> N80 N80 --> N81 N81 --> N82 N82 --> N83 N83 --> N84 N84 --> N85 N85 --> N86 N86 --> N87 N87 --> N88 N88 --> N89 N89 --> N90 N90 --> N91 N91 --> N92 N92 --> N93 N93 --> N94 N94 --> N95 N95 --> N96

flowchart LR N96 --> N97 N97 --> N98 N98 --> N99 N99 --> N100 N100 --> N101 N101 --> N102 N102 --> N103 N103 --> N104 N104 --> N105 N105 --> N106 N106 --> N107 N107 --> N108 N108 --> N109 N109 --> N110 N110 --> N111 N111 --> N112 N112 --> N113 N113 --> N114 N114 --> N115 N115 --> N116 N116 --> N117 N117 --> N118 N118 --> N119 N119 --> N120

Output source

# A long chain

```mermaid
flowchart LR
  Stage1[Nodes N0–N24] --> Stage2[Nodes N24–N48]
  Stage2 --> Stage3[Nodes N48–N72]
  Stage3 --> Stage4[Nodes N72–N96]
  Stage4 --> Stage5[Nodes N96–N120]
```

```mermaid
flowchart LR
  N0 --> N1
  N1 --> N2
  N2 --> N3
  N3 --> N4
  N4 --> N5
  N5 --> N6
  N6 --> N7
  N7 --> N8
  N8 --> N9
  N9 --> N10
  N10 --> N11
  N11 --> N12
  N12 --> N13
  N13 --> N14
  N14 --> N15
  N15 --> N16
  N16 --> N17
  N17 --> N18
  N18 --> N19
  N19 --> N20
  N20 --> N21
  N21 --> N22
  N22 --> N23
  N23 --> N24
```

```mermaid
flowchart LR
  N24 --> N25
  N25 --> N26
  N26 --> N27
  N27 --> N28
  N28 --> N29
  N29 --> N30
  N30 --> N31
  N31 --> N32
  N32 --> N33
  N33 --> N34
  N34 --> N35
  N35 --> N36
  N36 --> N37
  N37 --> N38
  N38 --> N39
  N39 --> N40
  N40 --> N41
  N41 --> N42
  N42 --> N43
  N43 --> N44
  N44 --> N45
  N45 --> N46
  N46 --> N47
  N47 --> N48
```

```mermaid
flowchart LR
  N48 --> N49
  N49 --> N50
  N50 --> N51
  N51 --> N52
  N52 --> N53
  N53 --> N54
  N54 --> N55
  N55 --> N56
  N56 --> N57
  N57 --> N58
  N58 --> N59
  N59 --> N60
  N60 --> N61
  N61 --> N62
  N62 --> N63
  N63 --> N64
  N64 --> N65
  N65 --> N66
  N66 --> N67
  N67 --> N68
  N68 --> N69
  N69 --> N70
  N70 --> N71
  N71 --> N72
```

```mermaid
flowchart LR
  N72 --> N73
  N73 --> N74
  N74 --> N75
  N75 --> N76
  N76 --> N77
  N77 --> N78
  N78 --> N79
  N79 --> N80
  N80 --> N81
  N81 --> N82
  N82 --> N83
  N83 --> N84
  N84 --> N85
  N85 --> N86
  N86 --> N87
  N87 --> N88
  N88 --> N89
  N89 --> N90
  N90 --> N91
  N91 --> N92
  N92 --> N93
  N93 --> N94
  N94 --> N95
  N95 --> N96
```

```mermaid
flowchart LR
  N96 --> N97
  N97 --> N98
  N98 --> N99
  N99 --> N100
  N100 --> N101
  N101 --> N102
  N102 --> N103
  N103 --> N104
  N104 --> N105
  N105 --> N106
  N106 --> N107
  N107 --> N108
  N108 --> N109
  N109 --> N110
  N110 --> N111
  N111 --> N112
  N112 --> N113
  N113 --> N114
  N114 --> N115
  N115 --> N116
  N116 --> N117
  N117 --> N118
  N118 --> N119
  N119 --> N120
```

`codex/gpt-6.1-sol`¶

pass 6 turns, an estimated $0.053

flowchart LR P1["Part 1: N0 to N20"] P2["Part 2: N20 to N40"] P3["Part 3: N40 to N60"] P4["Part 4: N60 to N80"] P5["Part 5: N80 to N100"] P6["Part 6: N100 to N120"] P1 --> P2 P2 --> P3 P3 --> P4 P4 --> P5 P5 --> P6

flowchart LR subgraph Part1["Part 1: N0 to N20"] N0 --> N1 N1 --> N2 N2 --> N3 N3 --> N4 N4 --> N5 N5 --> N6 N6 --> N7 N7 --> N8 N8 --> N9 N9 --> N10 N10 --> N11 N11 --> N12 N12 --> N13 N13 --> N14 N14 --> N15 N15 --> N16 N16 --> N17 N17 --> N18 N18 --> N19 N19 --> N20 end

flowchart LR subgraph Part2["Part 2: N20 to N40"] N20 --> N21 N21 --> N22 N22 --> N23 N23 --> N24 N24 --> N25 N25 --> N26 N26 --> N27 N27 --> N28 N28 --> N29 N29 --> N30 N30 --> N31 N31 --> N32 N32 --> N33 N33 --> N34 N34 --> N35 N35 --> N36 N36 --> N37 N37 --> N38 N38 --> N39 N39 --> N40 end

flowchart LR subgraph Part3["Part 3: N40 to N60"] N40 --> N41 N41 --> N42 N42 --> N43 N43 --> N44 N44 --> N45 N45 --> N46 N46 --> N47 N47 --> N48 N48 --> N49 N49 --> N50 N50 --> N51 N51 --> N52 N52 --> N53 N53 --> N54 N54 --> N55 N55 --> N56 N56 --> N57 N57 --> N58 N58 --> N59 N59 --> N60 end

flowchart LR subgraph Part4["Part 4: N60 to N80"] N60 --> N61 N61 --> N62 N62 --> N63 N63 --> N64 N64 --> N65 N65 --> N66 N66 --> N67 N67 --> N68 N68 --> N69 N69 --> N70 N70 --> N71 N71 --> N72 N72 --> N73 N73 --> N74 N74 --> N75 N75 --> N76 N76 --> N77 N77 --> N78 N78 --> N79 N79 --> N80 end

flowchart LR subgraph Part5["Part 5: N80 to N100"] N80 --> N81 N81 --> N82 N82 --> N83 N83 --> N84 N84 --> N85 N85 --> N86 N86 --> N87 N87 --> N88 N88 --> N89 N89 --> N90 N90 --> N91 N91 --> N92 N92 --> N93 N93 --> N94 N94 --> N95 N95 --> N96 N96 --> N97 N97 --> N98 N98 --> N99 N99 --> N100 end

flowchart LR subgraph Part6["Part 6: N100 to N120"] N100 --> N101 N101 --> N102 N102 --> N103 N103 --> N104 N104 --> N105 N105 --> N106 N106 --> N107 N107 --> N108 N108 --> N109 N109 --> N110 N110 --> N111 N111 --> N112 N112 --> N113 N113 --> N114 N114 --> N115 N115 --> N116 N116 --> N117 N117 --> N118 N118 --> N119 N119 --> N120 end

Output source

# A long chain

```mermaid
flowchart LR
  P1["Part 1: N0 to N20"]
  P2["Part 2: N20 to N40"]
  P3["Part 3: N40 to N60"]
  P4["Part 4: N60 to N80"]
  P5["Part 5: N80 to N100"]
  P6["Part 6: N100 to N120"]
  P1 --> P2
  P2 --> P3
  P3 --> P4
  P4 --> P5
  P5 --> P6
```

```mermaid
flowchart LR
  subgraph Part1["Part 1: N0 to N20"]
  N0 --> N1
  N1 --> N2
  N2 --> N3
  N3 --> N4
  N4 --> N5
  N5 --> N6
  N6 --> N7
  N7 --> N8
  N8 --> N9
  N9 --> N10
  N10 --> N11
  N11 --> N12
  N12 --> N13
  N13 --> N14
  N14 --> N15
  N15 --> N16
  N16 --> N17
  N17 --> N18
  N18 --> N19
  N19 --> N20
  end
```

```mermaid
flowchart LR
  subgraph Part2["Part 2: N20 to N40"]
  N20 --> N21
  N21 --> N22
  N22 --> N23
  N23 --> N24
  N24 --> N25
  N25 --> N26
  N26 --> N27
  N27 --> N28
  N28 --> N29
  N29 --> N30
  N30 --> N31
  N31 --> N32
  N32 --> N33
  N33 --> N34
  N34 --> N35
  N35 --> N36
  N36 --> N37
  N37 --> N38
  N38 --> N39
  N39 --> N40
  end
```

```mermaid
flowchart LR
  subgraph Part3["Part 3: N40 to N60"]
  N40 --> N41
  N41 --> N42
  N42 --> N43
  N43 --> N44
  N44 --> N45
  N45 --> N46
  N46 --> N47
  N47 --> N48
  N48 --> N49
  N49 --> N50
  N50 --> N51
  N51 --> N52
  N52 --> N53
  N53 --> N54
  N54 --> N55
  N55 --> N56
  N56 --> N57
  N57 --> N58
  N58 --> N59
  N59 --> N60
  end
```

```mermaid
flowchart LR
  subgraph Part4["Part 4: N60 to N80"]
  N60 --> N61
  N61 --> N62
  N62 --> N63
  N63 --> N64
  N64 --> N65
  N65 --> N66
  N66 --> N67
  N67 --> N68
  N68 --> N69
  N69 --> N70
  N70 --> N71
  N71 --> N72
  N72 --> N73
  N73 --> N74
  N74 --> N75
  N75 --> N76
  N76 --> N77
  N77 --> N78
  N78 --> N79
  N79 --> N80
  end
```

```mermaid
flowchart LR
  subgraph Part5["Part 5: N80 to N100"]
  N80 --> N81
  N81 --> N82
  N82 --> N83
  N83 --> N84
  N84 --> N85
  N85 --> N86
  N86 --> N87
  N87 --> N88
  N88 --> N89
  N89 --> N90
  N90 --> N91
  N91 --> N92
  N92 --> N93
  N93 --> N94
  N94 --> N95
  N95 --> N96
  N96 --> N97
  N97 --> N98
  N98 --> N99
  N99 --> N100
  end
```

```mermaid
flowchart LR
  subgraph Part6["Part 6: N100 to N120"]
  N100 --> N101
  N101 --> N102
  N102 --> N103
  N103 --> N104
  N104 --> N105
  N105 --> N106
  N106 --> N107
  N107 --> N108
  N108 --> N109
  N109 --> N110
  N110 --> N111
  N111 --> N112
  N112 --> N113
  N113 --> N114
  N114 --> N115
  N115 --> N116
  N116 --> N117
  N117 --> N118
  N118 --> N119
  N119 --> N120
  end
```

`nesting`¶

The original fixture is complexity-nesting.md. It reports complexity-subgraph-nesting-too-deep.

flowchart LR subgraph L1[Level one] subgraph L2[Level two] subgraph L3[Level three] A --> B end end end

Original source

# Subgraphs nested three deep

```mermaid
flowchart LR
  subgraph L1[Level one]
    subgraph L2[Level two]
      subgraph L3[Level three]
        A --> B
      end
    end
  end
```

`claude/claude-haiku-5-5`¶

pass 3 turns, an estimated $0.004

flowchart LR subgraph L1[Level one] subgraph L3[Level three] A --> B end end

Output source

# Subgraphs nested three deep

```mermaid
flowchart LR
  subgraph L1[Level one]
    subgraph L3[Level three]
      A --> B
    end
  end
```

`claude/claude-sonnet-5-5`¶

pass 3 turns, an estimated $0.056

flowchart LR subgraph L1[Level one] subgraph L3[Level two and three] A --> B end end

Output source

# Subgraphs nested three deep

```mermaid
flowchart LR
  subgraph L1[Level one]
    subgraph L3[Level two and three]
      A --> B
    end
  end
```

`claude/claude-opus-5-5`¶

pass 3 turns, an estimated $0.114

flowchart LR subgraph L1[Level one] subgraph L2[Level two and three] A --> B end end

Output source

# Subgraphs nested three deep

```mermaid
flowchart LR
  subgraph L1[Level one]
    subgraph L2[Level two and three]
      A --> B
    end
  end
```

`codex/gpt-6-luna`¶

pass 5 turns, an estimated $0.003

flowchart LR subgraph L1[Level one] subgraph L2[Level two] A --> B end end

Output source

# Subgraphs nested three deep

```mermaid
flowchart LR
  subgraph L1[Level one]
    subgraph L2[Level two]
      A --> B
    end
  end
```

`codex/gpt-5.6-terra`¶

pass 5 turns, an estimated $0.048

flowchart LR subgraph L1[Level one] subgraph L2[Level two] A --> B end end

Output source

# Subgraphs nested three deep

```mermaid
flowchart LR
  subgraph L1[Level one]
    subgraph L2[Level two]
      A --> B
    end
  end
```

`codex/gpt-6.1-sol`¶

pass 4 turns, an estimated $0.043

flowchart LR subgraph L1[Level one] subgraph L2[Level two / Level three] A --> B end end

Output source

# Subgraphs nested three deep

```mermaid
flowchart LR
  subgraph L1[Level one]
    subgraph L2[Level two / Level three]
      A --> B
    end
  end
```

`parse_repair`¶

The original fixture is parse-unparseable.md. It reports parse-diagram-unparseable.

flowchart LR
  A -->

Original source

# Unparseable diagram

An edge with no target. A syntax error needs an author, so it is reported and left alone.

```mermaid
flowchart LR
  A -->
```

`claude/claude-haiku-5-5`¶

pass 4 turns, an estimated $0.004

flowchart LR A --> B

Output source

# Unparseable diagram

An edge with no target. A syntax error needs an author, so it is reported and left alone.

```mermaid
flowchart LR
  A --> B
```

`claude/claude-sonnet-5-5`¶

pass 3 turns, an estimated $0.055

flowchart LR A --> B

Output source

# Unparseable diagram

An edge with no target. A syntax error needs an author, so it is reported and left alone.

```mermaid
flowchart LR
  A --> B
```

`claude/claude-opus-5-5`¶

pass 3 turns, an estimated $0.112

flowchart LR A --> B

Output source

# Unparseable diagram

An edge with no target. A syntax error needs an author, so it is reported and left alone.

```mermaid
flowchart LR
  A --> B
```

`codex/gpt-6-luna`¶

pass 5 turns, an estimated $0.002

flowchart LR A --> B

Output source

# Unparseable diagram

An edge with no target. A syntax error needs an author, so it is reported and left alone.

```mermaid
flowchart LR
  A --> B
```

`codex/gpt-5.6-terra`¶

pass 4 turns, an estimated $0.035

flowchart LR A --> B

Output source

# Unparseable diagram

An edge with no target. A syntax error needs an author, so it is reported and left alone.

```mermaid
flowchart LR
  A --> B
```

`codex/gpt-6.1-sol`¶

pass 4 turns, an estimated $0.029

flowchart LR A --> B

Output source

# Unparseable diagram

An edge with no target. A syntax error needs an author, so it is reported and left alone.

```mermaid
flowchart LR
  A --> B
```

What the results do not show¶

They do not judge meaning. A fix can lint clean and keep every node while changing what the diagram says, and no check here catches that.
They are one run. An agent's output varies from run to run, so one pass is evidence, not a rate.
They cover only the fixtures listed. Each case starts from one small file, not a real document set.

Run them yourself¶

The suite uses the pytest-xharness-eval plugin, a dev dependency of this repository. It needs:

The claude and codex CLIs on your PATH, each logged in.
bun on your PATH, with make install run, for the grader's lint step.
Network access to the npm registry from each agent's sandbox, because the skill runs the linter through bunx or npx.

Preview the matrix without spending anything:

make evals-dry

The dry run writes its report to tmp/evals-dry/report/report.json, so the last live run's report in .xharness_eval_cache/ stays as it was.

Run it live. Every cell is a billed agent session:

make evals

Then copy the run onto this page, and rebuild the site:

make eval-outputs site

The suite's README explains each case and its grader.

Agent evals¶

Which fixtures are evaluated¶

What a cell proves¶

Results¶

What each agent wrote¶

border_contrast¶

claude/claude-haiku-5-5¶

claude/claude-sonnet-5-5¶

claude/claude-opus-5-5¶

codex/gpt-6-luna¶

codex/gpt-5.6-terra¶

codex/gpt-6.1-sol¶

erd_contrast¶

claude/claude-haiku-5-5¶

claude/claude-sonnet-5-5¶

claude/claude-opus-5-5¶

codex/gpt-6-luna¶

codex/gpt-5.6-terra¶

codex/gpt-6.1-sol¶

label_colour¶

claude/claude-haiku-5-5¶

claude/claude-sonnet-5-5¶

claude/claude-opus-5-5¶

codex/gpt-6-luna¶

codex/gpt-5.6-terra¶

codex/gpt-6.1-sol¶

large_chain¶

claude/claude-haiku-5-5¶

claude/claude-sonnet-5-5¶

claude/claude-opus-5-5¶

codex/gpt-6-luna¶

codex/gpt-5.6-terra¶

codex/gpt-6.1-sol¶

nesting¶

claude/claude-haiku-5-5¶

claude/claude-sonnet-5-5¶

claude/claude-opus-5-5¶

codex/gpt-6-luna¶

codex/gpt-5.6-terra¶

codex/gpt-6.1-sol¶

parse_repair¶

claude/claude-haiku-5-5¶

claude/claude-sonnet-5-5¶

claude/claude-opus-5-5¶

codex/gpt-6-luna¶

codex/gpt-5.6-terra¶

codex/gpt-6.1-sol¶

What the results do not show¶

Run them yourself¶

`border_contrast`¶

`claude/claude-haiku-5-5`¶

`claude/claude-sonnet-5-5`¶

`claude/claude-opus-5-5`¶

`codex/gpt-6-luna`¶

`codex/gpt-5.6-terra`¶

`codex/gpt-6.1-sol`¶

`erd_contrast`¶

`claude/claude-haiku-5-5`¶

`claude/claude-sonnet-5-5`¶

`claude/claude-opus-5-5`¶

`codex/gpt-6-luna`¶

`codex/gpt-5.6-terra`¶

`codex/gpt-6.1-sol`¶

`label_colour`¶

`claude/claude-haiku-5-5`¶

`claude/claude-sonnet-5-5`¶

`claude/claude-opus-5-5`¶

`codex/gpt-6-luna`¶

`codex/gpt-5.6-terra`¶

`codex/gpt-6.1-sol`¶

`large_chain`¶

`claude/claude-haiku-5-5`¶

`claude/claude-sonnet-5-5`¶

`claude/claude-opus-5-5`¶

`codex/gpt-6-luna`¶

`codex/gpt-5.6-terra`¶

`codex/gpt-6.1-sol`¶

`nesting`¶

`claude/claude-haiku-5-5`¶

`claude/claude-sonnet-5-5`¶

`claude/claude-opus-5-5`¶

`codex/gpt-6-luna`¶

`codex/gpt-5.6-terra`¶

`codex/gpt-6.1-sol`¶

`parse_repair`¶

`claude/claude-haiku-5-5`¶

`claude/claude-sonnet-5-5`¶

`claude/claude-opus-5-5`¶

`codex/gpt-6-luna`¶

`codex/gpt-5.6-terra`¶

`codex/gpt-6.1-sol`¶