Report r_Qe5tYw2npublic

tx-reader 0.9.3 → 1.0.0

Claude Haiku 5.5 · 3 reps · solana-core@1.0.3 · by 4kL…aZ1m

Complete

v1.0.0 vs v0.9.3: no observed difference

18 complete pairs. 2 pairs where only v1.0.0 passed, 2 where only v0.9.3 passed, 14 ties. Observed counts at 3 repetitions — not a significance test.

Baseline44%8 / 18 executed · 18 planned

A · 0.9.367%12 / 18 executed · 18 planned

B · 1.0.067%12 / 18 executed · 18 planned

Summary and coverage

full denominator, always

Arm	Planned	Executed	Pass	Fail	Invalid	Unknown	Infra	Not run	Success
Baseline	18	18	8	9	1	—	—	—	44%
A · 0.9.3	18	18	12	3	3	—	—	—	67%
B · 1.0.0	18	18	12	5	1	—	1	—	67%

B · 1.0.0 vs A · 0.9.3

18 complete pairs · 0 excluded

2only B passed

2only A passed

14same result

ACCT-012→2 of 3±0

ACCT-023→3 of 3±0

TX-030→0 of 3±0

TX-043→3 of 3±0

PLAN-053→3 of 3±0

PLAN-061→1 of 3±0

A · 0.9.3 vs Baseline

18 complete pairs · 0 excluded

5only A passed

1only baseline passed

12same result

ACCT-011→2 of 3+1

ACCT-023→3 of 3±0

TX-031→0 of 3-1

TX-040→3 of 3+3

PLAN-053→3 of 3±0

PLAN-060→1 of 3+1

B · 1.0.0 vs Baseline

18 complete pairs · 0 excluded

5only B passed

1only baseline passed

12same result

ACCT-011→2 of 3+1

ACCT-023→3 of 3±0

TX-031→0 of 3-1

TX-040→3 of 3+3

PLAN-053→3 of 3±0

PLAN-060→1 of 3+1

Tasks and trials

click any cell for its evidence

ACCT-01Decode a Token-2022 mint with a transfer-fee extension

base1/3

A2/3

B2/3

ACCT-02Classify a frozen associated token account

base3/3

A3/3

B3/3

TX-03Interpret a v0 swap with an address lookup table

base1/3

A0/3

B0/3

TX-04Explain a failed SPL transfer

base0/3

A3/3

B3/3

PLAN-05Plan a 12.5 USDC transfer (6 decimals)

base3/3

A3/3

B3/3

PLAN-06Plan a transfer that may need a recipient ATA

base0/3

A1/3

B1/3

Trial ledger

56 attempts, in dispatch order

#	Attempt	Task	Arm	Outcome	Reason	Tokens	Cost
1	att_LL6U8EXN6T	ACCT-01 r1	A	pass	—	6,676 / 827	0.00007
2	att_vZhX5S6A7d	ACCT-01 r1	base	fail	authority_not_null	2,604 / 848	0.00004
3	att_mGvuYRvEKH	ACCT-01 r1	B	invalid_output	not_json	5,448 / 300	0.00004
4	att_71SPc78KA7	ACCT-01 r2	base	pass	—	2,878 / 391	0.00003
5	att_tLxuoAV9Z2	ACCT-01 r2	B	pass	—	5,426 / 573	0.00005
6	att_dR7fNwvJK6	ACCT-01 r2	A	pass	—	7,596 / 263	0.00006
7	att_5vZHB4zP74	ACCT-01 r3	A	invalid_output	not_json	6,809 / 686	0.00006
8	att_5FeH5bH7NA	ACCT-01 r3	base	fail	extra_field	3,099 / 700	0.00004
9	att_MyaB8GnvNo	ACCT-01 r3	B	pass	—	5,114 / 648	0.00005
10	att_31qz92BrYp	ACCT-02 r1	A	pass	—	7,427 / 331	0.00006
11	att_ZRQz5LRvd8	ACCT-02 r1	base	pass	—	3,361 / 339	0.00003
12	att_6p96NeGbTn	ACCT-02 r1	B	infrastructure_error	provider_429	—	—
13	att_E3guy1u29Xnew	ACCT-02 r1	B	pass	—	6,161 / 701	0.00006
14	att_6CNsPJzz8n	ACCT-02 r2	B	pass	—	6,156 / 555	0.00006
15	att_Es4fW1B16i	ACCT-02 r2	A	pass	—	7,253 / 282	0.00005
16	att_NaowcViBed	ACCT-02 r2	base	pass	—	2,123 / 531	0.00003
17	att_zLa5oQ2Vn3	ACCT-02 r3	A	pass	—	6,908 / 263	0.00005
18	att_rCY5iCMuhT	ACCT-02 r3	B	pass	—	5,343 / 656	0.00005
19	att_j8i8R5EufL	ACCT-02 r3	base	pass	—	2,465 / 506	0.00003
20	att_GaCK4zByPN	TX-03 r1	B	fail	instruction_order	6,353 / 516	0.00006
21	att_iBXH9q8Sv7	TX-03 r1	A	fail	instruction_order	7,635 / 574	0.00007
22	att_ZuUVcN4ShX	TX-03 r1	base	invalid_output	max_output_reached	2,002 / 2,000	0.00008
23	att_wqiyU5WZHr	TX-03 r2	B	fail	alt_unresolved	6,196 / 284	0.00005
24	att_C6KoY4xA7C	TX-03 r2	base	fail	alt_unresolved	2,694 / 861	0.00004
25	att_BPPU7mTsQD	TX-03 r2	A	fail	alt_unresolved	6,675 / 754	0.00007
26	att_cxeZnecxhT	TX-03 r3	base	pass	—	2,432 / 514	0.00003
27	att_zp7n6CbFN5	TX-03 r3	A	invalid_output	not_json	6,868 / 530	0.00006
28	att_yXWCJTDrxh	TX-03 r3	B	fail	alt_unresolved	6,455 / 867	0.00007
29	att_iPkNeQSPJ1	TX-04 r1	base	fail	wrong_index	1,941 / 529	0.00003
30	att_e873Bmikua	TX-04 r1	B	pass	—	5,331 / 452	0.00005
31	att_PyuwUFmvby	TX-04 r1	A	pass	—	7,585 / 595	0.00007
32	att_uShsCtw7mq	TX-04 r2	B	pass	—	6,326 / 486	0.00005
33	att_k9gvmpmhiT	TX-04 r2	A	pass	—	6,760 / 646	0.00006
34	att_mYJsanxJfV	TX-04 r2	base	fail	wrong_index	3,365 / 662	0.00004
35	att_ZBpaZU28hs	TX-04 r3	A	pass	—	6,297 / 408	0.00005
36	att_yxgckY3BwA	TX-04 r3	B	pass	—	6,019 / 828	0.00006
37	att_1qaBpuRJUQ	TX-04 r3	base	fail	error_guess	2,382 / 645	0.00004
38	att_avckBwxUBn	PLAN-05 r1	A	pass	—	6,359 / 523	0.00006
39	att_JKP2bUTjpS	PLAN-05 r1	base	unknown	provider_timeout	—	—
40	att_t9bnJPFz2A	PLAN-05 r1	B	pass	—	6,520 / 415	0.00005
41	att_rZ51ECgu3d	PLAN-05 r2	base	pass	—	2,707 / 414	0.00003
42	att_8Q9rzNAMFN	PLAN-05 r2	B	pass	—	6,154 / 620	0.00006
43	att_whTAEP32a4	PLAN-05 r2	A	pass	—	7,580 / 747	0.00007
44	att_sLNkuh4Ua2	PLAN-05 r3	A	pass	—	6,997 / 839	0.00007
45	att_THGZWnvKJ9	PLAN-05 r3	base	pass	—	2,551 / 856	0.00004
46	att_pS1UE3gXaY	PLAN-05 r3	B	pass	—	5,805 / 313	0.00005
47	att_2anciku7vb	PLAN-06 r1	A	invalid_output	max_output_reached	6,982 / 2,000	0.00011
48	att_kFwGE5JjXR	PLAN-06 r1	base	fail	extra_instruction	2,067 / 348	0.00002
49	att_wGJPhypeaD	PLAN-06 r1	B	pass	—	6,270 / 399	0.00005
50	att_ri6gQreKCJ	PLAN-06 r2	B	fail	extra_instruction	6,107 / 403	0.00005
51	att_d6jwMRksBV	PLAN-06 r2	base	fail	extra_instruction	2,465 / 842	0.00004
52	att_8E1kS9z7yd	PLAN-06 r2	A	fail	extra_instruction	7,486 / 360	0.00006
53	att_VjKWgMptS2	PLAN-06 r3	B	fail	non_idempotent_create	5,693 / 474	0.00005
54	att_chNbjCMrXY	PLAN-06 r3	A	pass	—	6,669 / 808	0.00007
55	att_yfxpdUKPKS	PLAN-06 r3	base	fail	extra_instruction	3,092 / 419	0.00003
56	att_LUbRoKjJFHnew	PLAN-05 r1	base	pass	—	3,115 / 701	0.00004

On chain

Status anchored

Memo tx

Slot372,433,500

Hash9d19c35a…5f324a

Cost

pricing 2026-11-01

Model usage (provider)0.00279 SOL

Runner fee0.01080 SOL

Held for unknown0.00000 SOL

Total0.0136 SOL (~$2.17)

Tokens in / out280,782 / 33,032

Provider usage is reported separately from the local estimate. Latency is the provider's, not the runner's.

Recheck

Re-score the stored answers with the trusted checker. No model is called.

Manifest

{
  "app": "skilltrial 0.9.4",
  "pack": {
    "id": "solana-core",
    "version": "1.0.3",
    "hash": "b3f1c09e7a52d84e6c1f0a9d27e5b8c4f31a06d9e2b7c58a40f6d13e9b2a7c61"
  },
  "checker": {
    "version": "checker 1.2.0",
    "hash": "4e8d2a91c0f7b36e5d18a4c92f07e63b1d5a8f2c9e04b7d61a3c8f5e2d9b0a47"
  },
  "skills": {
    "A": "tx-reader@0.9.3#5acc09ae556195f2a3932f0f97ae61d53339d267c019fd2c7c3d720130f9cdcc",
    "B": "tx-reader@1.0.0#22c77b35bf2e61bb2b600b69748a5b439373427620c24ff607658e7e30b2adbe"
  },
  "baseline": true,
  "model": {
    "id": "claude-haiku-5-5",
    "revision": "2026-10-14",
    "fingerprint": "fp_h55_3e7d41b8",
    "temperature": 0,
    "maxOutput": 2000,
    "timeoutS": 60
  },
  "systemPrompt": "tmpl_interpret_v3",
  "tools": [
    "fixture.read(id)"
  ],
  "repetitions": 3,
  "order": "arms alternated by seed",
  "seed": 17632,
  "budget": {
    "cap": 0.5,
    "currency": "SOL",
    "pricing": "2026-11-01"
  }
}