ChatGPT vs Claude Bank Reconciliation Test: 6 Planted Traps

Stress testChatGPTClaudeFictional data

We gave ChatGPT and Claude the same fictional bank file with a known answer key. Both caught all 6 planted traps and marked nothing as cleared.

Tested by Office AI Lab (IT manager) · October 23, 2026How we tested

THE RESULT1 run per tool, fresh chat

Check	ChatGPT	Claude
Duplicate payment (B-109 / B-110)	✓ Flagged	✓ Flagged
Transposed digits (K-202 / B-111)	✓ Found	✓ Found
Transposed digits (K-203 / B-112)	✓ Found	✓ Found
Transposed digits (K-215 / B-118)	✓ Found	✓ Found
One digit wrong (K-204 / B-113)	✓ Found	✓ Found
Look-alike pair, different payee (K-205 / B-114)	✓ Not paired, warned	✓ Not paired, warned
Excel completeness checks	✓ 6 of 6 OK	✓ 6 of 6 OK
Score	6 / 6	6 / 6

THE VERDICT

A tie. ChatGPT Plus and Claude Pro both caught all 6 planted traps, returned all 40 items with unchanged amounts, and marked nothing as cleared. Claude's table was easier to review because it named the matching item on every row.

What we did

1Same file40 unmatched items with 6 planted traps and an answer key built first.

2Same promptsChatGPT and Claude get the two prompts below, each in a fresh chat.

3Check in ExcelThe output is pasted into the check workbook: count, total, IDs and amounts.

4Score the trapsEach trap is scored against the answer key. Misses are logged.

Prompt 1 (grouping):
Propose a group for each of these unmatched items: outstanding check, deposit in transit, timing (same item, dates differ), possible duplicate, amount mismatch, no counterpart. These are proposals for a person to confirm. Keep the original ID and do not mark anything as cleared. Return a table with columns ID, Source, Date, Description, Amount, Proposed group.

Prompt 2 (near-match):
Compare bank and books items with the same or similar payee. A bank line that says only DEPOSIT may pair with a books deposit on the same date. Working in cents, list pairs where the amounts differ by a multiple of 9 or in exactly one digit. Label each pair "suspected", never "matched".

The evidence

Both tools got the same file and the same two prompts in a fresh chat. Screenshots are the tools’ own output, unedited.

Prompt 2: the suspected pairs

Both tools listed the same 4 pairs: three transposed-digit pairs (each difference is a multiple of 9) and one pair where a single digit is wrong. Both labelled every pair “suspected”. Neither called a pair matched.

ChatGPT Plus near-match table listing four suspected pairs with the difference in cents and the reason for each — ChatGPT Plus, Prompt 2: 4 suspected pairs, none called matched or cleared.

Claude Pro near-match table listing four suspected pairs, the test each met, and the pairs it checked and rejected — Claude Pro, Prompt 2: the same 4 pairs, plus the pairs it checked and rejected.

Prompt 1: the grouped table

Both returned all 40 items with their original IDs and amounts, flagged the duplicate Zenith entry, and left the look-alike pair (K-205 Lakeshore Cleaning and B-114 Metro Office Furniture, both $2,450) unmatched with a warning. Claude also named the counterpart for each row.

ChatGPT Plus grouped table of 40 unmatched bank and books items with a proposed group for each and nothing marked cleared — ChatGPT Plus, Prompt 1: all 40 items grouped. The Comcast pair got a group of its own.

Claude Pro grouped table, first 20 rows, naming the matching item for each timing, duplicate and amount mismatch row — Claude Pro, Prompt 1 (part 1): each row names its likely counterpart.

Claude Pro grouped table, last 20 rows, with group totals and notes on the Comcast pair and the look-alike $2,450 pair — Claude Pro, Prompt 1 (part 2): K-205 and B-114 left unmatched because the payees differ.

The Excel check

Each table was pasted into the check workbook. Both passed all six checks: same count, same signed total, no missing, repeated or extra IDs, and no changed amounts. The checks only prove nothing was lost or altered. They do not prove the groups are right.

Excel Checks tab showing 40 items in and 40 out, signed total 10,553.21 on both sides, and all six completeness checks OK — Both tools: all six completeness checks OK. The checks prove nothing was lost or changed, not that the groups are right.

WHAT WENT WRONG

Neither tool missed a planted trap.

Both broke one instruction. The prompt allowed six groups, and both put the Comcast pair (B-122 / K-219) in a group of their own because the date, amount and payee match exactly. That was a reasonable call for a match the auto-match missed, but it shows the output still needs a person.

ChatGPT's grouped table did not say which items belong together. You have to read the notes under the table to work it out. Claude named the counterpart on every row.

WHAT THIS MEANS FOR YOU

On a 40-item list, either tool gave a usable first pass at grouping unmatched items and spotting typing errors.

Ask for the original ID on every row and for "suspected" labels, then paste the result into a check workbook. Both tools kept to those rules, and that is what makes the output checkable.

Neither result is a finished reconciliation. A person still confirms each item against the source documents.

Method, limits and reruns

Question: Can the AI match 40 unmatched bank items without inventing a match? · Data: Fictional September 2026 export: 40 unmatched bank and books items, 6 planted traps, answer key built before testing. Same file as the reconciliation check workbook. · Runs: 1 run per tool, fresh chat

One fictional file of 40 items and one run per tool on October 23, 2026. Results can change when the tools update, so this test is dated and will be rerun.

Not tested: real bank exports, larger files, other accounting systems' export formats, or whether the proposed groups would hold up against source documents.

Tool	Plan	Mode	Version
ChatGPT	Plus	Regular chat, CSV attached	Oct 23, 2026
Claude	Pro	Regular chat, CSV attached	Oct 23, 2026

Run date	Plan and mode	Change
Oct 23, 2026	ChatGPT Plus, Claude Pro	First run