69.2% fewer executions · measured ground truth
Graph intelligence / evaluation 001
Does the graph pick the tests that expose the bug?
Every mutation runs alone. The complete pytest suite establishes ground truth. Then filename matching, direct callers, and IMPACT compete to select the failure-revealing tests.
15controlled mutations
24tests in the fixture
29/29failures found by IMPACT
69.2%fewer test executions
Three evidence cases
No reliable full-suite comparison; recall not measured
34 passed · 9 remote-service cases skipped · 0 failed
Real-PR evidence is not controlled ground truth. The Requests
full suite was not run. HTTPie's full-suite run was environment-blocked,
so neither real PR receives a failure-recall percentage. Paths show
possible impact; observed test statuses are reported separately.
Selection methods
80.0% fewer runs
91.4% fewer runs
69.2% fewer runs
0.0% fewer runs
Read this precisely. Failure recall counts tests that failed or
errored in the full run and were also selected. A selected passing test
can still cover relevant behavior, so failure yield is not general
relevance precision.
Every mutation
| Injected defect | Full-suite failures | IMPACT selected | Failure recall |
|---|---|---|---|
coupon_cap_removedAn oversized coupon is no longer capped at the subtotal.
|
4 | 11 / 24 | 100% |
negative_coupon_acceptedA negative coupon increases the amount charged.
|
1 | 11 / 24 | 100% |
quantity_ignoredLine pricing ignores item quantity.
|
6 | 10 / 24 | 100% |
zero_quantity_chargedAn empty line item is charged as one item.
|
1 | 10 / 24 | 100% |
negative_quantity_allowedA quantity of minus one bypasses validation.
|
1 | 10 / 24 | 100% |
percentage_rounding_changedPercentage discounts round down instead of to the nearest
cent.
|
1 | 8 / 24 | 100% |
combined_discount_droppedOrder pricing silently ignores percentage discounts.
|
1 | 7 / 24 | 100% |
tax_rounding_changedTax rounds down instead of to the nearest cent.
|
1 | 8 / 24 | 100% |
shipping_threshold_exclusiveOrders exactly at the free-shipping threshold are
charged.
|
1 | 9 / 24 | 100% |
shipping_fee_increasedPaid shipping changes without its callers being updated.
|
5 | 9 / 24 | 100% |
exact_stock_rejectedAn order using the final unit of stock is rejected.
|
1 | 5 / 24 | 100% |
reserve_does_not_decrementA successful reservation leaves inventory unchanged.
|
2 | 4 / 24 | 100% |
tax_omitted_from_orderCheckout omits tax from the order total.
|
2 | 6 / 24 | 100% |
receipt_decimal_shiftReceipt formatting moves the decimal point.
|
1 | 1 / 24 | 100% |
refund_cap_removedRefunds can exceed the amount originally paid.
|
1 | 2 / 24 | 100% |