Skip to main navigation menu Skip to main content Skip to site footer

Articles

Vol. 5 No. 1 (2026): Beyond Algorithms: The New Era of AI

Rediscovering Laboratory Rules Rather Than Predicting Disease: An Empirical Audit of Machine Learning on a Public Heart-Attack Dataset

Submitted
October 29, 2026
Published
2026-10-29

Abstract

A public heart-attack dataset of 1,319 patients from Erbil, Iraq, is widely used to benchmark machine-learning (ML) clas- sifiers, with reported accuracies of 96–99%. We asked whether these figures reflect clinical prediction or the reconstruction of a labelling rule. After excluding 14 physiologically impossible records, 1,305 records were split once into a develop- ment set, designated DEV, comprising 1,044 records, and a locked test set, designated TEST, comprising 261 records. On DEV, we first searched for a deterministic labelling rule, next re-ran four published pipelines over 100 random splits with data-dependent preprocessing fitted before or after splitting, and finally compared ten model families across seven predictor sets by nested cross-validation. Final models, thresholds and four comparisons were frozen before a single TEST evaluation. The label was reproduced by a two-assay rule: troponin > 0.014 ng/mL or CK-MB above sex-specific limits of 4.88 ng/mL for women and 6.22 ng/mL for men, values that match the published reference limits of one commercial assay family. On TEST this rule misclassified 2 of 261 records and the best random forest misclassified 1, with an exact McNemar p= 1.00. The sex-specific rule made fewer errors than a generic CK-MB limit of 5.0 ng/mL, yielding 2 versus 9 errors with p= 0.016. Fitting preprocessing before splitting changed mean accuracy by less than 0.4 percentage points, far less than split-to-split variation. Without troponin and CK-MB, AUROC was 0.61–0.64 and predicted probabilities were miscalibrated. Near-perfect accuracy on this dataset measures how well a model rediscovers a laboratory labelling rule, not its ability to predict myocardial infarction.