Dataset Label Audit for Image Datasets
Why
Bad labels, duplicates and data leakage quietly cap your model's accuracy, and a leaked test set makes your results look better than they are. This Agent finds those problems before you spend GPU hours training.
What it checks
| Priority | Problem |
|---|---|
| Critical | Train/val/test leakage (same or near-identical image in two splits) |
| Critical | Conflicting labels (same image saved under two classes) and suspected mislabels |
| Critical | Invalid class IDs, malformed label lines |
| High | Corrupt images, missing label files, zero-size or out-of-image boxes, non-normalized YOLO coordinates |
| High | Empty or tiny classes, class imbalance |
| Medium | Duplicate boxes, overlapping boxes with different classes, orphan or empty label files |
| Low | Same-split duplicates, tiny boxes, extreme aspect ratios |
Supported formats
- Classification folders:
root/<class>/*.jpg, optionally undertrain/ val/ test/ - YOLO:
images/+labels/, class names fromdata.yaml,classes.txtorobj.names - COCO: one COCO JSON plus images
Suspected mislabels
Give it a CSV of out-of-sample model probabilities (path,class1,class2,..., e.g. from k-fold cross-validation) and it ranks the images most likely to be mislabeled, confident-learning style. No predictions yet? It can write a k-fold script for PyTorch, Ultralytics or Keras.
Private by design
- Runs on your own machine. Your images are never uploaded.
- Read-only: it never moves or deletes your files. Fixes are suggestions, or a list you approve.
- Only needs Python 3.8+, numpy and Pillow. 6,000 images audited in about 30 seconds in our test.
What you get
A plain-language report in your language (English, Traditional/Simplified Chinese, Japanese, Korean, Vietnamese, Indonesian): a one-line verdict, problems ranked by severity with example files and concrete fixes, a per-split class table, and next steps. Optional fix_list.csv for large datasets.
