Smarter Sampling: Data Prioritization for Improved Object Detection

Abstract

The effectiveness of deep learning models is strongly influenced by the quality of training data. Traditional training approaches assume that all samples contribute equally to the learning process, leading to uniform data sampling. However, this assumption does not account for the varying informational content of different samples. This paper presents a novel data prioritization method for object detection that dynamically adjusts the sampling probability of training data based on its relevance. Focusing on object detection applications within the computer vision field, the proposed methodology introduces the Relative Detection Error metric to evaluate and prioritize samples during training. By selecting data points with higher informational value, our approach improves both classification and localization accuracy while maintaining minimal computational overhead. The method is demonstrated using YOLO architectures on diverse datasets, showcasing its ability to generalize across different settings. Experimental results indicate that prioritizing high-value samples enhances the F1 score and mean Average Precision (mAP), leading to more efficient training and robust performance.

Relative Detection Error

The primary goal of the proposed method is to assess the informational value of training samples based on their difficulty, which is determined by network uncertainty. Prioritized sampling focuses on the most uncertain data points, where the model struggles to make clear distinctions. These training samples are likely located near a decision boundary, causing the neural network to frequently produce incorrect predictions while trying to establish the exact decision boundary. The method's main objective is to identify and adequately emphasize such data points without overfitting.

Localization Error

The Localization Change value reflects the localization uncertainty of predictions for a given training sample. The calculation compares the bounding boxes from the previous and current predictions to determine positional differences. Specifically, the Intersection over Union (IoU) values of said predictions are utilized to measure overlap. Based on the IoU matrix, bounding boxes are matched if their IoU exceeds a predefined threshold. Localization Change shows the number of bounding boxes with IoU values below this threshold for all samples separately, highlighting instances where no corresponding bounding box was detected in the previous predictions.

Class Change Error

The other component of the overall error term, Class Change, is responsible for characterizing the classification uncertainty of predictions for a given training sample. Similar to the localization metric, the object classes in the previous and current predictions are compared, and discrepancies between them are used to determine the Class Change value. This value corresponds to the number of objects for which the predicted class varies between the current and previous iterations, serving as an indicator of classification instability.

Results

Convergence

The blue and black lines depict the average F1 and mAP50 values, while the shaded areas represent the corresponding standard deviation range. Prioritized sampling achieves better and slightly faster convergence, furthermore, it shows less variance between different seeds on the Pascal VOC dataset. The lower variance and faster convergence indicate that data prioritization contributed to a more stable learning process. A reduced variance suggests that model performance exhibits fewer fluctuations across epochs, resulting in a more predictable and reliable convergence trajectory. By enabling more efficient learning of complex features, data prioritization enhances both the consistency of the learning process and the model’s generalization ability. Moreover, the diminished performance variance implies decreased sensitivity to variations in training samples, leading to more robust and adaptable learning outcomes.

F1 and mAP50 Gain

F1 Score Comparison

Sampling	Dataset	YOLOv8	YOLOv5
Uniform	KITTI	81.18%	69.12%
Uniform	Pascal VOC	48.05%	49.41%
Prioritized	KITTI	81.63%	69.52%
Prioritized	Pascal VOC	49.24%	50.38%
F1 Gain	KITTI	+0.451%	+0.393%
F1 Gain	Pascal VOC	+1.184%	+0.965%

mAP50 Comparison

Sampling	Dataset	YOLOv8	YOLOv5
Uniform	KITTI	84.29%	73.06%
Uniform	Pascal VOC	54.90%	52.71%
Prioritized	KITTI	84.72%	73.48%
Prioritized	Pascal VOC	55.00%	53.60%
mAP50 Gain	KITTI	+0.429%	+0.418%
mAP50 Gain	Pascal VOC	+0.107%	+0.89%

Citation

Please use this BibTeX in case you would like to cite our work in your publications:

@article{,
  author    = {Balogh, Csan{\'a}d L. and Pap, Bence Szil{\'a}rd and K{\H o}v{\'a}ri, B{\'a}lint and B{\'e}csi, Tam{\'a}s},
  title     = {Smarter Sampling: Data Prioritization for Improved Object Detection},
  journal   = {},
  volume    = {},
  number    = {},
  pages     = {},
  year      = {},
}