Quantitative and Qualitative Evaluation of Visual Tracking Algorithms using Statistical Tests

Performance evaluation of visual tracking algorithms is a complex task requiring consideration of the robustness, accuracy and failure modes of any proposed method. Both artificial and real data sets are typically employed, with quantitative, distance- based measures of tracking accuracy supported by qualitative, manual analysis of robustness and failure modes. Failure is usually taken to mean dissociation of the tracker with its target, and is identified by eye. Although distance measures are valid given artificial data, manually generated ground truth is not sufficient to allow them to be used reliably on real data sets. This paper presents an alternative evaluation methodology in which quantitative measures, in the form of statistical tests, are applied to the evaluation of both accuracy and robustness. These tests then form the basis of a tool that automatically identifies tracking failures, focusing attention on a well-defined set of events.