As a fundamental task in computer vision, visual object tracking has received much attention in recent years. Most studies focus on short-term visual tracking which addresses shorter videos and always-visible targets. However, long-term visual tracking is much closer to practical applications with more complicated challenges. There exists a longer duration such as minute-level or even hour-level in the long-term tracking task, and the task also needs to handle more frequent target disappearance and reappearance. The research team of Prof. Wang Dong from Dalian University of Technology provides a thorough review of long-term tracking, summarizing long-term tracking algorithms from two perspectives: framework architectures and utilization of intermediate tracking results. Then it provides a detailed description of existing benchmarks and corresponding evaluation protocols. Furthermore, it conducts extensive experiments and analyses the performance of trackers on six benchmarks. Finally, it discusses the future prospects from multiple perspectives, including algorithm design and benchmark construction. The relevant content is available at https://github.com/wangdongdut/Long-term-Visual-Tracking.
Visual object tracking is a fundamental and essential task in computer vision, and it has many practical applications, such as smart surveillance and autonomous driving and so on. Many attempts and efforts have been carried out in recent decades. Benefiting from the development of deep learning, the visual tracking field has developed quickly and achieved remarkable success. However, most existing tracking algorithms and benchmarks focus on short-term tracking, which effectively deals with the appearance and motion changes of an always visible target in a short period of time, typically 20-30 seconds. Relatively less attention has been paid to long-term tracking.
The long-term tracking task aims at tracking the specific target in videos with minute-level or even hour-level duration, which is closer to practical application. The target can suffer more sophisticated and severe challenges than in short-term tracking. Besides, the task needs to handle frequent target disappearance and reappearance in tracking scenes due to out of view or full occlusion. The re-detection ability is essential.
Several recent studies have shown that short-term trackers perform poorly on very long sequences. Short-term trackers are more likely to drift and fail in long-term scenes due to template contamination, localization error accumulation over a long time, and lack the re-detection ability to tackle the target disappearing issue. Fig. 1 visualizes some representative challenging scenes in long-term tracking. In the first row, the target disappears from the bottom of view and reappears from the top-left. In the second and third rows, the target is fully occluded by the background and reappears after occlusion from another region of view. In the fourth row, the target suffers huge appearance variations due to the changes in the angle and distances of observation.
Many works have reviewed the short-term trackers. However, although a variety of long-term tracking algorithms have been proposed, there has been no work to make a comprehensive and in-depth survey of the algorithms, evaluation benchmarks and detailed performance analysis. This work revisits existing long-term tracking algorithms from unified views and compare them on popular benchmarks. The main contributions are summarized as follows.
Comprehensive review of long-term tracking algorithms from various aspects in unified views. This paper collects existing long-term tracking algorithms and categorizes them based on two views: framework architectures and utilization of intermediate tracking results. The long-term tracking benchmarks with corresponding evaluation protocols are also described in detail.
A comprehensive evaluation of popular long-term trackers on popular benchmarks. This paper collects representative long-term trackers and evaluates them on six benchmarks for comparison. It further analyses the advantages and drawbacks of different frameworks with the speed and accuracy results of experiments.
Prospective discussion for long-term tracking. This paper discusses the potential directions for long-term tracking from views of algorithm design and benchmark construction, which may provide possible guidance to researchers.
The rest of the paper is organized as follows. Section 2 introduces the development of short-term tracking and previous relevant summative works about long-term tracking. Section 3 describes the categories of existing long-term trackers with detailed analysis. The introduction of long-term tracking benchmarks with corresponding evaluation protocols is presented in Section 4. A comparison with short-term tracking benchmarks is also analysed in this section. Section 5 reports the experimental results of representative long-term trackers on several benchmarks. Finally, it provides discussions about further directions of long-term tracking in Section 6 and conclude the paper in Section 7.
Download full text：
Long-term Visual Tracking: Review and Experimental Comparison
Chang Liu, Xiao-Fan Chen, Chun-Juan Bo, Dong Wang