In the field of automatic target recognition and tracking, traditional image complexity metrics, such as statistical variance and signal-to-noise ratio, all focus on single-frame images. However, there are few researches about the complexity of image sequence. To solve this problem, a criterion of evaluating image sequence complexity is proposed. Firstly, to characterize this criterion quantitatively, two metrics for measuring the complexity of image sequence, namely feature space similarity degree of global background (FSSDGB) and feature space occultation degree of local background (FSODLB) are developed. Here, FSSDGB reflects the ability of global background to introduce false alarms based on feature space, and FSODLB represents the difference between target and local background based on feature space. Secondly, the feature space is optimized by the grey relational method and relevant features are removed so that FSSDGB and FSODLB are more reasonable to establish complexity of single-frame images. Finally, the image sequence complexity is not a linear sum of the single-frame image complexity. Target tracking errors often occur in high-complexity images and the tracking effect of low-complexity images is very well. The nonlinear transformation based on median (NTM) is proposed to construct complexity of image sequence. The experimental results show that the proposed metric is more valid than other metrics, such as sequence correlation (SC) and interframe change degree (IFCD), and it is highly relevant to the actual performance of automatic target tracking algorithms.