QoE-Based Performance Evaluation for Adaptive Media Playout Systems

To improve the playout quality of video streaming services, several adaptive media playout (AMP) mechanisms were proposed in literature. However, all performance evaluations and comparisons for AMPs were made in terms of quality of service (QoS) metrics. As one knows, there may exist a trade-off between QoS metrics, such as buffer underflow and overflow performance. Thus, it is not sufficient to only evaluate the performance of AMPs in terms of QoS metrics. In this paper, we will evaluate and compare the performance of several AMPs from the aspect of quality of experience (QoE). Numerical results will show that some existing AMP systems do not perform better than the nonadaptive playout system from the point of view of overall QoE. 1. Introduction Recently, multimedia streaming applications such as IPTV [1] have been increasing rapidly due to the significant growth of bandwidth in access networks, such as xDSL, FTTH, 3G/4G, and WiMAX. However, due to the random delay/jitter encountered in Internet, at the client side the video playout interruption, block distortions, and nonpredictive preroll time may occur during a playout session. To counteract the effects of network jitter on the quality of video streaming, adaptive media playout (AMP) techniques, which can control the media playout rate dynamically, have emerged [2–10]. Some AMP schemes, such as [2–7], adjust the media playout rate dynamically according to the buffer fullness. Another AMP based on buffer variation rather than buffer fullness was proposed in [8]. The other content-aware AMPs that take into account the content of a video sequence and motion characteristics of different scenes were presented in [9, 10]. The content-aware AMP only slows down the low-motion scenes such that the perceived effect is lower. The quality of service (QoS) refers to several related aspects of telephony and computer networks that allow the transport of traffic with special requirements. QoS metrics belong to quantitative metrics that can be measured objectively by using network equipment. Therefore, they are usually called objective QoS metrics. Several objective QoS metrics, such as underflow probability, overflow probability, variance of distortion of playout (VDoP) [3, 7], initial playout delay, playout curve, and mean playout rate have been used for evaluating the performance of AMP mechanisms. Detailed definitions of most of these objective metrics can be found in [7]. Several works had shown that there may exist a trade-off among these metrics mentioned above [4, 7]. For example, there exists a trade-off