Optimized Cross-Layer Unequal Error Protection for Wireless Video Communication Mohd Ayyub Khan1 , Athar Ali Moinuddin1 , Ekram Khan1 1
Department of Electronics Engineering Aligarh Muslim University, Aligarh, India (Email:
[email protected],
[email protected],
[email protected])
Abstract—Error control strategies to video bitstream are usually provided at different layers of the communication network. Optimizing the error control strategies at different layers independently leads to inefficient utilization of the resources. In this paper, a cross-layer strategy to optimally utilize the application and physical layer resources for unequal error protection (UEP) of wavelet coded scalable video is investigated. We propose a cross-layer optimized UEP scheme, combining optimal forward error correction (FEC) at application layer and hierarchical QAM (HQAM) at physical layer. A look-up table-based approach, to select optimal parameter values at both layers that maximizes the overall video quality for a given channel condition, is suggested. A wide range of video sequences of different degree of motion and texture are considered to design the look-up table. The major advantage of the look-up table based approach is the low computation complexity of the transmission system, which is one of the major issue in portable video communication devices. Simulation results show the proposed scheme outperforms other techniques comprehensively over a wide range of channel conditions. Keywords—Video Coding, Video Transmission, Cross-layer, Hierarchical QAM, Forward Error Control
I.
I NTRODUCTION
Efficiently coded image/video bitstreams are extremely sensitive to channel errors and therefore may not be suitable for transmission over error prone wireless channels [1]. A variety of solutions have been proposed to cope with these challenges at different network layers (such as application, transport, media access control (MAC) and physical layers). These layers provide error resiliency in the form of Forward Error Control (FEC), Automatic Repeat Request (ARQ), joint source-channel coding (JSCC), transport prioritization, Hierarchical modulation, error concealment, etc. [1]. Generally, these solutions provide the error resiliency by independently optimising the resources accessible at individual layers only. However, in a wireless communication systems, all available resources such as bandwidth and power, together should be optimally utilized. Under such scenario, the use of cross-layer approach is one of the viable solution to optimize the resources while achieving the best quality-of-service (QoS). In recent years, cross-layer designs for robust wireless video communication have been widely investigated [2]–[5]. For example, FEC and MAC layer ARQ is used to provide cross-layer error protection for wireless local area network [2]. However, for delay sensitive video applications, MAC layer
978-1-4799-2827-9/13/$31.00 ©2013 IEEE
and Mohammed Ghanbari2 2
School of Computer Science & Electronic Engineering University of Essex Colchester, UK Email:
[email protected]
ARQ may not be suited. For such scenarios, cross-layer error protection employing application layer FEC and adaptive modulation is a viable alternative [3], [4]. In [3], a combination of turbo code and hierarchical quadrature amplitude modulation (HQAM) is used to provide unequal error protection (UEP) for two-layer scalable H.264 bitstream. Similarly, a combination of rate compatible punctured convolution code and non-uniform phase shift keying modulation is suggested in [4] to achieve UEP in H.263 layered bitstream. Most of the existing works in this area have used DCTbased video codecs and optimization is performed either using MAC layer and Application layer or Physical layer and Application layer. Due to excellent R-D performance combined with scalability feature of wavelet-based image coding algorithms, a number of wavelet-based video coders has been developed in the recent past [6]–[8] These coders generates embedded bitstream having quality scalability, in the sense that every incoming bit in a frame contribute to enhance the video quality. In each frame early generated bits are more important than later generated bits and hence can be prioritised accordingly. Only limited attempts have been made to study the effect of cross-layer optimization for such wavelet based embedded video coders [9]. The motivation of this work is to jointly optimize the cross-layer protection (application layer source and channel coding, and hierarchical modulation at physical layer) for wavelet based embedded video bitstream. In this paper, we propose a cross-layered optimized UEP scheme, combining optimal non-uniform FEC at application layer and HQAM at physical layer. A look-up table based approach to select optimal value of parameters at both layers with constraint of maximizing the overall video quality, is recommended. It is suggested that the computation for optimization should be done off-line to obtained the optimal parameters so that encoder has no burden of computing these parameters on the fly-basis for real time video communication. Due to its reduced computational complexity, the proposed system is suitable for video communication through portable multimedia devices. In the proposed scheme, compressed video bitstream is partitioned into two priority substreams: high priority (HP) and low priority (LP). At the application layer, FEC-based UEP is achieved by providing more protection to HP bits than the LP bits. At physical layer, HQAM is used to provide UEP by
1st frame
Bit Plane #
mth frame
n
n-1
1011
Last frame
1001
0001
s3 1010
n-2
1000
s1
0000
d1 Bmv
0011
s4 s2
0010
d2
Bcoef Decreasing Importance
HP bitstream
1110
1100
0100
0110
1111
1101
0101
0111
LP bitstream
Fig. 1. Organization and partitioning of video bitstream into HP and LP substreams respectively
suitably mapping the signal points on the constellation digram such that important information are more protected at the cost of less important information. The proposed scheme is tested for a wavelet-based hybrid video coding system [7]. The codec provides inherent rate scalability and ease in prioritisation of video packets for UEP purposes. The rest of the paper is organised as follows. Wavelet based video codec is described briefly in Section II. Section III discusses the single layer UEP conventionally used. The proposed jointly optimized cross-layer UEP is described in Section IV. The simulation results and discussions are presented in Section V and finally the paper is concluded in Section VI. II.
WAVELET BASED V IDEO C ODEC
The excellent performance and feature rich properties of the wavelets for image compression has motivated researchers to extend them for video coding [8]. An efficient and low bitrate wavelet video coder is proposed in [7], that uses a hybrid of temporal motion compensated prediction and the discrete wavelet transform (DWT). The key element of the video coder is the use of wavelet block-tree (WBTC) algorithm, which generates embedded bitstream. The motivation for using WBTC is that after wavelet decomposition of the residual frames, the majority of the coefficients may be grouped and then quantized to zero values giving a good compression performance. The video frames are coded using IPPPP. . . structure. For a constant bit rate (CBR) video, the I-frame is coded at a fixed bit rate of 1.0 bits/pixel (bpp) and the remaining bit budget is equally distributed among all P-frames. The number of bits (Bf ) allocated to a P-frame is divided between the motion vector bits (Bmv ) and the wavelet coefficient bits (Bcoef ). That is Bf = Bmv +Bcoef . Since WBTC is a bitplane-based coder, Bcoef bits are distributed among various bitplanes as depicted in Fig. 1. III.
U NEQUAL E RROR P ROTECTION AT I NDIVIDUAL L AYER
The layered protocols architecture for wireless/IP networks have provision of providing error protection at different layers. Further, the varying importance of bits in wavelet coded video streams suggest the use of UEP. In this paper work, the forward error control (FEC) at application layer and hierarchical QAM at physical layer are widely used to provide unequal error protection. These two schemes are briefly discussed below.
Fig. 2.
Constellation diagram of Hierarchical 16 QAM
A. UEP at Application layer Forward Error Control (FEC) at application layer is widely used to ensure a reliable transmission. In wavelet-based video coders, since different parts of video streams have different degree of sensitivity to channel errors and therefore can be prioritized accordingly. Considerable quality improvement can be achieved from the use of UEP in erroneous channels [1], while slightly increasing the system complexity. In wavelet-based embedded video coders, depending upon their sensitivity to channel errors, the bitstream of each frame, Bf , can be separated into High Priority (HP), BHP , and Low Priority (LP) , BLP , substreams respectively (as shown in Fig. 1) subject to condition that Bf = BHP + BLP . For FEC-based UEP, if HP and LP bits are coded with channel code rates rHP and rLP respectively, then the average channel code rate, ravg is defined as: ravg =
BHP + BLP BHP BLP + rHP rLP
=
rHP rLP (BHP + BLP ) rLP BHP + rHP BLP
(1)
The relative protection of HP and LP bits is measured in term of γ, defined as: r γ = LP (2) rHP The FEC based UEP can be achieved by controlling the value of γ (γ ≥ 1). Increasing the value of γ gives more protection to HP substream and relatively lesser protection to LP substream. The γ = 1 corresponds to equal error protection (EEP), which means that each substream is uniformly protected. B. UEP at Physical layer Modulation is one of the key functions performed at the physical layer in transporting information over wireless networks. The non-uniform or multi-resolution signal-space constellations can be used to provide different degrees of error protection [3] by protecting high priority bits at the cost of low Priority bits. Constellation diagram of a Hierarchical 16-QAM (16HQAM) with grey-coded mapping is shown in Fig. 2. A symbol si contains four bits bi,3 bi,2 bi,1 bi,0 , where bi,3 is MSB, bi,0 is LSB of ith symbol and i is an integer with i = 1, 2, 3 . . ..
VideoS Encoder
HPSbitSstream kMappingSbitsSb3b2 )
Channel CodingSkrHP)
LPSbitSstream kMappingSbitsSb1b0 )
Channel CodingSkrLP)
DataS Partitioning
FECS ParametersS kγ,Sravg) Cross-layer Allocator
SourceSRateS kRs)
Algorithm 1 Steps for obtaining optimized parameters, ravg , γ and α, and Look-up table
HQAM Modulator ksm =Sb3b2b1b0) Modulation ParameterSkα) ChannelSState Information
Channel
Look-upStable
VideoS Decoder
Fig. 3.
HPSbitSstream kMappingSbitsSb3b2 )
Channel Decoding
LPSbitSstream kMappingSbitsSb1b0 )
Channel Decoding
DataS Reassembly
HQAM Demodulator ksm =Sb3b2b1b0)
Cross-layer video communication system
The bits are mapped in the constellation diagram such that two most significant bits (MSBs) bi,3 bi,2 map the quadrant in which the symbol points lie with minimum inter-quadrant euclidean distance d1 whereas two least significant bits (LSBs) bi,1 bi,0 map the location of symbol points within the quadrant with their minimum euclidean distance of d2 . Two bits from HP and two bits from LP substreams are used to form a symbol. By varying the distances d1 and d2 , different degree of relative protection to HP and LP substreams can be provided. By controlling the ratio d1 α= (3) d2 known as modulation parameter (α > 1), UEP can be achieved in which HP substream gets high error protection while LP substream gets lower protection. Whereas, α = 1 corresponds to symmetrical constellation and provide equal error protection of each bits. Since HQAM does not consume any extra bandwidth to provide UEP, it can be combined with application layer FEC to achieve better UEP without any additional resources, which is the main theme of this work. IV.
O PTIMIZED C ROSS - LAYER UEP S CHEME
The idea behind proposed cross-layer UEP is to facilitate the transmission of wavelet coded video over portable devices under power and bandwidth constrained environments while achieving the best overall quality. This can be achieved by combining the application layer FEC with physical layer adaptive modulation such as HQAM. The block diagram of proposed video communication system is shown in Fig. 3. As discussed in Section III-A, at the application layer, the encoded video bitstream is split into two separate substreams: HP and LP substreams. These substreams are protected unequally by controlling γ and ravg defined in (2) and (1) respectively. The FEC coded HP and LP substreams are then multiplexed to form symbols to be mapped over asymmetrical constellations of HQAM with modulation parameter α. The cross-layer allocator gets the channel state information (CSI) and selects optimal values of γ, ravg and α from the predesigned look-up table. These values are then send to video coder and prioritised bitstreams of the application layer and physical layer to achieve optimal UEP so that the best overall video quality at constant bit rate (CBR) can be achieved. The end-to-end quality of service of video sequence transmitted
1: Initialize: CNRmin , δCNR, CNRmax , δr, δγ, γmax , δα, αmax 2: for CNR = CNRmin : δCNR: CNRmax do 3: for ravg = 0 : δr : 1 do 4: for γ = 1 : δγ : γmax do 5: for α = 1 : δα : αmax do 6: Do 20 simulation to obtain XPSNR 7: Calculate PSNRavg = PSNR 20 Simulation
8: 9: 10: 11: 12: 13: 14: 15:
end for end for end for opt Find (ravg , γopt , αopt ) = (ravg , γ, α) PSNRavg is maximum end for Repeat all the above steps for chosen test sequences opt Compute the average (ravg , γopt , αopt ) for the set of CNR opt Construct a look-up table for average (ravg , γopt , αopt ) for the CNR set
over channel is widely measured in term of Peak Signal-toNoise Ratio (PSNR). Due to random nature of the video sequences and time varying nature of wireless channels, real time calculation of optimal parameters (γ, ravg , α) for cross-layer UEP is almost impossible, even by using the fastest optimization tools. Therefore, exhausted off-line optimization steps given in Algorithm 1 is proposed. According to the algorithm, for wide range of CNR, optimum values of γ, ravg and α are to be selected which maximized the overall video quality. These parameters are then averaged for large number of test video sequences. Based upon the search results, a look-up table containing optimal parameters is to be designed that will maximise the overall video quality for each possible carrier to noise ratio (CNR) irrespective of video sequence. The proposed method will reduce computational complexity of the encoder and will be suitable to use on low power portable devices. V.
S IMULATION R ESULTS
In this section, the performance of proposed cross-layered UEP for wavelet coded video transmitted over AWGN channel is evaluated and compared with other single-layer optimized UEP. For obtaining the optimal cross-layer parameters and to design look-up table, seven video sequences, each of CIF resolution with 4:2:0 chroma sub-sampling, but of varying characteristics are considered. These sequences are Akiyo (100 frames), Flower garden (100 frames), Football (90 frames), Foreman (100 frames), Hall (100 frames), Bus (100 frames) and Mobile (100 frames). Each video is encoded by using WBTC based video codec [7], IPPP.. structure, 4 level of wavelet decomposition and half-pixel accurate motion estimation. At the application layer, FEC is applied to HP and LP substreams using (N, K) RS codes over a packet size of N = 255 bytes. The value of ravg is varied from 0 to 1. At the physical layer, HQAM with modulation parameter, α, varying from 1 to 20 is used. The simulation results are averaged over 20 independent channel conditions for each CNR varying in the range 6 to 27 dB. The objective video quality is measured in terms of average peak signal-to-noise ratio (PSNR) at 1000 kbps Constant Bit Rate channel.
40
35
35
30
30 PSNRF(dB)
PSNRF(dB)
40
25
20
15
25
20
15 QAMFonly OptimizedFHQAMFonly OptimizedFFECFwithFQAM Cross-layerFoptimizedFFECFandFHQAM
10 10
15 CNRF(dB)
20
10 25
10
(a) Foreman Sequence Fig. 4.
QAMFonly OptimizedFHQAMFonly OptimizedFFECFwithFQAM Cross-layerFoptimizedFFECFandFHQAM 15 CNRF(dB)
20
25
(b) Hall Sequence
Performance comparison of proposed cross-layer UEP.
Figs. 4 compares the performance of proposed cross-layer optimized UEP using application layer FEC and physical layer HQAM with following schemes: 1) QAM (EEP only), 2) optimized UEP using HQAM only and 3) optimized UEP using FEC only (with QAM). These comparisons are shown for two CIF sequences namely: Foreman and Hall Monitor, respectively. It can be seen from the Fig. 4(a) for Foreman sequence that optimized UEP using HQAM only provides improvement of upto approx. 13 dB than QAM only for CNR in the range of 12-24 dB. However, when optimized UEP using FEC at application layer (with QAM only) is used, a further gain of upto upto 10.5 dB (for the same range of CNR) over optimized physical layer UEP is obtained. It means that either the single layered optimised UEP is suitable for medium to high CNR only. At low CNR (below 10 dB), their performance is similar to QAM only. On the other hand, the cross-layer optimized UEP using FEC and HQAM outperforms both the single layer optimized scheme. It gives improvement of approx. 15 dB at lower CNR (6-15 dB) and same performance at higher CNR when compared ot single layer optimized UEP using FEC.
texture,etc in video communication system. It also offers low computational complexity due to off-line optimization and suitable for devices requiring low computational complexity due to limited power and limited processing capabilities. The results show that proposed system achieves the best overall performance for wide range of channel noise conditions. Although the performance of system is verified for waveletbased embedded bitstream but it can be used for any scalable bitstream. R EFERENCES [1]
[2]
[3]
[4]
Similarly, same observation can be seen for Hall Monitor sequence shown in Fig. 4(b). It is concluded form these figures that single layer optimized UEP only provides improvement for CNR in the range of 12-24 dB while at lower CNR, both the single layer optimized scheme fails to improve over simple QAM scheme, whereas cross-layer scheme gives excellent improvement at the lower CNR (6-15 dB) and provides same performance as single layer optimized UEP using FEC at higher CNR.
[5]
[6]
[7]
VI.
C ONCLUSIONS
In this paper a reliable video communication system using joint cross-layer optimized UEP for wavelet coded embedded bitstream has been proposed. The system selects optimized cross-layer parameters from pre-designed look-up table to protect two priority video streams. The look-up table can be used for different type of video sequences in term of motion,
[8]
[9]
Yao Wang and Qin-Fan Zhu, “Error control and concealment for video communication: a review,” Proceedings of the IEEE, vol. 86, no. 5, pp. 974 –997, May 1998. M. van der Schaar, S. Krishnamachari, Sunghyun Choi, and Xiaofeng Xu, “Adaptive cross-layer protection strategies for robust scalable video transmission over 802.11 WLANs,” IEEE Journal on Selected Areas in Communications, vol. 21, no. 10, pp. 1752 – 1763, dec. 2003. B. Barmada, M. M. Ghandi, E. V. Jones, and M. Ghanbari, “Combined turbo coding and hierarchical QAM for unequal error protection of H.264 coded video,” Signal Processing: Image Communication, vol. 21, no. 5, pp. 390–395, 2006. Y. Pei and J. W. Modestino, “Cross-layer design for video transmission over wireless Rician slow-fading channels using an adaptive multiresolution modulation and coding scheme,” EURASIP Journal on Advances Signal Processing, vol. 2007, pp. 1–12, 2007. A. A. Khalek, C. Caramanis, and R. W. Heath, “A cross-layer design for perceptual optimization of H.264/SVC with unequal error protection,” IEEE Journal on Selected Areas in Communications, vol. 30, no. 7, pp. 1157 –1171, Aug. 2012. Ekram Khan and Mohammed Ghanbari, “An efficient and scalable low bit-rate video coding with virtual SPIHT,” Signal Processing: Image Communication, vol. 19, no. 3, pp. 267 – 283, March 2004. A.A. Moinuddin, E. Khan, and M. Ghanbari, “Wavelet based low bit rate video coding using block-tree approach,” in IET International Conference on Visual Information Engineering, 2006. VIE 2006., Sept. 2006, pp. 287 –291. J.E. Fowler and B. Pesquet-Popescu, “An overview on wavelets in source coding, communications, and networks,” EURASIP Journal on Image and Video Processing, vol. 2007, pp. 1–27, 2007. Hyungkeuk Lee, Sanghoon Lee, and A.C. Bovik, “Cross-layer optimization for downlink wavelet video transmission,” IEEE Transactions on Multimedia, vol. 13, no. 4, pp. 813 –823, Aug. 2011.
