A Particle Swarm Optimization approach for lower order ... - IJCIM

Comparison of Perceptual Voice Quality of VoIP from G.711 and G.729 Using Conversation-Opinion Tests

Comparison of Perceptual Voice Quality of VoIP Provided by G.711 and G.729 Using Conversation-Opinion Tests Therdpong Daengsi Faculty of Information Technology, KMUTNB, Thailand E-mail: [email protected]

Saowanit Sukparungsee Dept. of Applied Statistics, Faculty of Applied Science, KMUTNB, Thailand E-mail: [email protected]

Chai Wutiwiwatchai Human Language Technology Laboratory, NECTEC, Thailand E-mail: [email protected]

Apiruck Preechayasomboon Network Planning Department, TOT, Thailand E-mail: [email protected]

Abstract

1.

This paper reports the results from the study about perception of Thai users to the voice quality of VoIP provided by two codecs, G.711 and G.729, referring to packet loss. The conversation-opinion tests have been conducted with 250 participants to obtain the Mean Opinion Score. This study found that the user perception to G.711 and G.729 is not different significantly referring to the packet loss rates of 0%, 2%, 6%, 10% and 20% approximately. However, the trend of the difference is increased according to the increment of packet loss rates of 10% and 20% in the network. Therefore, it can be recommended to use G.729, which requires only 8 kbps for its voice payload, compared to G.711 that requires 64 kbps, particularly with packet loss of 6% or below, to reduces traffic in the IP network.

Internet is a powerful technology that becomes the backbone of many emerging services/applications. It is estimated that now there are around one billion computers connected to the Internet, as in Figure 1. The Internet has become a part of daily living for a lot of people. Besides, it has been applied to many sectors, such as, ecommerce, eLearning and telecommunications.

Keyword: VoIP, voice quality evaluation/assessment/measurement, packet loss, conversation-opinion tests.

Introduction

Figure 1 Internet Hosts by September 2011, adopted from [1] For e-commerce, the combination of traditional commerce and information and communication technologies, allows

International Journal of the Computer, the Internet and Management Vol. 20. No.1 (January-April, 2012) pp 21 -26

21

Therdpong Daengsi, Saowanit Sukparungsee, Chai Wutiwiwatchai, and Apiruck Preechayasomboon

businesses/organizations to exchange information to support sale of products and/or services electronically, based on the Internet. Amazon® is a well-known example of this which has been referred to as the number 1 online bookstore [2]. Whereas, eBay® has become the world leader in the online auction market [3]. The success of ecommerce also requires support from ebanking/Internet banking. The security issues of e-payments have also been improved and have become more trustable with such methods as Paypal® [4]. For eLearning, a kind of distance learning, educational materials can be delivered to remote students, based on Internet technology. eLearning can be real-time with an instructor and/or self learning using eLearning materials and tools, online and/or offline, analog and/or digital, and narrow cast and/or broadcast, for students who have difficulties attending traditional classes [5-7]. Therefore, they can watch online lectures/presentations and practice with interactive learning material which can respond automatically [8]. However, eLearning students may need to discuss with classmates and/or the instructor using e-mail, chatting and telephone [9]. For telecommunications, with Internet protocol (IP), Voice over IP (VoIP) becomes the modern telecommunication. It has many advantages for businesses, consumers and services providers [10]. The biggest advantage of VoIP for general users/consumers is extra-low rates for international calls. Therefore, e-commerce customers could use VoIP to contact suppliers and enquire about delayed goods or any other issues from the merchant directly if he/she did not get a reply from e-mailing. Another example would be eLearning students using Voice over IP (VoIP) to talk and discuss with classmates and/or the instructors who live in a different city/country easily. Focusing on telecommunications that are necessary to support a lot of activities, this paper presents the study of VoIP, and covers the voice quality perception of Thai

users to G.711 and G.729, referring to packet loss effects, which has been extended from [11]. 2.

Background

A. VoIP Overview [12] VoIP is based on IP network, packetbased technology, as in Figure 2. Basically, after call set-up using IP signaling such as Session Initiation Protocol (SIP), voice signals are converted into voice packets using a codec such as G.711. Then, voice packets are transported to the destination before converting back into voice signals eventually. B. Call Types of VoIP Services VoIP services, provided by operators can be classified based on the call types as follows [13-14]: 1) PC-to-PC: normally, this call type can be used free-of-charge. One of the most popular VoIP applications used widely is Skype [15]. Skype can also support video call if each PC has a webcam. Moreover, in Thailand, all major PSTN operators also provide this kind of services/application such as TOT netcall by TOT, True NetTalk by True [13-14]. 2) PC-to-phone: this call type is not free, for example, Skype users of this service must buy credit using Paypal® and/or a credit card [15]. Similar to TOT netcall, True Nettalk and CAT2call, users have to register and pay in advance, before using this kind of services [13-14, 16].

Figure 2. VoIP system overview, adopted from [14]

22


C. VoIP Limitation: Voice Quality Issue One major limitation of VoIP applications/services is voice quality when compared to traditional telephone calls provided by PSTNs. Thus, due to Internet Protocol being based on data communication, VoIP which is a real time application can be affected by factors in the IP network [12], such as: 1) Packet loss: this can be the cause of voice clipping/chopping. Therefore, it is recommended to control packet loss rates in an IP network at 3-5% maximum [17] 2) Packet delay: it is recommended that packet delay in an IP network for VoIP applications is 400 ms maximum [18].

whereas, G.711 is used over LAN. Therefore, only these two codecs are described: G.729: its MOS is 3.92. It is an 8 Kbps coding technique called Conjugate Structure - Algebraic code-excited linear prediction (CS-ACELP). Therefore, for this reason it is recommended to be used over WAN. 2) G.711: its MOS is 4.1. It is a 64 Kbps coding technique called Pulse Code Modulation (PCM). Subtypes of G.711 are G.711-law and G.711A-law. G.711-law is used in North America and Japan but G.711A-law is used in the rest of the world, including Thailand and Southeast Asia

D. Voice Quality Metric: MOS In the same situation with the same network factors, level of satisfaction to perceptual voice quality from one person may be different to another person. Therefore, to judge voice quality with accuracy and reliability, ITU-T, which is the major organization for telecommunication standardization, recommends using MOS. It stands for Mean Opinion Score. Basically, it is obtained from the judgment by a group of subjects, typically 24-32 subjects [19], using a 5-point grading scale where 5 to 1 represents excellent, good, fair, poor and bad voice quality respectively [18]. However, there are limitations to measuring voice quality using subjective methods, such as time consumption, high cost and collaboration. Therefore, many objective methods for voice quality measurement/evaluation have been developed to obtain the MOS easily [12, 20-22], for example, PESQ and E-model. The overview of measurement methods is presented in Figure 3. E. Codecs [11] 1) Codec selection can also affect voice quality. Codec is the part which changes voice signals into voice packets before transportation at the source and vice versa. Normally, G.729 is used over WAN,

Figure 3. Voice quality measurement methods, adopted from [14, 22-24]


23


3.

System, Environment, Experimental Design

and

and Figure 4. However, most MOSs from G.711 and G.729 are almost the same, particularly at the packet loss rates of 0-6%, whereas, most values of the standard deviation (SD) are about 0.5, except at the packet loss rates of 10% and 20%. Thus, these hypotheses have been investigated with the raw data from opinion scores using Ttest, a statistic method, then compared to 95% Confidence Interval (CI), as H, and then analyzation of results are presented in Table 2: H: The user perception to G.711 and G.729 is the same

The system and laboratory for this study were in the studio area at the Central Library of KMUTNB, consisting of two low background noise rooms. Conversation-opinion tests by each pair of subjects/participants per round were conducted there. The main system consists of the VoIP system using Asterisk 1.6.2, network emulator using Dummynet for packet loss generation and 2 SIP phones [11]. This study was designed to evaluate the voice quality of VoIP referring to packet loss rates of 0%, 2%, 6%, 10% and 20% approximately, with G.729 codec and G.711 (A-law) codec. Therefore, there were ten conditions to test. Following [25], each condition has been conducted with at least 24 inexperienced subjects/participants or at least 240 inexperience subjects/participants totally, who were science and technology students, Thai native speakers, in KMUTNB. They had few criteria of homogeneity which only consisted of the range of age, nationality, and the background of education (based on science and technology in the same university). The method for voice quality evaluation is conversation-opinion tests, using paper based Richard’s task, following ITU-T recommendations. The advantages of conversation-opinion tests are reaching the realism and obtaining two sets of data per round of tests [13]. However, each condition has been tested by about 30 subjects/participants due to some data classified as outliers (e.g. distinguishable high or low scores comparing to the majority), to validate the gathered data.

5. Discussion From Table 1 and Figure 4, at 10% and 20% packet loss, MOS results of G.711 are higher than G.729 obviously, compared to the packet loss of 0%, 2% and 6% which shows the MOS results of G.711 and G.729 are slightly different. Moreover, after the investigation using T-test, this study found that all p-values are higher than 0.05, although the p-value at 10% packet loss is 0.63, which is the lowest. That means there is no significance about Thai subjects/users’ perception to G.711 and G.729, within 95% CI, referring to packet loss rate of 20% maximum. 6. Conclusion and Future Work From this study, it has been found that although there is packet loss rates in the network up to 20%, the user perception of G.711 and G.729 are not significantly different. This is consistent with the previous works based on Thai users [11, 18] and this evidence confirms that G.729 can be used instead of G.711 without significant difference of voice quality perception, whereas, G.729 only requires about one third of the bandwidth used by G.711, including the packet header. However, packet loss rates are more than 20% (e.g. 30%-40%). This will be the focus of future work.

4. Result and Analysis After the conversation-opinion tests, a subjective voice quality evaluation was conducted with 24-28 subjects/participants per codecs, the result is presented in Table I

24


Acknowledgement

References

Thank you to the students and lecturers of KMUTNB who participated and supported this research. Special thanks to the Central Library of KMUTNB and the studio staff, Mr. Wiwat Suwanuntawong, for his kind support. Lastly, the first author would like to dedicate this paper to his original advisor Dr. Gareth Clayton, who sadly passed away.

[1]

Table 1. Comparison of MOS provided by G.711 and G.729 from Subjective Tests with Packet Loss Conditions, where Ns is the number of subject and SD is the standard deviation. Ns

MOS

G.729 G.711 G.729 4.13 3.78 3.50 3.04 2.42

0.54 0.51 0.50 0.56 0.72

0.54 0.42 0.58 0.62 1.02

MOS

Packet G.711 G.729 G.711 Loss 0% 26 24 4.15 2% 24 24 3.79 6% 24 26 3.58 10 % 28 24 3.34 20 % 24 26 2.79

SD

Packet Loss (%)

Figure 4 The Chart of the MOS Results from Table 1. Table 2 The Analyzed Result of Hypothesis Test Hypotheses H: 0 % packet loss H: 2 % packet loss H: 6 % packet loss H: 10 % packet loss H: 20 % packet loss Remark: p-value < 0.05 is significant

p-value 0.851 0.950 0.590 0.063 0.147

(2012) The Zakon website. [Online]. Available: http://www.zakon.org/robert/internet/ti meline/ [2] (2012) The 20 Best Books Sites Online Today on Net Top 20 [Online]. Available: http://books.nettop20.com/ [3] (2012) The 20 Best Auction Sites Online Today on Net Top 20 [Online]. Available: http://auctions.nettop20.com/ [4] (2012) The PayPal website. [Online]. Available: https://www.paypal-apac. com/th/getting-started/about-paypal. aspx?-outside. [5] F. Anaraki (2004), “Developing an Effective and Efficient eLearning Platform,” IJCIM, vol. 12(2), pp. 5763, May. [6] C. Brahmawong (2005), eLerning Courseware Production System: Underlying Principles, Major Components, and Evaluation Criteria,” in Proc. 2nd Int. Conf. on eLearning for Knowledge-Based Society, paper. 7.1. [7] A. Kamsin (2005), “Is E-Learning the Solution and Substitute for Conventional Learning?,” IJCIM, vol. 13(3), pp. 79-89, Sep. [8] A. Kok (2010), “An Activity System Perspective of E-Learning and the reframing of Knowledge,” IJCIM, vol. 18(2), pp. 17-25, May. [9] U. Jamornmann (2004), “Techniques for Assessing Students’eLearning Achivement,” IJCIM, vol. 12(2), pp. 26-31, May. [10] (2012) The International Telecommunication Union website. [Online]. Available: http://www.itu.int/itunews/manager/dis play.asp?lang=en&year=2007&issue=0 2&ipage=futureVoice2&ext=html. [11] T. Daengsi, C. Wutiwiwatchai, A. Preechayasomboon and S. Sukparungsee (2012), “A Study of VoIP Quality Evaluation: User


25


Perception of Voice Quality from G.729, G.711 and G.722,” in Proc. 9th IEEE-CCNC’2012–SS QoE, paper QoE.5, p. 358. [12] F. D. Rango, M. Tropea, P. Fazio, and S. Marano (2006), “Overview on VoIP: Subjective and Objective Measurement Methods”, IJCSNS, Vol. 6(1B), pp. 140-153, Jan. [13] (2012) The TOT Netcall website. [Online]. Available: http://www.totnetcall.com/VoIP/tdetinf o.aspx. [14] (2012) The True NetTalk website. [Online]. Available: http://www.truenettalk.com/en/about_tr uenettalk/what_is_truenettalk.html [15] (2012) The Skype website. [Online]. Available: http://www.skype.com/intl/en/home. [16] (2012) The CAT2call website. [Online]. Available: http://www.cat2call.com/index.php?opt ion=com_content&task=view&id=1&It emid=3 [17] “Quality Management: Troubleshooting Techniques for Voice over IP,” Fluke networks, NJ, 2005 [18] T. Daengsi, C. Wutiwiwatchai, A. Preechayasomboon and S. Sukparungsee, “VoIP Quality Measurement: Insignificant Voice Quality of G.711 and G.729 Codecs in Listening-Opinion Tests by Thai Users,” Inform Technol J., In Press. [19] A. W. Rix (2003), “Comparison between subjective listening quality and P.862 PESQ score,” Psytechnics, Sep. [20] M. Goudarzi (2008), “Evaluation of Voice Quality in 3G Mobile Networks,” M.S. Thesis, University of Plymouth, Plymouth, UK, Sep. [21] (2012) The ITU website. [Online]. Available: http://www.itu.int/ITUD/tech/events/2011/Moscow_ZNIIS_A pril11/Presentations/09-PomyPOLQA.pdf.

[22] S. Karapantazis and F.-N. Pavlidou (2009), “VoIP: A comprehensive survey on a promising Technology,” Comput. Netw., vol. 53(12), pp. 20502090. March.

26