Influence of Compression and Network Impairments on the Picture ...

4 downloads 7921 Views 344KB Size Report
compression formats and describes the resulting picture qualities. ... Erlangen (RRZE) and the Leibniz Computing Center ... After a connection call the tele-.
Proceedings of the 35th Hawaii International Conference on System Sciences - 2002

Influence of Compression and Network Impairments on the Picture Quality of Video Transmissions in Tele-Medicine Susanne Naegele-Jackson, Dr. Peter Holleczek Regional Computing Center of the University of ErlangenNuremberg (RRZE), Germany {Susanne.Naegele-Jackson, Peter.Holleczek} @rrze.uni-erlangen.de

Dr. Thomas Rabenstein, Juergen Maiss, Prof. Dr. Eckhart G. Hahn Department of Medicine I of the University of ErlangenNuremberg, Germany {Thomas.Rabenstein, Juergen.Maiss,Eckhart.Hahn} @med1.med.uni-erlangen.de

Abstract With the continuous increase of network capacities, video transmissions in medicine are becoming an effective tool for second opinion diagnosis, archiving and teaching environments. This study investigates video compression delay times and transmission impairments for different compression formats and describes the resulting picture qualities. For the evaluation endoscopic video sequences were produced with different compression formats and bandwidth requirements. In a second step an impairment tool was used to introduce error rates to the video material to simulate network behavior. A group of medical experts evaluated the video sequences rating visibility of errors, artifacts, sharpness, overall picture quality and suitability for a medical diagnosis. The results clearly establish lower boundaries for picture quality deteriorated by compression and network impairments, and introduce limits for medical assessments.

1. Introduction In 1999 a project was started at the Department of Medicine I of the University of Erlangen-Nuremberg and the Department of Medicine II of the LudwigMaximilian-University of Munich, Germany, to promote high-resolution video transmissions between the departments of endoscopy. The focus of the project emphasizes the use of bandwidth-intensive video transmissions for second-opinion diagnoses, education and archiving. Regularly scheduled video sessions are transmitted across high bandwidth networks between the two university hospitals to evaluate the usability of video

material for medical assessments after it has been subjected to network trafficking and compression algorithms. The quality of such bi-directional video transmissions is influenced primarily by compression algorithms and network impairments causing delays or loss of information. In contrast to non-real-time services such as file transfers or email, video applications are subjected to very strong timing constraints. To obtain an uninterrupted flowing motion picture without irregularities or jerky movements the underlying network must be capable of transmitting 25 frames per second fast enough to allow the receiver to display the video at the sender’s rate. If frames are lost or arrive too late at their destination due to network congestion or transmission errors there is not enough time to retransmit the missing information and the quality of the video is severely affected. To simplify multimedia transfers and reduce their bandwidth requirements, digital audio and video signals are compressed with lossy algorithms before the network transmission and must be decoded at their arrival before the information can be properly displayed. The complex process of compression and decompression adds a significant amount of delay to the overall transmission time and the loss of information due to the encoding algorithms also affects the picture quality. The study investigates the picture quality of endoscopic video sequences using several compression formats and different amounts of bandwidth to determine the usability of the material for medical applications. In addition, impaired video streams are generated to simulate network behavior and to establish upper and lower limits for video quality affected by transmission errors. The evaluation includes delay times caused by the encoding and decoding processes of the digital video signals for

0-7695-1435-9/02 $17.00 (c) 2002 IEEE

Proceedings of the 35th Annual Hawaii International Conference on System Sciences (HICSS-35’02) 0-7695-1435-9/02 $17.00 © 2002 IEEE

Prof. Dr. Michael Sackmann Department of Medicine II of the Ludwig-MaximilianUniversity of Munich, Germany Michael.Sackmann @med2.med.uni-muenchen.de

1

Proceedings of the 35th Hawaii International Conference on System Sciences - 2002

compression. The work was completed as part of the project “Transmission of High-Resolution Video in TeleEndoscopy” between the Department of Medicine I of the University of Erlangen-Nuremberg and the Department of Medicine II of the Ludwig-Maximilian-University of Munich in Germany. The project is funded by the DFN Association (German Research Network) and BMBF (Federal Ministry of Education and Research of Germany). The paper is organized as follows: The first paragraph focuses on the technical background of the project and is followed by a description of the project’s routine video transmissions. The third paragraph discusses different types of networks and transmission impairments that have a strong impact on video transmissions. Chapter 4 examines delay measurements of two different types of codecs and compares the compression delays to the transmission delays of the signals across the networks. Chapter 5 focuses on a double blind evaluation of different picture qualities using 27 samples to determine minimal and optimal standards for video used in medical applications. The study concludes with a summary and future outlook.

2. Technical environment The video transmissions of the project “Transmission of High-Resolution Video in Tele-Endoscopy” between the two university hospitals are implemented across the Gigabit Testbed South/Berlin (GTB), a special testbed that connects the cities of Munich, Erlangen and Berlin via an exclusive optical fiber pair. A WDM (Wave Length Division Multiplexing) system is used as an optical splitter to provide three channels, where each channel offers 2.5 Gigabit/s of bandwidth over ATM (Asynchronous Transfer Mode) technology for different applications and projects. The clinic networks are linked to the testbed at the Regional Computing Center of Erlangen (RRZE) and the Leibniz Computing Center (LRZ) in Munich over 155 Megabit/s connections. Tektronix codecs (M-2 Series Video Edge Device) [17] are used in the project to reduce bandwidth requirements and study the feasibility of the application in environments beyond the GTB. Without compression the data rate of an SDI (Serial Digital Interface) signal of a full-sized picture amounts to 270 Megabit/s (625 lines with each line containing 864 pixels represented by 864 luminance and 2*432 chrominance values using 10 bit encoding and a frame rate of 25 frames per second [10]). In the project the compression format MPEG-2 [4:2:2] with a bandwidth of 40 Megabit/s and a compression delay of c. 200 ms is used for the video transmissions to obtain a full-sized picture suitable for medical applications and to ensure optimal picture quality. In an initial market analysis several different ATM codecs were

tested with the main objective to minimize compression delays. However, the ATM codecs available on the market focused only on DCT encoding techniques and did not allow an investigation of other compression methods such as wavelets, for instance, where the video signal is divided into a series of frequency bands, or lossless compression algorithms. Both departments of endoscopy set up video control areas in close proximity to the surgery rooms used for endoscopic procedures where the video signals of the different rooms are gathered and linked to a cross connect to provide easy access to all video signals from a single location. From there it is also possible to direct the video signal to student auditoriums for display during seminars or to external events such as congresses and expositions. In addition to the endoscopic cameras the surgery rooms are equipped with cameras and wireless microphones for the general orientation of the personnel to facilitate communication on both sides and to provide a total overview of the surgical area. The RRZE and LRZ computing centers provided the necessary infrastructures and implemented the network configurations. The RRZE in Erlangen also conducted codec performance tests and supported the hospitals in the acquisition of network components. During the initial phase of the transmissions the medical staff was assisted by RRZE computer personnel with trouble-shooting until routine operations could be established.

3. Tele-endoscopy in the Gigabit Testbed South As part of the project a total of 44 bi-directional video sessions have been conducted between the two university hospitals so far [14, 15]. A typical session consists of a live endoscopic examination of a patient with a duration of 8 ± 7 minutes. In some instances high-quality video tapes are also used. Doctors from either clinic may initiate the video conferences. After a connection call the teleconsultations are conducted following a fixed protocol and second-opinion diagnoses are established. During the session, the type and length of the transmitted video sequences, their picture quality and definition, but also interferences and technical problems are documented by the receiver. For quality control the video sequences are retransmitted back to the sender who is then able to compare outgoing and incoming video signals on two separate monitors. The protocol data of the session is entered into a web-based module along with laboratory results and patient information previously collected offline in preparation for the video consultation and is stored on the sender’s side. The regular video transmissions have been integrated into the daily routines of the hospitals as a weekly procedure with a fixed time slot. Establishing such a time frame where all medical experts could be available during

0-7695-1435-9/02 $17.00 (c) 2002 IEEE

Proceedings of the 35th Annual Hawaii International Conference on System Sciences (HICSS-35’02) 0-7695-1435-9/02 $17.00 © 2002 IEEE

2

Proceedings of the 35th Hawaii International Conference on System Sciences - 2002

their busy schedules proved to be one of the biggest challenges of the project. At the beginning of the project it had been intended to transmit video tapes from archives between the two locations and then meet in a video conference to discuss the material. But since the medical staff was able to establish the video connection with ease and without outside help after an initial introduction to the technical equipment, the project partners agreed on using mainly bi-directional live examinations and simultaneous discussions. The video transmissions are conducted using MPEG-2 [4:2:2] format to ensure highest video quality. The required amount of bandwidth of 40 Megabit/s for each video signal is reserved by the underlying ATM network connection and ensures the video material arrives at its destination without competition by other network traffic. The protocol data documenting the 44 routine video transmissions therefore rated the quality of the picture 27 times as “excellent” and 13 times as “good”. Four of the videos could not be used due to problems related to peripheral equipment, such as missing time-base correctors of X-ray equipment or video tape recorders. The sequences with a diminished rating of “good” could all be attributed to the fact that X-ray equipment without time-base correctors was used, which caused the picture to run initially until it stabilized after a short period of time. In all of the 40 valid video transmissions of the project the picture definition was rated as excellent and no motion artifacts or distorted movements were reported. In 36 cases there were also no picture interferences; during 4 of the transmissions slight blocking could be observed which had no bearing on the usability of the material for diagnostics. The findings [14] show that the project’s video transmissions are reliable enough with adequate picture quality to be used for tele-consultations between two hospitals to provide expert opinion to outlying areas or for use in education. Encountered problems could be reduced to secondary peripheral devices. The project is sponsored by the DFN Verein (Association of the German Research Network) and funded by the BMBF (Federal Ministry of Education and Research).

4. Impairments on high-quality video transmissions Video material used for medical assessments should be of highest quality and the transmission of the data across networks should not render the video unusable for the receiver. Parameters that may severely affect the quality of a video signal are delays or bit errors occurring during the transmission. To ensure that the strong timing constraints of multimedia data can be met the ATM transfer protocol was developed. In contrast to regular best effort IP networks, ATM technology offers the possibility for

resource allocation. Extensive Quality of Service (QoS) guarantees and traffic management functionalities are implemented that make it possible to assign the required service parameters to individual audio and video streams before their transmission and shield them from traffic competition. Calls are only accepted if enough resources are available and are rejected otherwise. To avoid congestion in network components, the Usage Parameter Control (UPC) monitors the validity of a connection. Cells are discarded if the sending rate is higher than the rate previously negotiated during the connection setup. IP networks do not offer such Quality of Service guarantees [12, 5]. Heavily congested networks may prevent data packets from arriving at the receiver too late or may have to be discarded by network components along the way. In this case the video stream cannot be displayed at the correct rate and the loss of information makes it impossible for the decoder to reconstruct the frames to their original contents. If the delay variation of the frames cannot be kept within very small limits, the steady stream of the video becomes jerky and movements lag behind. Minor loss of information is immediately noticeable as blocking in the picture [12]. Compression delay is another type of latency associated with network transmissions of multimedia data and may have a strong impact on the synchronicity of bidirectional communication. Before a valid SDI video signal with a bandwidth requirement of 270 Megabit/s is transmitted across the network, it is encoded to a compressed video stream typically requiring less than 40 Megabit/s. At its destination the compressed signal must be decoded again before display. Codecs use complex and thus very time-consuming algorithms to accomplish this task and are available for both ATM and IP traffic for different compression formats. Both IP and ATM codecs currently need between 80 ms and 400 ms for compression and decompression, depending on the device and complexity of the encoding algorithm that is used. Especially for the control of hand movements during surgical procedures it is very important to transmit synchronized audio and video signals and minimize compression delays. According to the ITU-T Recommendation G.114 [9] the one-way latency of a bidirectional transmission should not exceed 150 ms if reliable coordination and sequencing of audio and video signals of the communication between the medical staff on both sides is to be ensured. Aside from network and compression delays, impairments such as random bit errors occurring during the transmission also have an impact on picture quality. In ATM networks bit errors may occur at the physical transmission layer due to noise, interference or fading over wireless channels [7] and cells may be lost if the headers of ATM cells are corrupted [4]. Cells may also be discarded during congestion periods in ATM multiplexers or buffers if traffic behavior deviates from the originally

0-7695-1435-9/02 $17.00 (c) 2002 IEEE

Proceedings of the 35th Annual Hawaii International Conference on System Sciences (HICSS-35’02) 0-7695-1435-9/02 $17.00 © 2002 IEEE

3

Proceedings of the 35th Hawaii International Conference on System Sciences - 2002

negotiated contracts. Since the strong timing constraints of real-time data does not permit a retransmission of the lost cells, decoding errors will occur. Error detection schemes are offered at the ATM Adaptation Layer (AAL) where a compressed video stream is mapped onto ATM cells. There are several types of AALs providing different services. AAL1 for example, supports detection of cell loss and sequencing errors, but does not perform any payload integrity checks. AAL5, on the other hand, provides error detection using a CRC check on the payload data only. If errors are detected, it may discard the entire AAL PDU (AAL Protocol Data Unit). Video data discards of large PDUs could result in severe decoder errors and may be more detrimental to the overall picture quality than if no detection mechanism was in place. Without any error detection scheme at the AAL in a reliable physical transmission link with only small bit error rates of less than 10-10 most of the errors passed to the application layer may have only minor impacts on the picture quality with merely single errored blocks visible to the user (most fiber optic networks have error rates of 10-9 or better [7]). Bit error correction mechanisms on the other hand, tend to be very complex and are consequently less suitable to handle cell loss of bursty video traffic. An occurrence of bit errors during the transmission can therefore not always be avoided. Since both delays and errors may lead to severe impairments of the quality of a video, lower limits and standards must be established for video applications used for diagnostics in medicine to ensure that the transmitted data is still fully suitable for medical assessments.

5. Network and compression delays The longest delays are caused by the encoding and decoding processes of the codecs. Encoding and decoding delay times are very much determined by the equipment that is used and as such have - next to the underlying quality of the network - a strong impact on the overall quality of a transmission [13]. A pair of codecs can be fine-tuned for optimal picture quality, latency or limited use of bandwidth. All of these parameters cannot be optimized at the same time, however, since their realizations are conflicting goals. For the project in teleendoscopy the video signals are compressed into MPEG-2 [4:2:2] I-frames only with 625 lines per frame and 720 pixels per line. Whereas the setting of I-frames only favors short latencies of c. 200 ms, since complicated prediction patterns such as IBBP are avoided, bandwidth demands are high in order to preserve contrasts, color and luminance information. The ATM PVCs (Permanent Virtual Circuits) configured for the transmission of the video signals over the ATM network are implemented as CBR (Continuous Bit Rate) traffic streams with a bandwidth requirement of 50 Megabit/s to include embedded audio.

On the other hand, the transmission time for the ATM cells over the distance of 200 km from Erlangen to Munich and the same distance back to Erlangen again across the Gigabit Testbed has been measured to range only from 2.2870ms (in the case of a 0.67% workload on a STM-1 interface (1 Megabit/s)) to 2.2981ms (in the case of a 99.99% workload on a STM-1 interface (149.745Megabit/s)) and can be considered negligible in comparison to the compression delays. The cell delay variations ranged from 5.4µs to 10.9µs. The mere optical signal transmission time takes about 2ms over the round trip distance of approximately 400km and can be calculated as (Distance/Speed of optical signal over fiber) or (400km / 200*103km/s = 2ms). To compare encoding and decoding latencies for different compression formats the codecs were setup back-to-back without any network components involved (Fig. 1):

Figure 1. Test Setup for Delay Measurements As a video source a PAL sequence was used where 24 black and 24 white frames alternated in a nonstop loop. The video signal was connected to an oscilloscope (Tektronix 2220) [18] and at the same time fed into the encoder video input slot. After the encoding process the compressed signal arrived at the decoder and was subsequently transmitted to a second input channel on the oscilloscope. For observation and control the signal was also displayed on a monitor. With the original signal on one input channel and the delayed signal on the other channel the oscilloscope showed the time spent on the encoding and decoding process. Table 1 indicates delay values for different compression formats with varying complexities as far as encoding algorithms are concerned. The use of prediction in IP and IBBP algorithms increases encoding and decoding delays substantially. MPEG-1 compression latencies are also extremely high due to the fact that the digital video signal of 270 Megabit/s must be compressed to extremely low bandwidths. To optimize compression delays algorithms set to I frames only should be used and a large amount of bandwidth should be made available.

0-7695-1435-9/02 $17.00 (c) 2002 IEEE

Proceedings of the 35th Annual Hawaii International Conference on System Sciences (HICSS-35’02) 0-7695-1435-9/02 $17.00 © 2002 IEEE

4

Proceedings of the 35th Hawaii International Conference on System Sciences - 2002

Table 1. Delay measurements Codecs

Compression Format

CellStack Classic [2, 3] Tektronix M2T300 [17]

MJPEG MPEG-2 [4:2:2] MPEG-2 [4:2:2] MPEG-2 [4:2:2] MPEG-2 [4:2:0] MPEG-2 [4:2:0] MPEG-2 [4:2:0] MPEG-1 MPEG-1

GOP size (group of picture) I frames only

Bandwidth

Delay

15 Megabit/s

96 ms

I 40 (I frames Megabit/s only, GOP=1) IP 40 megabit/s (GOP=7, B=0) IBBP 40 (GOP=15) Megabit/s I 15 (I frames Megabit/s only, GOP=1) IP 15 (GOP=7, Megabit/s B=0) IBBP 15 (GOP015) Megabit/s IP 3 Megabit/s IP 1.5 Megabit/s

208 ms

230ms 375 ms 236 ms

242 ms 360 ms 318 ms 385 ms

6. Influence of compression formats and transmission impairments on the picture quality There are “lossless” and “lossy” compression algorithms. “Lossless” encoding reduces the amount of data for transmission or storage without diminishing the quality of the image. “Lossy” encoding techniques such as MJPEG, MPEG-1 or MPEG-2 on the other hand, reduce the quality of a video to meet a given target bit rate [16], but aim to still retain the optimal quality possible for the data at the specified rate. If very small target bit rates are specified the necessary compression of the data will be high and artifacts may become visible during complex video scenes containing a lot of movements and detailed information. Simple image textures and low activity may only lead to an objective degradation of the video quality that is not visible subjectively to the human eye. If video sequences are to be used for medical diagnoses, the question arises of how much image degradation is still acceptable if reliable assessments are to be reached? With the additional possible occurrence of errors across the transmission networks that may lead to a further decline of the picture quality, it is essential to both compare individual compression formats with different bandwidth specifications, but also evaluate their performance under the added influence of network impairments. To investigate these questions a study was conducted as part of the project for tele-endoscopy at the Regional Computing Center of Erlangen in connection with the Department of Medicine I of the University of ErlangenNuremberg. At the RRZE an endoscopic video sequence

of 60 seconds was varied 27 times using different compression formats and target bit rates. To simulate WAN behavior an impairment tool (InterWATCH 95000) [6] was used to generate transmission errors with variable error rates for some of the sequences. 14 experts of the centers of endoscopy then evaluated the video sequences in a double blind test according to a questionnaire prepared by the RRZE. The survey focused on picture interferences, motion artifacts, image definition, overall picture quality and an assessment if a medical diagnosis could still be considered possible with the video material in question. The tape contained sequences in MPEG-2 [4:2:2] standard ranging from 8 to 40 Megabit/s, MPEG-2 [4:2:0] format between 4 and 15 Megabit/s, MPEG-1 standards with target bit rates of 1.5 and 3 Megabit/s and MJPEG at a rate of 15 Megabit/s. Error rates ranged from 0 up to 10-6. The evaluation showed a remarkable accordance of the given assessments. Interferences as well as the lack of impairments were generally noticed correctly. The continuous decline of picture quality from the compression standard of MPEG-2 [4:2:2] from 40 to 8 Megabit/s or from MPEG-2 [4:2:0] with bandwidths from 15 to 4 Megabit/s was also observed. The optimal standard of MPEG-2 [4:2:2] at 40 Megabit/s was repeatedly recognized as optimum. The question concerning the usability of a sequence for medical diagnostics corresponded with the assessment of the picture quality: MPEG-2 [4:2:2] was rated as better suitable than MPEG-2 [4:2:0]. Motion artifacts and image definition were not always evaluated uniformly, but the assessments were generally made correctly. The subjects were also able to point out an increase of error rates from 10-8 to 10-6. The following paragraphs give a more detailed view of the findings. An abbreviated description of the chosen parameters was adopted in the charts: 422/40, for example, denotes the compression format MPEG-2 [4:2:2] with a target bit rate of 40 Megabit/s; similarly, 420/15 will stand for MPEG-2 [4:2:0] with 15 Megabit/s. J/15 describes the compression format MJPEG with a bandwidth of 15 Megabit/s, and 1/3 or 1/1.5 represent MPEG-1 format with bandwidths of 3 Megabit/s or 1.5 Megabit/s respectively.

6.1. Evaluation of the Picture Different Compression Formats

of

In the questionnaire image interferences, definition and motion artifacts were rated in separate categories. In an additional category the subjects were asked to give an assessment of the overall picture quality as a summary of the single categories. Possible answers for picture interferences were “none”, “occasionally” and “permanently”; an observation of motion artifacts could

0-7695-1435-9/02 $17.00 (c) 2002 IEEE

Proceedings of the 35th Annual Hawaii International Conference on System Sciences (HICSS-35’02) 0-7695-1435-9/02 $17.00 © 2002 IEEE

Quality

5

Proceedings of the 35th Hawaii International Conference on System Sciences - 2002

10 8 6

422/15

4

MJPEG/15

2

MPEG-1/3

420/15

420/6

MPEG-1/1.5

420/15

422/15

bad

422/40

acceptable

excellent

0

MJPEG/15 420/6 MPEG-1/3 MPEG-1/1.5

frequency of rating

422/40

12 10 8 frequency of 6 rating 4 2 0

422/15 420/15 MJPEG/15 420/5

420/15

420/4

MPEG-1/3

420/5

MJPEG/15

422/15

no

partially

420/4

yes

be rated from “none”, to “minimal”, “frequent” or “substantial”. There were only two categories to choose from for the perception of image definition comprising the values “good definition” and “blurred”. For the overall picture quality the categories “excellent”, “good”, “acceptable”, “fair” and “bad” could be chosen, in accordance with the Mean Opinion Scores suggested by the ITU-T [8]. The usability of the video material for a medical diagnosis could be answered with “yes”, “partially” or “no”. For the statistical evaluation the categories were associated with numerical values ranging from 1 through 5, and the weighted mean of the given answers was calculated. In the evaluation of the overall picture quality of video sequences without added impairments, the compression format MPEG-2 [4:2:2] was repeatedly rated as “excellent”; MPEG-2 [4:2:2] with 15 Megabit/s received a rating of “good” and was considered slightly better than the compression format MPEG-2 [4:2:0] at the same target bit rate (Fig. 2). MJPEG was rated between “good” and “acceptable”, and MPEG-1 sequences were only categorized as ranging between “acceptable” and “fair”.

MPEG-1/3

Figure 3. Evaluation of picture quality for diagnosis (without impairment) MPEG-2 [4:2:0] with 5 Megabit/s and MPEG-1 with 3 Megabit/s were clearly rated as only “partially” suitable for a diagnosis; MPEG-2 [4:2:0] with 4 Megabit/s scored close to “no diagnosis possible” with a mean of 2.71.

6.2. Evaluation of the Picture Quality of Different Compression Formats with Additional Network Impairments The video sequences were encoded and traveled from one ATM switch (FORE LE-155) to a second switch (also FORE LE-155) (Fig. 4). The connection between both network components served as a network link. Typical WAN behavior was simulated by using the impairment tool [6] which generated errors into the video streams. After decoding the resulting video sequences were displayed on a control monitor and recorded.

Figure 2. Evaluation of the overall picture quality (without impairment) Compression formats below 15 Megabit/s were given lower quality ratings: The overall picture quality of MPEG-2 [4:2:0] at 6 Megabit/s was evaluated between “good” and “acceptable” with a mean of 2.71, but was still considered worse than MJPEG with a mean of 2.43. Both MPEG-1 at a target rate of 3 Megabit/s and MPEG-1 with 1.5 Megabit/s were rated below “acceptable” and scored between MPEG-2 [4:2:0] at 5 Megabit/s and MPEG-2 [4:2:0] at 6 Megabit/s. The question relating to the suitability of the video material for medical diagnoses was mostly answered with “yes” for compression formats MPEG-2 [4:2:2] at 15 Megabit/s and above (Fig. 3), as well as for MPEG-2 [4:2:0] and MJPEG each at 15 Megabit/s.

Figure 4. Test Setup for Impairment Measurements With an error rate of 10-8 most codecs are still able to produce pictures where only single blocks occur (Fig. 5a: lower left corner, upper right corner). As soon as the errors are increased to a rate of 10-7 the blocking can be observed to cover several lines and appears more frequently (Fig. 5b) [12]. With an impairment rate of 10-6 the pictures start trembling and areas with strong

0-7695-1435-9/02 $17.00 (c) 2002 IEEE

Proceedings of the 35th Annual Hawaii International Conference on System Sciences (HICSS-35’02) 0-7695-1435-9/02 $17.00 © 2002 IEEE

6

Proceedings of the 35th Hawaii International Conference on System Sciences - 2002

12 10 frequency of 8 6 rating 4 2 0

422/40 422/15

MPEG-1/1.5

420/8

MPEG-1/3

420/15

422/15

422/40

blurred

good definition

420/15

MPEG-1/3

MPEG-1/1.5

420/15

MPEG-1/3

422/15

good

MPEG-1/1.5

bad

Figure 5c. Error rate 10-6

In the survey the introduction of an error rate of 10-8 led to a decrease in the overall picture quality assessment (Fig. 7); however, for higher bit-rate compression formats such as MPEG-2 [4:2:2] at 40 Megabit/s as well as 15 Megabit/s, and MPEG-2 [4:2:0] at 15 Megabit/s very little difference was observed concerning image definition (Fig. 6). MPEG-1 at 3 Megabit/s and 1.5 Megabit/s were considered “blurred” with and without impairments.

420/15

fair

Figure 5b. Error rate 10-7

422/15

acceptable

Figure 5a. -8 Error rate 10

12 10 frequency of 8 6 rating 4 2 0 excellent

movements turn into mosaic patterns and the motion lags behind (Fig. 5c). There may even be a trembling of the picture and the video may occasionally come to a stand still. At error rates of 10-5 the picture of most codecs freezes.

Fig. 7: Evaluation of the overall picture quality -8 with an error rate of 10 The compression format MPEG-2 [4:2:2] at 8 Megabit/s was almost always evaluated incorrectly. The errors can most likely be attributed to the fact that this video sequence filled the third slot of the sequence test (immediately following the optimal quality of MPEG-2 [4:2:2] at 40 Megabit/s and the lowest quality of MPEG-1 at 1.5 Megabit/s), where the test subjects were still in the process of establishing their individual subjective assessment ranges. Another problem could be observed with the evaluation of image interferences for the compression format MPEG-1. The deviations are probably due to the fact that the picture quality of MPEG1 is already unfavorable and interferences are not perceived as very obvious.

420/8 MPEG-1/3 MPEG-1/1.5

Figure 6. Perception of picture definition with -8 a transmission error rate of 10 With an error rate of 10-8 the sharpest decline in overall picture quality could be observed for MPEG-1 with 1.5 Megabit/s now rated between “fair” and “bad” with a mean of 4.43. Error rates of 10-8 were generally still considered as acceptable as far as reaching a medical diagnosis was concerned, but led to a degradation of the overall picture quality (Fig. 7). However, the subjects considered MPEG1 with a bandwidth of 1.5 Megabit/s and an error rate of 10-8 as no longer suitable to support a medical diagnosis. Impairments with a rate of 10-7 led to an evaluation of “diagnosis partially possible”, comparable to a bandwidth reduction of MPEG-2 [4:2:0] to 6 Megabit/s without impairments. A complete listing of the results can be found in Table 2.

7. Summary and Future Outlook The picture quality of the compression standard MPEG-2 [4:2:2] with a bandwidth of 40 Megabit/s was repeatedly considered as optimal. MPEG-2 [4:2:0] standard received lower markings, but was still considered acceptable for medical diagnostics as long as bandwidths were not reduced below 6 Megabit/s. The compression format MJPEG also received favorable ratings. Video sequences with low bandwidths under 6 Megabit/s for MPEG-2 [4:2:0] were rated as only partially suitable and completely unsuitable in the case of MPEG-1 for reaching a diagnosis. Error rates of 10-8 did not cause the video sequences to become unusable for the receiver. The ranking of MJPEG compressed video in the area between MPEG-2 [4:2:0] with 6 Megabit/s of bandwidth and MPEG-2 [4:2:0] with 7 Megabit/s of bandwidth could possibly pose a cost-effective alternative for some applications in medicine where an excellent picture quality may not be a primary issue such as in video conferencing, for instance. MJPEG codecs are available for a fraction of the costs of MPEG-2 [4:2:2] codecs; however, a signal compressed with MJPEG still does require a bandwidth of at least 15 to 20 Megabit/s for an

0-7695-1435-9/02 $17.00 (c) 2002 IEEE

Proceedings of the 35th Annual Hawaii International Conference on System Sciences (HICSS-35’02) 0-7695-1435-9/02 $17.00 © 2002 IEEE

7

Proceedings of the 35th Hawaii International Conference on System Sciences - 2002

error free video. To ensure correct sequencing in bi-directional communication codecs should be optimized to small compression delays at the cost of added bandwidth. In a future effort codec performance could even be improved further with the use of an ATM adapter: The Institute of Broadcasting Technology (Institut fuer Rundfunktechnik – IRT) in cooperation with the Fraunhofer Institute just recently developed these special video adapters that are capable of mapping a digital video signal of 270 Megabit/s directly onto ATM cells using a lossless algorithm and reducing the latency from 200 ms to a mere 350 µs [11, 12]. However, such video transmissions would require the currently still expensive upgrade of the local network components to STM-4 interfaces with bandwidth capacities of 622 Megabit/s. Although video codecs for IP networks are now also becoming available and offer comparable encoding and decoding delays compared to their ATM counterparts, the codecs are still no alternative to ATM codecs for highquality video transmissions, since network congestions may cause unpredictable delays in IP environments. To receive QoS guarantees in networks video codecs must be able to allocate resources in the networks, for example by using the Resource Reservation Protocol (RSVP) [1, 11]. However, this protocol does not offer scalability in WANs because of its complex signaling and storage of flow states in each network component and end device. Since ATM technology was especially developed for multimedia traffic and provides excellent mechanisms for bandwidth reservations and quality control, most of the high-quality video today is still transmitted over ATM networks.

8. References [1] B. Braden, D. Clark, S. Shenke, Integrated Services in the Internet Architecture: An Overview (RFC 1633) [2] CellStack decoder, Version PCB: 3i, Logic 2d, Firmware CellStack Video 1.3f over 0.7a (Master), Release: Build 588, June 18, 1997 [3] CellStack encoder, Version PCB: 3l, Logic: 2d, Firmware CellStack Video 1.4d over 0.9e (Master), Release 745, March 12, 1998 [4] Sudhir Dixit, Paul Skelly, MPEG-2 over ATM for Video Dial Tone Networks: Issues and Strategies, IEEE Network, September/October 1995, pp. 30-40

Option, GNNettest, Documentation 96103242RA, http://www.gnnettest.com [7] Steven Gringeri, Bhumip Khasnabish, Arianne Lewis, Khaled Shuaib, Roman Egorov, Bert Basch, Transmission of MPEG-2 Video Streams over ATM, IEEE Multimedia, 1998 [8] ITU-T Recommendation P.800 - Methods for Subjective Determination of Transmission Quality, 1996 [9] ITU-T Recommendation G.114 - Transmission Systems and Media. General Characteristics of International Telephone Connections and International Telephone Circuits, 1996 [10] Matthias Loeser, Untersuchungen zur Uebertragung von Echtzeitsignalen mit variabler Bitrate ueber ATM-Netze (STM4) am Beispiel von verlustfrei codierten TV-Studiosignalen (SDI) unter Einsatz einer geeigneten ATM-Adaptionsschicht (z.B. AAL5 und/oder AALx), Diplomarbeit, Universitaet Ilmenau, Germany, November 1999 [11] Andreas Metz, SDI over ATM - Hoechstqualitative Videouebertragung in Echtzeit, to be published at PEARL Workshop 2001, Echtzeitkommunikation und Ethernet/Internet, Fachtagung der GI-Fachgruppe 4.4.2 Echtzeitprogrammierung, Peter Holleczek (ed.), Boppard, Germany, Nov. 22-23, 2001, Springer 2001 [12] Susanne Naegele-Jackson, Ursula Hilgers, Peter Holleczek, Evaluation of Codec Behavior in IP and ATM Networks, 7th International Conference of EUNIS 2001, Berlin [13] Susanne Naegele-Jackson, Michael Graeve, Peter Holleczek, Spontaneity and Delay Considerations in Distributed TV Productions, 7th International Conference of EUNIS 2001, Berlin [14] T. Rabenstein, J. Maiss, S. Naegele-Jackson, K. Liebl, M. Radespiel-Troeger, R. Rosette, P. Holleczek, E.G. Hahn, M. Sackmann, Teleendoskopie im Gigabit Testbed Sued (Teilprojekt 1.15): Eine Prospektive Anwendungsstudie, 35. Jahrestagung der deutschen Gesellschaft fuer Biomedizinische Technik e.V. (DGBMT), Ruhr Universitaet Bochum, Germany, Sept. 19-21, 2001 [15] T. Rabenstein, J. Maiss, S. Naegele-Jackson, K. Liebl, M. Radespiel-Troeger, R. Rosette, P. Holleczek, E.G. Hahn, M. Sackmann, Teleendoskopie im Gigabit Testbed Sued (Teilprojekt 1.15): Einfluss von Datenkomprimierung, Bandbreite und Bildstoerungen auf die medizinischdiagnostische Beurteilbarkeit des endoskopischen Videobildes, 35. Jahrestagung der deutschen Gesellschaft fuer Biomedizinische Technik e.V. (DGBMT), Ruhr Universitaet Bochum, Germany, Sept. 19-21, 2001

[5] Ursula Hilgers, Susanne Naegele-Jackson, Peter Holleczek, Richard Hofmann, Bereitstellung von Dienstguete in IP- und ATM-Netzen als Voraussetzung fuer die Videouebertragung mit Hardware Codecs, invited paper, 15. DFN-Arbeitstagung ueber Kommunikationsnetze, Duesseldorf, Germany, June 5-7, 2001

[16] Thomas Sikora, MPEG Digital Video-Coding Standards, IEEE Signal Processing Magazine, September 1997, pp. 82-100

[6] InterWATCH 95000/96000 ATM Network Impairment

[18] Tektronix 2220 60 MHz Digital Storage Oscilloscope Rev. 1986

[17] Tektronix M2-Series Video Edge Device, Software Version 2.0.2, http://www.tektronix.com

0-7695-1435-9/02 $17.00 (c) 2002 IEEE

Proceedings of the 35th Annual Hawaii International Conference on System Sciences (HICSS-35’02) 0-7695-1435-9/02 $17.00 © 2002 IEEE

8

Proceedings of the 35th Hawaii International Conference on System Sciences - 2002

Table 2. Complete listing of the evaluation results

Interferences Motion artifacts

Quality

Diagnosis? Weighted mean

no

partiallyt yes

Weighted mean

bad fair acceptable good excellent

Weighted mean

blurred Good definition

Weighted mean

substantially

frequently minimal none

Weighted mean

permanently occasionally none

Seq. no.

Definition

Without impairments 422/40 422/40 422/30 422/15 422/10 422/8

1 27 22 18 23 3

11 13 10 10 12 14

2 0 1.15 12 2 1 1.07 8 6 4 1.29 7 6 4 1.29 8 5 2 1.14 6 8 1 12 2

420/15 420/10 420/8 420/7 420/6 420/5 420/4

6 21 26 16 25 20 14

10 10 9 10 8 9 9

3 1 1.36 6 6 1 1 1.57 3 1 1.36 7 4 3 1.71 5 1.36 8 2 4 1.71 4 1.29 8 4 2 1.57 6 1.43 2 9 3 2.07 5 1.36 7 5 2 1.36 4 1 1.43 7 3 2 1 1.54

J/15

10 11 2 1 1.29 4 7 2

1/3 1/1.5

0 0 1.14 14 0 0 1.43 10 3 1 1.57 9 4 1 1.5 12 2 1.57 13 1 1.14 14 11 9 9 7 4 2

3 5 5 7 10 12 14

1 1.23 1.31 1.14 1.07 1

10 7 2 6 5 9

3 6 8 4 8 5

1 0 0 1.36 13 1 0 1.07 1 1.57 13 1 1.07 4 2.14 12 2 1.14 2 2 2 11 3 1.21 1 1.71 12 2 1.14 0 1.36 13 1 1.07

1.21 1.36 1.36 1.5 1.71 1.86 2

3 3 4 2

8 6 6 7 6 4

2 4 4 3 6 2 2

1.85 9 5 1.36

1 2.14 12 1 0 2.21 9 2 10 2 2.36 10 2 2.71 10 7 1 3.36 3 2 10 4.57

9 4 1

2 1.14 5 1.36 4 1.29 4 1.29 4 1.29 7 4 2.07 4 10 2.71

2.43 11 3

1.21

8 9 4 1 1.43 2 8 3 1 2 14 2 3 4 6 1 3.36 1 12 1 2 2 8 2 3 1.62 3 4 6 0 2.23 1 12 1.92 0 1 5 6 2 3.64 1 11 2 2.07

With impairments 422/40 422/40/-8 422/40/-7 422/40/-6

1 11 2 0 1.15 12 2 0 0 4 14 2 5 7 1 13 1 3 10 2.64 1 3 7 3 7 0 1 13 2.93 6 8

1.14 14 0 1 10 1.69 13 1 1 2.21 11 2 1.15 1.86 2 12 1.86

422/15 422/15/-8 422/15/-7

18 10 4 1.29 8 5 1 11 4 9 1 1.79 10 4 24 6 8 2.57 3 11

1.5 12 2 1.14 6 4 2 2 1.29 11 2 1.15 3 8 3 2.79 7 7 1.5 5 9

2 11 3 2 12 2 3.64 3 11

1.21 1.14 1.79

420/15 420/15/-8 420/15/-7

6 10 3 1 1.36 6 6 1 1 1.57 11 3 1.21 3 8 2 0 1 2.14 12 2 15 6 8 1.57 6 8 1.57 12 2 1.14 2 7 5 2.21 10 4 17 2 8 4 2.14 1 8 4 2.23 8 5 1.38 2 6 6 3.27 4 10

1.14 1.29 1.71

420/8 420/8/-8 420/8/-7

26 9 5 1.36 8 2 4 12 3 11 1.79 7 6 1 5 4 10 2.71 9 3

1.29 1.29 1.43

1/3 1/3/-8 1/1.5 1/1.5/-8

3 1 0 0 1.36 13 8 5 0 2.29 12 2 4 6 2 3.57 7 1 2 10 4.62 2

1.71 9 5 1.36 4 6 4 1.57 8 6 1.43 3 6 4 1 2.25 12 1 1.08 2 9 3

8 9 4 1 1.43 2 8 3 1 2 14 2 9 7 7 1.5 3 9 1 1 1.79 3 11 1.79

1 0 1.07 2 1.14 6 1 1.57 8 4 2.14

2 10 4 2.21 10 4 3.07 8 6

3 4 6 1 3.36 1 12 1 2 6 1 6 3 5 9 1.64

2 8 2 3 1.62 3 4 6 0 2.23 1 12 1.92 0 1 5 6 2 3.64 1 11 2 2.07 19 9 2 3 1.57 0 8 4 2 2.14 2 12 1.86 1 6 7 4.43 8 6 2.43

0-7695-1435-9/02 $17.00 (c) 2002 IEEE

Proceedings of the 35th Annual Hawaii International Conference on System Sciences (HICSS-35’02) 0-7695-1435-9/02 $17.00 © 2002 IEEE

9