An Image Sensor Node for Wireless Sensor Networks - Amazon Web ...

An Image Sensor Node for Wireless Sensor Networks Zhi-Yan Cao, Zheng-Zhou Ji, and Ming-Zeng Hu School of Computer Science and Technology, Harbin Institute of Technology, Harbin, China [email protected]

Abstract Visual information is the most intuitive information perceived by human. Image sensors are able to provide the visual information for recognition, monitoring and surveillance. Networks of visual sensors are the solution of choice for a number of societal, research, and educational applications. Architectural challenges are posed for designers of image sensor nodes such as computational power, energy consumption, energy sources, communication channels, and sensing capabilities. We discuss the characteristics and requirements for an image sensor node. An image sensor node is designed and implemented for wireless sensor network. It includes an embedded processor, memory, a CMOS image sensor, image acquisition unit, RF module and power unit etc. Power-efficient hardware management strategies are proposed for the image sensor node. Image compression problem of image sensor nodes is discussed in detail.

1. Introduction Recent advances in wireless communication, electronics, MEMS (micro-electro-mechanical systems), sensor and battery technology have made it possible to manufacture low-cost, low-power, multifunctional tiny sensor nodes. A large number of tiny sensor nodes form sensor network through wireless communication. Sensor networks represent a significant improvement over traditional sensors [1]. Research on wireless sensor network has been a topic in industrial and scientific fields. In general, a sensor node contains sensor, processing unit, wireless communication unit and power supply unit. The sensor nodes are deployed randomly in all kinds of environments, such as disaster area, combat field, industrial plants and building, to detect the interesting physical phenomena such as temperature, humidity, light, pressure, seismic or monitor the interesting events such as moving objects,

human etc. Different functional sensor nodes collect different information, acoustic and visual information can also be gathered by acoustic and image sensors. The collected information is processed by sensor nodes and transmitted to base station over the sensor network. Wireless sensor network has a potential in a wide range of application areas such as military, health, home. Visual information is the most intuitive information perceived by human. Image sensors are able to provide the visual information for recognition, monitoring and surveillance [2]. Networks of visual sensors are the solution of choice for a number of societal, research, and educational applications [3]. Including: Surveillance: Protection of large facilities (airports, plants, stadiums) requires that mechanisms for detecting and tracking intruders over large areas be put in place. If a large number of miniaturized image sensors are disseminated throughout the facilities, events may be detected and analyzed by visual processing, and visual information of interest may be transmitted to the base station. Environmental monitoring: There are many situations in which vast and inaccessible areas should be visually monitored to detect unusual events or to acquire environmental data over long periods of time. Examples include toxic locations, disaster sites, traffic control in freeways, as well as natural environments such as forests, deserts, and even planetary exploration. Several years ago, Sensor Webs [4] have been developed by JPL (Jet Propulsion Laboratory) of NASA to detected surface of Mars. Military reconnaissance: High quality images are acquired by image sensors to improve the perceptibility of command state, reliably make correct estimation about menace and track it. Many countries have developed some military scientific research on image sensors network to improve the combat and command ability of army. Medical treatment assistant: With the image sensor nodes built in patients, doctors can monitor physical

Proceedings of the International Conference on Information Technology: Coding and Computing (ITCC’05) 0-7695-2315-3/05 $ 20.00 IEEE Authorized licensed use limited to: Politecnico di Milano. Downloaded on December 9, 2009 at 11:25 from IEEE Xplore. Restrictions apply.

situation of patients. Image sensors can also assist surgeons to perform an operation. For example, television thorax mirror operation makes use of image sensors to assist surgeon to operate. Besides, operation process is also monitored through many image sensor nodes to be studied by medics. The most important design goal of wireless sensor networks is prolonging the lifetime of the whole network because of the limited power supplies. Image sensor nodes require more power processor, more energy consumption, and higher bandwidth. Architectural challenges of image sensor node are posed for designers such as computational power, energy consumption, energy sources, communication channels, and sensing capabilities. In the same time, a large number of data generated by it are transmitted to the base station. Energy-efficient image transmission and network protocol are required to resolve the limited energy, limited bandwidth problem. This paper is organized as follows. In Section 2, the characteristics and requirements of image sensor nodes are presented. In Section 3, proposed image sensor node is described in detail. Power-efficient hardware management strategies are proposed for the image sensor node. In Section 4, image compression of image sensor node is discussed in detail. Finally, Section 5 gives the conclusion.

2. Characteristics and requirements In this section, we discuss the characteristics and requirements of image sensor node. 1. Higher processing power and more memory Image acquisition and processing are compute and memory intensive tasks due to the high information content of image. For example, a 640×480 resolution Bayer pattern image with 8bits/pixel needs 300KB memory. An RGB mode image after interpolation needs 900KB memory. Histogram computation of green component of a color image needs 300K×256 arithmetic compare. In order to reduce memory requirement, image compression is necessary. However, image compression is a more compute intensive task. 2. Real-time and high communication bandwidth Real-time is a requirement for image sensor network. For example, when intruder enters the monitoring area, image sensor node must real-time gather the image, fast process and transmit it. So, high speed image acquisition and processing is needed. Due to the large amount of image data, high communication bandwidth is needed. But image sensor node has limited communication bandwidth. In order to reduce the requirement of high communication bandwidth,

image compression and partial image analysis need to be implemented at the node. Through simple image analysis, only necessary information is transmitted to the base station. 3. Energy-efficient Since energy of each image sensor node is supplied with a limited energy battery, low power design technology must be considered. Image transmission consumes more energy than computation. Energyefficient image transmission scheme is needed. Due to the multi-hop of wireless sensor network, in formation is not directly transmitted from image sensor node to the base station. Energy-efficient network routing protocol can prevent the nodes of some path from depletion due to the continually use the path, finally the lifetime of the whole network can be prolonged. 4. Robust transmission and quality of service (QoS) Due to the wireless channel is prone to be interfered by noise, the failure of information transmission often occurs. Robust transmission strategies are needed to resolve the problem. Typically, forward error coding (FEC) and automatic retransmission request (ARQ) are used as robust strategies. Consider the following scenario: In a battle environment it is crucial to locate, detect and identify a target. In order to identify a target, we should employ imaging sensors. After locating and detecting the target, those sensors can be turned on in order to identify and track that. This requires a realtime data exchange between sensors and controller in order to take the proper actions. However, dealing with real-time multimedia data requires certain bandwidth with minimum possible delay, and jitter. Therefore, a service differentiation mechanism is needed for guaranteeing the reliable delivery QoS traffic [5]. 5. Distributed processing and collaboration Each image sensor node capture the image in it own view. Due to fault consideration or densely deployment, the field of view of neighbor image sensor nodes may overlap. Collaborative processing is needed to reduce the redundant. Image sensor nodes implement object tracking task through the exchange of information between neighbor image sensor nodes. Image sensor nodes can also collaborate with other type nodes to implement some task efficiently. Besides the above the characteristics and requirements, low cost and small size also need to be considered for image sensor nodes.

3. Proposed image sensor node Based on the above the analysis of the characteristics and requirements, we design an image sensor node for wireless sensor networks. It consists of


embedded processor, memory, CMOS image sensor, image acquisition and processing circuit, RF module and power supply unit. System architecture of image sensor node is shown in Figure 1.

Figure1. System architecture of image sensor node Image sensor has two catalogs: CCD image sensor and CMOS image sensor. Due to the low power characteristics of CMOS image sensor, it is the ideal choice for image sensor nodes. We adopt LM9628 from National Semiconductor [6]. The LM9628 is a high performance, low power, 1/3” VGA CMOS Active Pixel Sensor capable of capturing color digital still or motion images and converting them to a digital data stream. In addition to the active pixel array, an onchip 12 bit A/D converter, fixed pattern noise elimination circuits, a video gain and separate color gain amplifier are provided. Furthermore, an integrated programmable smart timing and control circuit allows the user maximum flexibility in adjusting integration time, active window size, gain, and frame rate. Various control, timing and power modes are also provided. Image processing module is implemented in a field programmable gate array (FPGA) chip. It contains image acquisition, compression and processing circuits. Image acquisition circuit receives pixels from CMOS image sensor. Image processing circuit implements the interpolation of Bayer pattern image or other image processing algorithms. Image compression can be implemented in FPGA or in embedded processor. Hardware implementation of image compression can reduce the power and improve the performance. But its design cost is higher. A static random access memory (SRAM) is used as the frame memory for image processing module.

We adopt Chipcon’s CC1000 as the RF Module, Which is a true ultra-low-power single-chip RF transceiver for e.g. the 315, 433, 868, 915 MHz bands. It has been specifically designed to comply with the most stringent demands of the low power radio market. Based on a pure CMOS technology this is the first product in the market that offers a unique combination of low cost and high integration, performance and flexibility, thus setting a new standard for short-range wireless communication. It adopts Frequency Shift Key (FSK) modulation with data rates up to 76.8Kbit/s. It has an internal bit synchronizer that simplifies the design of a high speed radio link with the microcontroller. The signal interface can also be configured for a Universal Asynchronous Receiver and Transmitter (UART) serial bus interface taking benefit of the hardware UART in a microcontroller. In power down mode, the CC1000 current consumption is 0.2uA [7]. SAMSUNG’s S3C44B0X is adopted as the embedded processor of image sensor node. The 16/32bit RISC microprocessor is developed using an ARM7TDMI core, 0.25 um CMOS standard cells, and a memory compiler. Its low-power, simple, elegant and fully static design is particularly suitable for costsensitive and power sensitive applications. By providing a complete set of common system peripherals, the S3C44B0X minimizes overall system costs and eliminates the need to configure additional components [8]. In the image sensor node, S3C44B0X controls and configures the image sensor through a multi-master IIC bus controller and some general I/O ports. S3C44B0X exchanges information with RF Module through the UART interface. The S3C44B0X memory controller provides the necessary memory control signals for external memory access. A Flash memory stores the program, a synchronous dynamic random access memory (SDRAM) serves as the main memory for S3C44B0X. An important feature of S3C44B0X is its outstanding power management. It mainly consists of four modes: normal mode, slow mode, idle mode, and stop mode. The normal mode is used to supply clocks to CPU as well as all peripherals in S3C44B0X. In this case, the power consumption will be maximized when all peripherals are turned on. The user can control the operation of peripherals by software. For example, if a timer and direct memory access (DMA) are not needed, the user can disconnect the clock to the timer and DMA to reduce power. The slow mode is no Phase-Locked Loop (PLL) mode. Unlike the normal mode, the slow mode uses an external clock directly as master clock in S3C44B0X without PLL. In this case, the power consumption depends on the frequency of


the external clock only. The power consumption due to PLL itself is excluded. The idle mode disconnects the clock only to CPU core while it supplies the clock to all peripherals. By using this idle mode, power consumption due to CPU core can be reduced. Any interrupt request to CPU can wake-up from idle mode. The stop mode freezes all clocks to the CPU as well as peripherals by disabling PLL. The power consumption is only due to the leakage current in S3C44B0X, which is less than 10uA. The wake-up from stop mode can be done by external interrupt to CPU. The system state transition graph is illuminated in Figure 2. One of the work flows is described as following:

Third, CPU wakes up image sensor from power down mode. Image sensor starts to capture image and transmit it to Image Processing Module collaborating with Image acquisition circuit in Image Processing Module. Image Processing Module executes image interpolation and other simple image processing algorithms. Final, necessary image is compressed in Image Compression Circuit of Image Processing Module or in CPU. Then, the compressed image is packed into the package format required by network protocol. CPU configures CC1000 into transmission mode. Packages are transmitted to other nodes by CC1000. System returns to the receive state. Due to image sensor node also serves as routing node to use. When CPU identifies that the message are routing messages, it only switches into transmit state to transmit them. If the power is not enough, system also switches into the transmit state to transmit alarm of power not enough to the base station. The power consumption of the system is dramatically reduced in terms of the above the power management strategies.

4. Image compression

Figure2. State transition state graph of image sensor node First, CPU resets when power on, reads the program in the flash memory, and executes the initial process of CPU. Through turning off the unused block by the special register, the CPU can save up to 16.7% portion from total power consumption. We only turn on IIC, UART and Timer block. CPU configures the LM9628 through IIC bus interface. Then LM9628 enters power down mode. CPU configures CC1000 through a Timer and two I/O pin. CC1000 initializes into receiver mode. CPU enters idle mode to wait the UART interrupt. Second, if CC1000 receives the alarm messages from other sensor nodes, it will transmit the message to CPU UART interface. UART interrupt wakes up CPU from idle mode to normal mode.

Image compression is an import function of the image sensor nodes. Image compression can be implemented in hardware or software. Hardware implementation needs long design cycle, but low power can be achieved. Software implementation can be easily accomplished in the embedded processor in high flexibility. But the real-time and low power can not be satisfied. In the long term perspective, hardware implementation is the good choice for image sensor nodes. Discrete Wavelet Transform (DWT) is a state-ofthe-art image compression transform method which has been used in many applications. Nevertheless, main bottleneck seems to be its computational toughness. Lift scheme has demonstrated that the computation of DWT can be reduced of about a factor of two [9]. A further improvement can be achieved resorting to the joint benefits offered by the Integer Wavelet Transform (IWT) and the employment of filters with the integer lifting coefficients. In particular, the LeGall(5,3) filter, which is the default one for lossless compression in JPEG2000, grants very good de-correlation performance with a very simple structure. While its performance is slightly inferior to CDF(9,7), the relax computational requirements make it be an ideal choice in low energy systems. A reconfigurable IWT Intellectual Property (IP) implemented in Complex Programmable Logic Device (CPLD) has been evaluated for wireless sensor


network [10]. The reported measures show that very interesting low-energy figures can be achieved on modern CPLD. In the image sensor nodes, we also adopt the lift scheme based LeGall(5,3) wavelet transform as the transform core and implement it in FPGA. Because the image sensor nodes are basically static, the images gathered have a large amount of redundant background information. To further reduce the information transmitted, Shape-Adaptive Discrete Wavelet Transforms (SADWT) can be used [11]. A background image can be got in image sensor nodes when there are no objects. In Image Processing Module, an input mask image can be got through comparing the gathered image with the background image. The method outperforms the region-of-interest (ROI) coding in reducing the amount of data. Wavelet coefficients can be further processed by wavelet region compression algorithm. Wavelet transform based embedded image coding methods have been applied in many areas by virtue of its many advantages: resolution scalability, quality scalability, and fast codec speed etc. Embedded image coding has two folds goal: 1) obtaining the best image quality for a given bit rate, and 2) accomplishing this task in an embedded fashion, i.e. , in such a way that all encodings of the same image at lower bit rates are embedded in the beginning of the bit stream for the target rate [12]. Embedded coding method need to resolve two problems: 1) how to sort and code the wavelet coefficients in terms of importance, and implicitly include the position information of the important coefficients in the code stream, 2) how to accomplish the progressive coding of important coefficients. Problem 2) can be resolved by the bit planes coding method. Methods to resolve the problem 1) mainly consist of two catalogs: structure-based and context-based coding. Structure-based coding algorithms mainly include embedded zerotree wavelet (EZW), set partitioning in hierarchical trees (SPIHT), set partitioning embedded block (SPECK) etc [12], [13], [14]. Context-based coding algorithms include embedded block coding with optimized truncation (EBCOT), morphological representation of wavelet data (MRWD), pixels classification and sorting (PCAS) etc [15], [16], [17]. In general, context-based methods adopt the complex context model to classify the wavelet coefficients and adaptive arithmetic coding to achieve more performance than structure-based methods. But the complex algorithms can not adapted to the low power requirements for image sensor nodes. The comparison of the three algorithms can be found in [14]. We listed the performance comparison in Table 1 for discussion convenience.

Table1. Comparison of lossy coding methods for three common test images PSNR(dB) Coding method 0.25bpp 0.5bpp 1.0bpp Lena image (512 × 512 × 8bpp) EZW 33.17 36.28 39.55 SPIHT 34.11 37.21 40.44 SPECK 34.03 37.10 40.25 Barbara image (512 × 512 × 8bpp) EZW 26.77 30.53 35.14 SPIHT 27.58 31.40 36.41 SPECK 27.76 31.54 36.49 Goldhill image (512 × 512 × 8bpp) EZW 30.31 32.87 36.20 SPIHT 30.56 33.13 36.55 SPECK 30.50 33.03 36.36 Table 1 shows that SPIHT and SPECK outperform EZW. SPECK is slightly worse than SPIHT for both Lena and Goldhill, but slightly better than SPIHT for Barbara. SPIHT makes use of the correlation between the sub-bands of wavelet pyramid decomposition, while SPECK makes use of the energy convergence in the sub-bands. When the SPIHT scans the spatial orient tree to sort the coefficients, it needs to access the different sub-bands, which makes it address memory more difficult. In the same time, coefficients of different sub-bands interleave in coding stream, which makes that SPIHT is not resolution scalable and worse in resistance to Bit Error Rate (BER). SPECK has the following main advantages: 1) Progressive transmission: source samples are coded in decreasing order of their information content. 2) Low computational complexity: the algorithm is very simple, consisting mainly of comparisons, and does not require any complex computation. 3) Low dynamic memory requirements: at any given time during the coding process, only one connected region (e.g., a 32 × 32 block lying completely within a sub-band) is processed. 4) Fast encoding/decoding: this is due to the lowcomplexity nature of the algorithm, plus the fact that it can work with data that fits completely in the CPU’s fast cache memory, minimizing access to slow memory. The above features make SPECK be ideal choice for low power image compression algorithms for image sensor nodes.

5. Conclusion In this paper, we design and implement an image sensor node. It consists of embedded processor, memory, CMOS image sensor, image acquisition and processing circuit, RF module and power supply unit.


Presented efficient power management strategies make the image sensor node minimize the power consumption. We adopt LeGall(5,3) wavelet transform as the transform core of image compression in order to save the power consumption. SADWT can further reduce the data required to transmit. Several embedded image compression algorithms are evaluated for image sensor node. SPECK is an ideal choice due to its low computational complexity and low dynamic memory requirements. In the future, we will further improve our image sensor nodes. Besides, energy-efficient image transmission and network routing protocol will be our work focus.

[15] D. Taubman, “High performance scalable image compression with EBCOT”, IEEE Trans. Image Processing, vol. 9, 2000, pp. 1158-1170. [16] S. Servetto, K. Ramchandran, and M. Orchard, “Image coding based on a morphological representation of wavelet data”, IEEE Trans. Image Processing, vol. 8, 1999, pp. 11611173. [17] K.W. Peng and J.C. Kieffer, “Embedded Image Compression Based on Wavelet Pixel Classification and Sorting”, IEEE Trans. Image Processing, vol. 13, Aug. 2004, pp. 1011-1017.

6. References [1] I.F. Akyildiz, W.-L.Su, Y. Sankarasubramaniam, and E. Cayirci, “A survey on sensor networks”, IEEE Communications Magazine, vol. 40, Aug. 2002, pp. 45–50. [2] M. Wu and C.-W. Chen, “Multiple bitstream image transmission over wireless sensor networks”, in Proc. IEEE Int. Sensors, vol. 2, 2003, pp. 727–731. [3] K. Obraczka, R. Manduchi, and J.J. Garcia-Luna-Aveces, “Managing the information flow in visual sensor networks”, in Int. Symp. Wireless Personal Multimedia Communications, vol. 3, 2002, pp. 1177–1181. [4] Sensor Webs. http://sensorwebs.jpl.nasa.gov. [5] K.Akkaya, and M.Younis, “An energy-aware QoS routing protocol for wireless sensor networks”, in Proc. Int. Conf. Distributed Computing Systems Workshops, 2003, pp. 19–22. [6] National Semiconductor Data Manual, National Semiconductor Inc., 2002. [7] Chipcon Data Manual, Chipcon AS, 2002. [8] SAMSUNG Electronics Data Manual, SAMSUNG Electronics Inc.. [9] I. Daubichies and W. Sweldens, “Factoring wavelet transform into lifting steps”, Tech. Rep., Bell Lab., Lucent Tech., 1996. [10] M. Martina, G. Masera, G. Piccinini, F. Vacca, and M. Zamboni, “Embedded IWT evaluation in reconfigurable wireless sensor network”, in Int. Conf. Electronics, Circuits and Systems, vol. 3, 2002, pp. 855–858. [11] S. Li and W. Li, “Shape-Adaptive Discrete Wavelet Transforms for Arbitrarily Shaped Visual Object Coding”, IEEE Transactions on Circuits and Systems for Video Coding, vol. 10, August 2000, pp. 725-743. [12] J.M. Shapiro, “Embedded Image Coding Using Zerotrees of Wavelet Coefficients”, IEEE Trans. Signal Processing, vol. 41, 1993, pp. 3445-3462. [13] A. Said and W. Pearlman, “A new, fast and efficient image codec based on set partitioning in hierarchical trees”, IEEE Trans. Circuits and Systems for Video Technology, vol. 6, 1996, pp. 243-250. [14] A. Islam and W. A. Pearlman, “An Embedded and Efficient Low-Complexity Hierarchical Image Coder”, in Proc. of SPIE, Visual Communications and Image Processing, Vol. 3653, 1999, pp. 294-305.
