2012 IEEE Second International Conference on Consumer Electronics - Berlin (ICCE-Berlin)
CQR CODES: COLORED QUICK-RESPONSE CODES Max E. Vizcarra Melgar1 , Alexandre Zaghetto2 , Bruno Macchiavello2 , Anderson C. A. Nascimento1 Universidade de Bras´ılia Dept. of Electrical Engineering and 2 Dept. of Computer Science E-mail:
[email protected], {zaghetto, bruno}@image.unb.br,
[email protected] 1
ABSTRACT This paper proposes a new way to store/transmit information using a Colored QR Code structure. Instead of using only black and white modules, the proposed code is designed to employ 5 different RGB colors (red, green, blue, black and white), which enables twice as much storage capacity compared to traditional binary QR Codes. Reed-Solomon error-correcting code with a theoretical correction capability of 38.41% is also applied. In our experiments, each Colored QR Code in the test set is printed, scanned using a 3.2 megapixel digital camera and decoded. We show that the proposed scheme can consistently decode 1024 bits of information stored on a 1.3 cm × 1.3 cm printed area. Index Terms— Colored QR Codes, Error-correcting Codes, Reed-Solomon, Image Processing. I. INTRODUCTION QR Code, proposed in 1994 by the japanese company Denso Wave Incorporated and approved in 2000 as the ISO/IEC 18004 standard [1], is a two-dimensional structure used to transmit information through a print-scan channel. QR stands for Quick Response since it is intended to be decoded at high speed. Conceptually, the idea behind this technology is not much different from linear or matrix barcode [2], [3], but its superior data density allied with high speed reading made it very popular along the past few years. Nowadays, it is being used in advertisements, business cards, storefront displays, t-shirts, letter stamps, movie posters, product labels etc. Most applications link the viewer to additional on-line content through a digital snapshot of the printed code. Typically, a QR Code is composed by black and white modules, as illustrated in Figure 1. In this paper we propose the use of colored modules in order to increase data density. The proposed structure is capable of transmitting 1024 bits of information plus 3392 bits of Reed-Solomon redundancy [4]. It is meant to create a channel that enables, for instance, the transmission of 1024-bit digital signatures or symmetric and asymmetric cryptography [5]. The proposed Colored QR Code, which from now on will be reffered to as CQR Code, uses 4 different information modules (red, green, blue and white). Black modules are
978-1-4673-1547-0/12/$31.00 ©2012 IEEE
Fig. 1. Example of QR Code, typically composed by black and white modules. used only for alignment purposes. Since each information module represents 2 bits, the new structure is capable of storing twice as much information as traditional black and white codes. In the next section we describe the CQR Code. II. COLORED QR CODE The proposed CQR Code is made up of 49 × 49 modules. A module is defined as a colored square area that represents data/redundancy bits (00 - red, 01 - green, 10 - blue, 11 white) or function patterns (black over white background). The size of each module is n × n pixels, depending on the desired overall CQR Code size. Modules are distributed over two discriminate regions, the function patterns and the encoding region. The function patterns have exactly the same aspect for all CQR Codes and are divided into: (a) quite zone; (b) finder patterns; and (c) separators. The quite zone is the 4module white area that surrounds the code on all four sides. Finder patterns are the three identical symbols located at the upper-left, upper-right and lower-left corners of the code and are used for correct image positioning at the decoder. Separators are the 1 × 8 or 8 × 1-module elements that separates encoding region and finder patterns. The encoding region contains information and redundancy bits. Figure 2 illustrates the above described CQR Code elements. From the 2401 (49 × 49) modules, 192 are inside the standard region and 2209 are inside the encoding region. Since each module represents 2 bits, we have 4418 bits available, from which 1024 are information bits and 3392 are redundancy bits. Two bits (1 module) are left unused
BLUE (0,0,255) 10
WHITE (255,255,255) 11
BLACK (0,0,0)
GREEN (0,255,0) 01
RED (255,0,0) 00
Fig. 2. The CQR Code structure. Modules are distributed over function patterns and encoding region. because of the characteristics of the Reed-Solomon errorcorrecting code, described in Section III. III. ERROR-CORRECTING CODE Error-correcting codes are widely used in communication systems. They are meant to prevent transmission errors by adding extra information (redundancy) that cross-checks the original data [4]. Since the print-scan process may be interpreted as part of a communication system (Figure 3), the information transmitted through a CQR Code structure may be susceptible to errors caused by the noisy channel.
Fig. 4. A CQR Code module is defined as a colored square area that represents data/redundancy bits (00 - red, 01 green, 10 - blue, 11 - white) or function patterns (black over white background). Equation 1. RS(276, 64) = [D1 · · · D64 RS1 · · · RS212 ].
(1)
The primitive polynomial used in the generation of redundancy bits is shown in Equation 2. p(D) = D16 + D12 + D3 + 1.
(2)
A Reed-Solomon decoder can correct up to t symbols that contain errors in a codeword. Equation 3 shows how t is determined. t = (n − k)/2. (3) In our case, t is 106, which means that even if 38.41% of the symbols are degenerated, it is possible to recover all the original 64 data symbols. In effect, this is the theoretical maximum percentage of allowed distortion for correct decoding of the codeword. IV. ENCODING AND DECODING INFORMATION USING CQR CODES
Fig. 3. The print-scan process may be interpreted as part of a communication system. There is a wide range of codes that fit to this purpose. Here we use the Reed-Solomon error-correcting code [4] to control transmission errors. Adapted to the proposed CQR Code, the encoding procedure consists in creating a single Galois Array which takes 64 symbols of 16 bits each (data bits or Di , i = 1 · · · 64) as the input and outputs 212 additional symbols of 16 bits each (redundancy bits or RSj , j = 1 · · · 212). These 16 data or redundancy bits are stored on 8 consecutive CQR Code modules of 2 bits. The Reed-Solomon process takes k data symbols of s bits and adds redundancy symbols to make an n symbol codeword. In our case, k = 64, s = 16 and n = 276. The resulting codeword in the Galois Array, RS(n, k), is shown
The proposed CQR Code has the same size as the traditional QR Code version 8 (49 × 49 modules). The difference is that now each module is represented by one out of five possible colors taken from the 24-bit RGB color space [6], as illustrated in Figure 4. The correspondence between colors and binary codes, presented in Section II, is also shown. Red, green, blue and white colors are chosen because of their maximum equidistance on the RGB color space. This facilitates color thresholding on the decoding step. The positioning of the modules in the encoding region is vertical bottom-up, from the most right to the most left column, as shown in Figure 5. The encoding process is this simple: first we binarize the information that must be transmitted. Considering that each group of 16 bits represents one symbol, the redundancy symbols are generated using the Reed-Solomon algorithm and, then, combined with the original data symbols. Each
Fig. 5. The positioning of the modules in the encoding region is vertical bottom-up, from the most right to the most left column. group of two bits is mapped into one of the 4 possible colors. Finally, the modules are organized as illustrated in Figure 5. Figure 6 shows an example of synthesized Colored QR Code. This example will be used throughout the paper.
Fig. 8. Typical histogram used to calculate the threshold parameter th. The decoding process starts when the CQR Code is captured by a digital camera [8]. In many cases the positioning of the camera is not ideal and a rotated version of the CQR Code may be acquired. Figure 7(b) shows an example where such a distortion may be observed. The rotation angle is approximately 180◦ in relation to its original version. In order to correct the rotation angle, the finder patterns must be located. To accomplish this goal, first the image is converted to gray scale and a threshold th is determined by, th = (lm1 + lm2 )/2,
Fig. 6. Example of a Colored QR Code that will be used throughout the paper. Once the information has been properly encoded, the synthesized CQR Code is printed. In our experiments, we considered the size of 1.3 cm × 1.3 cm, as show in Figure 7(b), which represents the print-scanned version of the reduced CQR Code shown in Figure 7(a). Results for inferior data density have been reported in one of our previous work [7].
(a)
(b)
(c)
Fig. 7. (a) Reduced version of the CQR Code shown in Figure 6; (b) printed (size is 1.3 cm × 1.3 cm) and scanned version of the same CQR Code. It has been captured with approximately 180◦ rotation in relation its original version; and (c) CQR Code after thresholding, used to locate the finder patterns.
(4)
where lm1 is the gray level that corresponds to the first local maximum c1 of the image histogram and lm2 is the minimum gray level such that c2 = 2 × c1 , as depicted in Figure 8. The variables c1 and c2 represent the counts for gray levels lm1 and lm2 , respectively. Once th is determined, the C OLOR -T HRESHOLD procedure is called to estimate the original CQR Code colors (C is the CQR Code image). A typical result is show in Figure 7(c). The finder patterns are now well defined and can be easily located. The algorithm used to determine where the finder patterns are is a variation of the algorithm suggested by the ISO/IEC 18004 standard. The mutual distances between the center of the finder patterns are calculated and the orientation of the isosceles triangle from which they are the vertices is determined. The intersection of the two shortest sides, d1 and d2 , defines vertex A. B is the next vertex in a counterclockwise sense of rotation. The remaining vertex is C. After correctly identifying the vertices and sides of the triangle, the image is rotated until d1 is aligned with the vertical direction and A is at the up-left position, as illustrated in Figure 9. Since the number of horizontal and vertical modules and the distances between the vertices are known, the original CQR Code is estimated from its thresholded version. Each module in a regular grid is assigned to its most frequent
color. The result is shown in Figure 10. C OLOR -T HRESHOLD(C, th) 1 for i = 1 to C. height 2 for j = 1 to C. width 3 M axC = max(C(i, j, :)) 4 M inC = min(C(i, j, :)) 5 D = M axC − M inC 6 if D > th 7 if M axC == C(i, j, R) // red 8 C(i, j, RGB) = {255, 0, 0} 9 elseif M axC == C(i, j, G) // green 10 C(i, j, RGB) = {0, 255, 0} 11 else // blue 12 C(i, j, RGB) = {0, 0, 255} 13 else 14 if C(i, j, RGB) < {th, th, th} // black 15 C(i, j, RGB) = {0, 0, 0} 16 else // white 17 C(i, j, RGB) = {255, 255, 255} 18 return C B
d1
A
d2
Fig. 10. Decoded CQR Code. Table I. Results: the first column shows the CQR Code index. Second column shows the number of correctly decoded snapshots of the same CQR Code. The last column shows the average symbol error percentage of distorted modules.
C
d1
VI. CONCLUSION C
d2
A
B
Fig. 9. Rotation correction. V. RESULTS Our test set is composed by 15 snapshots of 8 different 1024-bit CQR Codes, which makes a total of 120 images. The printed versions of the codes have the size of 1.3 cm × 1.3 cm, resulting on a transmission capacity of approximately 605 bits/cm2 . The images were captured by a mobile phone with a 3.2 megapixel built-in digital camera under different illumination conditions. Currently, mobile phones with built-in camera are wildly used to recognize conventional QR codes [9]. Table I summarizes the results. All 15 snapshots of codes 1, 2, 4, 5, 7 and 8 were correctly decoded. On the other hand, for CQR Code 3 and 6 the performance was not as good. One of the snapshots for code 3 and two for code 6 were not correctly decoded. However, it is possible to observe that the average error rates are only 5.49% and 14.83%, respectively, which are way below the 38.41% Reed-Solomon error-correction capacity. In other words, the vast majority of the captured CQR Codes was consistently decoded. Figure 11 shows the snapshots that presented the highest symbol error rate for each CQR Code in the test set.
In this paper we showed how to encode and decode 1024 bits of information using Colored QR Codes. The proposed structure enables greater transmission capacity and also the use of more complex aesthetic patterns. Further investigations may consider a larger number or a different set of colors and an algorithm to counterbalance distortions caused by the inclination of the sensor. In addition, the impact of compression algorithms, commonly used in digital cameras, may also be evaluated. VII. REFERENCES [1] “ISO/IEC 18004:2005 - Information technology - Automatic identification and data capture techniques - QR Code 2005 bar code symbology specification,” August 2005. [2] H. Kato and K. T. Tan, “2D Barcodes for mobile phones,” in Proc. of the 2nd Int. Conf. on Mobile Tech., Applications and Systems, November 2005. [3] Dita I.C., M. Otesteanu, and Q. Franz, “Data Matrix Code - A reliable optical identification of microelectronic components,” in Proc. of the IEEE 17th Int. Sym. for Design and Tech. in Electronic Packaging, October 2011. [4] R. E. Blahut, Algebraic Codes for Data Transmission, Cambridge University Press, Cambridge, 2003. [5] B. Schneier, Applied Cryptography: Protocols, Algo-
(a) CQR Code 1.
(b) CQR Code 2.
(c) CQR Code 3.
(e) CQR Code 5.
(f) CQR Code 6.
(g) CQR Code 7.
(d) CQR Code 4.
(h) CQR Code 8.
Fig. 11. Snapshots that presented the highest symbol error rate for each CQR Code in the test set.
[6]
[7]
[8]
[9]
rithms, and Source Code in C, John Wiley and Sons, 2nd edition, 1996. R. C. Gonzalez and R. E. Woods, Digital Image Processing, Prentice Hall, New Jersey, 2nd edition, 2002. M. E. Vizcarra; A. C. A. Nascimento and A. Zaghetto, “Implementacion y Lectura de QR Codes con Cinco Colores y 512 bits,” in Proc. of the XVIII Int. Cong. of Electronic, Electrical and Systems Eng., August 2011. E. Ohbuchi, H. Hanaizumi, and L. A. Hock, “Barcode Readers using the Camera Device in Mobile Phones,” in Proc. of the 3rd Int. Conf. on Cyberworlds, December 2004, pp. 260–265. Y. Liu, J. Yang, and M. Li, “Recognition of QR Code with mobile phones,” in Proc. of Control and Decision Conf., July 2008, pp. 203–206.
