ECAI 2018 - International Conference – 10th Edition Electronics, Computers and Artificial Intelligence 28 June -30 June, 2018, Iasi, ROMÂNIA
Image Restoration for Halftone Pattern Printed Pictures in Old Books Adrian Ciobanu, Tudor Barbu, Mihaela Luca Institute of Computer Science, Romanian Academy – Iasi Branch Iasi, Romania
[email protected] Abstract – An image restoration technique for halftone pattern printed pictures is provided in this paper. Our approach recovers the original analogue halftoning of the old pictures into a digital clean halftoning. It starts from a thresholded version of the scanned images from old original books, and applies several black and white spots detection and replacement operations. Successful restoration experiments have been performed on numerous old photos related to Romanian participation in the Word War I. Keywords-image restoration; old pictures; analogue halftoning; digital halftoning
I.
INTRODUCTION
Romanian old books dated printed obtained using conventional-dot halftoning, a procedure that simulates shades of gray by varying the size of tiny black ink dots arranged in a regular pattern named halftone pattern. The reader of such a book perceives these kind of pictures almost as their original because the eyes blend together these dots to give the appearance of appropriate grays [3][6]. Halftoning was achieved by photographing the original image (in those years available as impressed photographic glass plates) through a screen consisting of another pair of glass plates on which fine opaque lines were etched, cemented together at right angles to each other, and thus forming a structure of square interstices. These interstices act as crude pinhole lenses and the result is a picture consisting only of light and dark dots. Light areas in the original picture produce larger areas in the screened image and hence smaller black dots to pe printed. Darker areas in the original produce smaller areas in the screened image and bigger black dots to be printed. The quality of such a reproduction was given by the fineness of the screen, measured in terms of number of lines per inch. In the books we referred to in this article, a low 70 lines/inch resolution was used. Because the visual system is most acute for orientations at 0 and 90 degrees and least acute at 45 degrees, the screen was oriented at 45 degrees so that the eyes are easier to fool. A historical background of this technique is presented in “Halftone, The Atlas of Analytical Signatures of Photographic Processes” [1]. Digital halftoning [5] [8] was introduced in 1970, based on printing technology advances. It uses a raster 978-1-5386-4901-5/18/$31.00 ©2018 IEEE
image or bitmap within which each monochrome picture element or pixel may be on or off, ink or no ink. To minimize problems that arise from using groups of monochrome pixels of the same size, the digital halftone pixels must be quite small, numbering from 600 to 2,540, or more, pixels per inch. There are also more sophisticated dithering algorithms to decide which pixels to turn black or white, some of which yield better results than digital halftoning, and nonlinear diffusion and stochastic flipping has also been proposed. Inverse halftoning is the reconstruction of a continuous tone image from its halftoned version. It constitutes a very challenging image restoration problem task because it is known to introduce visible distortions into the recovered digital images [2]. As pictures in old books are often affected by various printing manufacturing defects and there are also scanning related noises, an effective halftone image restoration technique is proposed in this paper. It represents the needed pre-processing step for a future inverse image halftoning approach which will restore almost complete the old original image that was used to obtain its printed version. The considered restoration technique is detailed in the following section. Then, our image recovering experiments are described in the third section. The conclusions of this article are drawn in the last section, where our future research plans in this domain are also discussed. II.
HALFTONE IMAGE RESTORATION TECHNIQUE
For pictures in old books (like those issued around 1930), printed using the conventional-dot halftoning technology, we are proposing the following procedure to restore as much as possible the original pictures (that no longer exists): •
Scan a picture from an original book, preferably with the highest resolution possible and in grayscale.
•
Find a global optimum threshold and transform the scanned image in black and white. The result is a collection of black spots on a white background and white spots on a black background. These spots are centered on a grid specific to each halftone pattern method.
•
Preprocess the thresholded image by detecting and deleting isolated black pixels on white background.
•
Detect all the black spots on white background with less than 20 pixels in composition and transform them in black spots with regular shape (close to a disk) depending on their size. In this way the analogue halftoning is converted in a digital halftoning. Black spots with more than 19 pixels will be addressed at a later step.
•
Detect all the white spots on black background with less than 20 pixels in composition and transform them in white spots with regular shape depending on their size. Again, the analogue halftoning is converted in a digital halftoning by this procedure. White spots with more than 19 pixels will be addressed at a later step.
•
Based on the obtained regular black spots with less than 20 pixels in composition, detect the grid points of the original halftone pattern as their center of mass or centroid. Apply a recurrent procedure to fill as much as possible the missing grid points if three already known neighbor grid points are available.
•
For black spots with more than 19 pixels apply cleaning techniques to divide them in several black spots with less than 20 pixels in composition, using as context information the already detected grid points. Then apply a regularization of shape also for them.
•
A similar technique should be applied for white spots with more than 19 pixels (rarer and not so important for the quality of the restored image).
•
The final restored halftone pattern image will contain only regular black and white digital spots, with leaks between them almost completely reduced, so the restored image will look very clean.
•
The last step will consist of an inverse halftoning procedure that will produce a gray level image with superior quality, ready to be used in common everyday tasks.
Figure 1. Graylevel scanned picture from old book (600 dpi).
Then, for preprocessing purposes, we need to know which regions are mostly white and which are mostly black, so that black spots on white background to be differently processed than white spots on black background. We used squares of 9×9 pixels, compute the majority of pixels (more white pixels or more black pixels) inside them and decide the nature of squares according to this majority, resulting in Fig. 3.
Figure 2. Black and white version of our example picture.
III.
IMPLEMENTATION AND RESTORATION EXPERIMENTS
The proposed halftone technique was implemented in MATLAB. We worked on 63 images, scanned with an usual scanner at 600 dpi, from the General Alexandru Averescu book “Daily War Notes (1916 – 1918)” – Second Edition – issued after 1935. We use as an example a picture on page 138, showing General Averescu inspecting the front line during heavy snow. The scanned version of this picture (graylevels, 600 dpi) is presented in Fig. 1. By applying the optimal threshold given in MATLAB by the graythresh function (global image threshold using Otsu's method), this image is transformed in its black and white version (Fig. 2).
Using the thresholded image and the black and white region information, we apply four steps of preprocessing on black spots over white background, eliminating prominent individual pixels on the black spot edges or individual pixels connecting two black spots. Figs. 4 to 6 show this kind of preprocessing in a detailed window of the example image. There is another kind of connecting pixel that is eliminated, but it is not present in the detailed window, so we cannot show it here. The last preprocessing step consist of eliminating isolated pixels or isolated pairs of connected pixels. They must be the result of noise from scanning or a byproduct of thresholding.
Figure 5. Eliminating prominent individual pixels on edges.
Figure 3. Mostly black and mostly white regions.
Anyhow, around 1935 it was not possible to obtain so small ink spots on purpose, so eliminating them seems a fair idea. Since in the detailed window we have no example of such pixels we cannot show it here. Then we start processing black spots on white background, taking into consideration their size.
Figure 6. Eliminating the first kind of connecting pixels on edges (see the black spot in center).
Figure 4. Detail window before preprocessing.
We first targeted black spots having from 3 to 6 pixels. If the contained pixels are grouped together, then we replace these black spots with a perfect shape, a rectangular shape of four pixels, as can be seen in Fig. 7, as a continuation of Fig. 6. To select such black spots we use the regionprops function properties ‘Area’ and ‘Extent’, working on an inverse version of the image. If the pixels are less grouped then we replace the detected black spots with a cross made from 5 pixels. This situation does not appear in our detailed window, but we will show this kind of regular shape when we will discuss white spots on black background.
Figure 7. Replacing grouped 3 to 6 pixels black spots with 4 pixels rectangles.
Then we targeted black spots with 7 to 10 pixels, which are replaced by rectangles of 9 pixels (Fig. 8). The black spots with 11 to 14 pixels are replaced with a kind of cross composed of 12 pixels (Fig. 9) and the last processed black spots are those consisting of 15 to 19 pixels, which are converted in rectangles of 16 pixels (Fig. 10).
Figure 8. Replacing grouped 7 to 10 pixels black spots with 9 pixels rectangles.
Figure 10. Replacing grouped 15 to 19 pixels black spots with 16 pixels rectangles (only 3 cases in this detail window).
Because with each replacement we also delete a row of pixels surrounding the regular shape, some artifacts may appear, like isolated 1 to 3 pixels black spots. To get rid of these we implemented a last cleaning step and the final result for processed black spots on white background is shown in Fig. 11. Fig. 12 presents, for the sake of comparison, the same detail window taken from the scanned image before thresholding. We can see that the majority of black spots were converted to regular shapes. Some regular shapes were trimmed due to the order of replacement and the proximity of black spots. Only one big black spot remained unprocessed, with more than 20 pixels. This will be addressed in a later step of our implementation.
Figure 9. Replacing grouped 11 to 14 pixels black spots with 12 pixels crosses.
The same kind of processing is applied to white spots on black background, after processing black spots. In this case, as preprocessing, we just eliminate isolated pixels or isolated pairs of connected pixels. We have selected a new appropriate detail window to show what happens for white spots (Fig. 13) and the result of preprocessing can be seen in Fig. 14.
Figure 11. Final detail window with processed black spots after cleanning.
Figure 12. The same detail window in the scanned image, before thresholding.
First we deal with white spots having from 3 to 6 pixels. If the contained pixels are grouped together, then we replace these white spots with a rectangle made of four white pixels, as can be seen in Fig. 15. If the pixels are less grouped then we replace the detected white spots with a cross made from 5 pixels (Fig. 16).
Figure 13. New detail window to show white spots processing.
Figure 14. Eliminating isolated white pixels.
Figure 15. Replacing grouped 3 to 6 pixels white spots with 4 pixels rectangles.
Then we targeted white spots with 7 to 10 pixels, which are replaced by rectangles of 9 pixels (Fig. 17), followed by replacing white spots with 11 to 14 pixels by a kind of cross composed of 12 pixels (Fig. 18). The last processed white spots are those consisting of 15 to 19 pixels, which are converted in rectangles of 16 pixels (Fig. 19).
Figure 16. Replacing less grouped 3 to 6 pixels white spots with 5 pixels crosses.
Figure 17. Replacing grouped 7 to 10 pixels white spots with 9 pixels rectangles.
Figure 18. Replacing grouped 11 to 14 pixels white spots with 12 pixels crosses.
The last cleaning step and the final result for processed white spots on black background is shown in Fig. 20. We can see that again the majority of black spots were converted to regular shapes. For easy comparison, Fig. 21 present the same detail window before thresholding.
the feeling of nowadays graylevel images and ready to be used in any computer environment.
Figure 19. Replacing grouped 15 to 19 pixels white spots with 16 pixels rectangles (only 1 case in this detail window).
Figure 20. Final detail window with processed white spots after cleanning.
The result of all these processing steps can be seen for the entire picture in Fig. 22. The picture has 6,134,912 pixels, from which 3,088,838 pixels were processed (50.35%). We have processed 141,733 black and white spots. This means that only computers can do such a job. It took a pretty fast laptop almost five minutes to complete the processing together with displaying all the intermediary images showing the progress of the processing. IV.
CONCLUSIONS AND FURTHER WORK
A novel halftoning image restoration algorithm for pictures printed in old books has been described in this paper. Its implementation is now in the stage of completely and successfully processing black and white spots with less than 20 pixels in composition, replacing them with digital regular shapes, which makes the image clearer and cleaner. Further work will imply implementing the cleaning of black and white spots with more than 20 pixels in composition, based on information extracted about the grid of halftoning points. This will completely clean the image and restore it almost perfectly to the halftone image that the publishing house wanted to print in the book. The last stage should implement an inverse halftoning technique meant to give the image
Figure 21. The same detail window in the scanned image, before thresholding.
Figure 22. Final result after processing black and white spots with less than 20 pixels in composition.
ACKNOWLEDGMENT This paper is dedicated to the centenary of the Great Union of Romania (1918 – 2018). REFERENCES [1]
[2]
[3]
[4]
[5] [6] [7]
[8]
D. C. Stulik and A. Kaplan, Halftone, The Atlas of Analytical Signatures of Photographic Processes, The Getty Conservation Institute, 2013, J. Paul Getty Trust. C. H. Son, “Inverse halftoning based on sparse representation,” Opt. Lett., vol. 37, no. 12, Jun. 2012, pp. 2352–2354. R. L. Adler, B. P. Kitchens, M. Martens, C. P. Tresser and C. W. Wu, “The mathematics of halftoning,” IBM J. Res. & Dev., vol. 47, no. 1, Jan 2003. Tien-Tsin Wong, “The Modelling of Natural Imperfections and an Improved Space Filling Curve Halftoning Technique”, Dissertation for Master of Philosophy Division of Computer Science, The Chinese University of Hong Hong, June 1994. Daniel L. Lau and Gonzalo R. Arce, Modern Digital Halftoning, Marcel Dekker Inc., New York, USA 2001. D. Anastassiou and S. Kollias, “Progressive halftoning of images”, Electronics Lett. 24(8), 1988, pp. 489–490. Y. Pnueli and A. M. Bruckstein, “Gridless Halftoning: a reincarnation of the old method”, in Graphical Models and Image Processing Vol. 58, No. 1, Jan., 1996, pp. 38–64. C. A. Bouman: Digital Image Processing, Chapter Digital Halftoning, January 8, 2018.
