Image Processing for Overnight Lighting ...

1 downloads 0 Views 2MB Size Report
Oct 6, 2003 - In order to understand the extent of the problem, one approach involved .... (One Canada Square) in London E14, which houses around 20 ...

Image Processing for Overnight Lighting Quantification in Buildings (presented at Improving Energy Efficiecy in Buildings Conference, Frankfurt, April 2010)

Dr Neil Brown, Institute of Energy and Sustainable Development, De Montfort University. Abstract For non-domestic buildings, the issue of the spread of unused office lighting has become topical, whereby a significant amount of electricity may be wasted, as suggested by initial surveys in London. In order to understand the extent of the problem, one approach involved night-time surveying of office buildings. The process of surveying buildings from the outside is laborious and expensive, and as such has been limited in the past to a small percentage of the national stock. This paper describes the application of time lapse photography and image processing for counting lit windows at night, in order to establish lighting usage patterns. Sample results are presented showing system performance, and illustrating the extent of unoccupied office lighting. These results also begin to validate the hypothesis that office lighting outside normal office hours in the UK is a prevalent waste of electricity, offering a considerable opportunity for energy savings.

Introduction In UK non-domestic buildings, a significant amount of electricity may be wasted though unoccupied lighting, as shown by initial surveys in London [1]. The process of surveying buildings from the outside can be laborious and expensive, and as such has been limited in the past to a small percentage of the national stock. Lighting use patterns when compiled from measured energy meter data normally would only be possible through extensive sub metering at considerable cost, and as such, have also been limited to a small percentage of the UK national stock. Non-invasive instrumentation offers a key to understanding the extent of unoccupied lighting. This paper suggests the application of time lapse photography and image processing for rapid surveys of lighting in the non-domestic stock, in order to establish [unoccupied] lighting usage patterns.

Background Energy consumed in buildings accounts for around 44% of UK CO2 emissions [2]. With concerns over global warming and climate change, building services have become a focus of attention. Building size, age, and glazing type can inform theoretical energy consumption, particularly for heating [3,4], but electricity use (e.g. lighting) is more carbon intensive than gas, yet often overlooked. One study, took after-hours ‘snapshot’ surveys from buildings to establish usage patterns of IT and office equipment [5]. This showed that from a sample of 1329 CRT monitors, the switch-off rate was only 32%. In an initial study on lighting, by H Bruhns at University College London, 140 West End office buildings in London were surveyed [1]. The sample included central city office buildings of varying ages, and sizes from office blocks to converted town houses. Daytime surveys established glazing, then each building was viewed twice between 10 pm and 3 am, and lit window percentages were found by counting. The surveys found 25% of lights on at 10pm falling to 17% at 2 am. These data were used to formulate 24 hour and full week lighting profiles of useful and wasted lighting use, showing that overnight and weekend lighting amounted to 23 – 30% of total lighting use. National electricity waste was estimated by extrapolating from these profiles to the UK, amounting to 1500 - 1900 GWh electricity, or 0.8 – 1.1 Megatonnes of CO2 per year, although this would assume that central London behaviour is replicated across the UK, and sample size would need to be increased to ensure accuracy. Research shows a 9% annual increase in electrical baseloads for Local Authority Buildings [6], a portion of which may be due to unoccupied lighting. Some efforts have been made to reduce unoccupied lighting in other countries: In Boston [7] in late 2008, lights in buildings above the 30th floor were extinguished as part of a trial to save electricity, from 11 pm and 5 am. In New York, improved building controls and conservation are cited as useful in reducing unoccupied lighting, but many companies’ buildings remain illuminated as status symbols [8]. An article describing a study of businesses in the City of London, estimated 200 kilotons of CO2 per year due to unoccupied lighting,

equating to 15 – 20% of energy bills [9], although the survey methodology is not described. The European Climate Change Programme identified lighting as one use of energy with high potential for cost-effective reduction of greenhouse gas emissions, described in a 2005 directive [10]. Technical solutions such as automatic lighting controls, and more efficient lighting are more visible in new-build, and to a lesser degree refurbished buildings, but are still a very small part of the market [11]. It is clear that as policymakers legislate further for cuts in unnecessary energy consumption from lighting, there is a need for much more comprehensive and reliable data. This particular waste of energy in buildings has wider reaching effects on the built environment and wider society. No professional astronomers remain in the UK, although a community of amateurs remains, whose efforts are hampered by ‘sky glow’ [12]. Many children in urban areas are no longer familiar with the night sky because so little of it can be seen. Also, light pollution from tall buildings has been cited as a major cause of death to migrating birds [13]. Light pollution can seriously affect enjoyment of the environment, including disruption of sleep, and as such is now treatable as a statutory nuisance in legal cases [14]. A health issue exists since recent research indicates increased rates of certain cancers through disruption of melatonin production, one cause being light pollution [15]. In any case, it is clear that the scale of electricity wastage is potentially significant and must be investigated. A major difficulty in estimating the scale of the problem has been manually surveying buildings, compounded by night-time working. Time-lapse videos were made by the author of Canary Wharf (One Canada Square) in London E14, which houses around 20 businesses, and central Sheffield, in early 2007. It became clear from the Canada Square video that occupancy patterns were more complex than previously thought, with lights being switched on and off throughout the night. This suggests strongly that future studies require fine-grained data, with several samples per hour (which would not normally be practicable manually). It also suggests that distinct differences exist between lighting use patterns for occupied and unoccupied buildings, and ordinary occupancy hours vs. 24 hour operation.

Time lapse photography Canon EOS -30D digital cameras were used in conjunction with TC80N3 external controllers, which can be set to trigger the camera shutter at preselected intervals, from seconds to hours in length. 1855mm zoom lenses were used, set to manual focus, with image resolution set to 8.2 megapixels, and 1GB memory cards used. Aperture and shutter speeds were set to automatic. Cameras were placed indoors, facing outwards through windows chosen for accessibility and view. It was found that battery life for the manufacturer’s rechargeable batteries was adequate for at least 96 hours’ use with a ten minute interval between shots. In order to shield the cameras from indoor lighting, a cardboard ‘hood’ was used (as shown to the right of figure 1), which was taped to the window.

Figure 1 - Camera setup

Video Analysis Initially a time lapse series of images can be converted into a short movie, e.g. in avi format, which was carried out using Jasc Animation Shop software. This enables basic manual analysis of night lighting usage patterns. Initially a time lapse series of images was taken of the skyline of Canary Wharf, looking southwards from the Bow Triangle (London E3), at a distance of around 2km. These images were taken on the morning of 27th January. Figure 2 shows a series of images extracted from the time lapse series of one building, show the uppermost 18 floors of No.1 Canada Square, London E14 (this tower is commonly referred to as ‘canary wharf’), which houses around 19 businesses. It was thought that many such buildings have lighting used all night, although the buildings may be unoccupied. As can be seen from figure 2, some activity is clearly visible even between 2.36 and 5.36 AM on certain floors. A complete floor can be seen to have the lighting cycled, and several windows change in intensity – This may be due, for example to deep plan office lighting use varying with respect to depth from the facing elevation. Certain floors however, remain constantly lit.

Figure 2 - Time lapse photography of No.1 Canada Square

Image Processing – window identification In order to produce useful information which can be used to produce a model of night-time illumination, an automated count of illuminated windows can be used to provide lighting use profiles. For an evenly illuminated building (it is rare that a city building is not lit at night from the outside by light pollution) with distinguishable windows, as opposed to curtain walling, it was thought that the task of automatic window counting would be quite straightforward. A simple program (called an igraph) was written using WiT image processing software and is shown in figure 3. A directory of images is selected, and is fed into a sequencer. This is used to ‘pump’ the main code with the directory file names. The readObj operator is used to extract image data from files, whilst a counter keeps track of the number of images processed. A region of interest is extracted for processing, which is converted to a grey-scale image. An automatic threshold of the image is taken based on the standard deviation of the pixel grey levels within the image. This produces a binary image, where pixel values of 1 = white, and 0 = black. Since window frames may give a false count, these are artificially ‘blurred’ using an erosion operator, which adds a ring of white pixels around each thresholded object. For cosmetic purposes (ease of visualisation), the resulting image is then eroded such that the identified windows may be superimposed on the original colour image. The ‘getblobfeatures’ operator then is used to identify the properties of connected regions (clusters of ones within the binary image). Output parameters include features such as area identified (blob) blob count, area, perimeter, and xy coordinates of centroid. For the purposes of demonstration, the blob area is utilised, and the count. Blobs (in this case, illuminated windows) are then superimposed upon the original colour image. Figure 4 shows the results of running this code on a sample image taken mid-evening on Friday 25th January. The output from the software is presented as three tiled images. From left to right we see the original image, followed by the binary image. The windows identified as lit may be seen to the right. As can be seen, there is a small degree of error whereby some windows which are lit have not been picked up by the processing algorithm. One difficulty realised with applying a threshold is that some parts of the image may be lighter than lit windows, for example walls illuminated by a streetlight. Figure 3 - Basic processing algorithm for single building

Figure 4 - Apartment blocks, image, thresholded and lit windows identified

Processing method for large scenes A typical scene for processing is shown below in figure 5. In order to compensate for illumination of building surfaces, a lowpass filter is applied. This extracts ‘low frequency’ or gradually changing features from an image, such as a large dimly lit wall, or as can be seen in figure 5, a floodlit building such as the church towards the bottom left of the image. Also when imaging large scenes, image brightness can vary considerably from one part of the image to another. Adaptive thresholding was found to work for images of single buildings as shown in figure 4, but ease of use was limited by the need to provide a kernel to describe the area of image which would be averaged in order to produce the local threshold. Since the number of pixels representing each window varies considerably from one building to another. It was found that taking a moving average of the lowpass image produced a more stable result, compensating for shifts in illumination due to moonlight and varying cloud cover (figure 6). A highpass filter is used to extract high frequency data, as illustrated in figure 7. In practice, this means features which rapidly change with respect to x and y positions throughout the image, or small objects. The method effectively ‘amplifies’ lights and illuminated windows, and reduces larger features. Subtracting the lowpass image from the highpass image reveals lighting, with all building detail removed (figure 8). The image is then thresholded based upon standard deviation of the image pixel values, in order to produce a binary image such as that shown in figure 9. Blob analysis is then used to count visible lights, as can be seen in figure 10. Figure 5 - City of Sheffield night image

Figure 6 - Low Pass Filtered image (mono)

Figure 7 - High Pass Filtered image

Figure 8 - (High Pass image) - (Low Pass)

Figure 9 - Thresholded image

Figure 10 - Automatic identification of lighting

Figure 11 shows the dataflows for the image processing and image feature extraction system. Image files may be interrogated in order to extract accurate timestamp data from images. Exchangeable image file format (EXIF) is a specification for the image file format used by digital cameras. The specification uses the existing file formats including JPEG and TIFF, with the addition of specific metadata tags. Matlab software version 7.5.0 was used to interrogate image files, using the Matlab function exifread to extract timestamp data. Timestamps were added to the textfiles of window counts provided by Wit, and the resultant data uploaded to a Mysql driven open source database for storage, and because of ease of use of Mysql’s functions for processing data by time and date. Figure 11 - Dataflows for image processing and feature extraction

Figure 12 shows the (graphical) code implemented in Wit for processing large scenes (with a feature to extract individual buildings) in more detail. The actual image processing part of the code is relatively small, with much code used for housekeeping. Code operation is described as follows. Upon start-up, the image directory is scanned and image names are loaded in to the software for scanning (A). Images are imported from JPEG files and converted to greyscale for processing (B). Daytime images are not useful for this particular analysis, but order to retain time stamps for plotting, all images are processed to be dropped at a later stage based on daylight levels. This may not at first appear to be efficient coding, but this code in the future may be used for identifying lighting which is used unnecessarily in the daytime. A region of sky is extracted from the image and a mean value taken, which is subjected to a hard threshold in order to separate night from day data. In practice, this has not required adjustment, and is set at 100 in the range 0-255 (8 bit pixel intensity, unit less). Daytime data is replaced with zeroes (L). The building under observation is selected using user input with a mouse, selecting a rectangular area (D). The mean total image brightness (E) has been found to work well for thresholding in conjunction with the combination of lowpass (where the kernel size is around 10x the largest window size), (F) and highpass (where the kernel size is smaller than the minimum expected window size), filtering (G). The image is thresholded and the binary image of the building is fed to an erosion operator (H). This cleans up any moderately blurred images to separate out individual windows at times of low visibility, such as during rain or fog. The blob features (features of connected regions) are extracted (J) and as a quality check, are plotted overlayed on the original greyscale image (K). Window counts are grouped and stored for plotting (L). Filenames are assembled and the resultant vector of window counts is saved to file (M).

Figure 12 – Large Scene processing algorithm implemented in WiT

Survey Areas Sheffield Time lapse photographic surveys were carried out during the winter between Feb 1st and Feb 5th. Figure 13 shows the Sheffield University Arts Tower, where pictures were taken from an office window on the 18th floor, looking South East towards the City Centre (Figure 15), and North East towards the Kelham Island and Don Valley districts (Figure 16). Photographic surveys in Birmingham were taken from the 16th floor of the Chamberlain Tower, looking North towards the city centre, East and West areas (Figures 17 and 18 respectively). These photographs were taken from May 13th – 22nd. Photographs were taken at 15 minute intervals with fixed infinite focus, aperture and shutter speeds were set to automatic. Figure 13 - Arts Tower, Sheffield University

Figure 14. Chamberlain Tower, Birmingham University

Figure 15 - Buildings in central Sheffield

Figure 16 - North Sheffield buildings

Figure 17 - Central Birmingham (West)

Figure 18 - Central Birmingham (East)

Results In order to check the accuracy of the image processing system when compared to human analysis, a number of time lapse images were manually checked. Referring to figure 19, the correlation between manual and automatic window counting for the London tower blocks illustrated in figure 4 is 93%. For non-domestic buildings also using the processing method for large scenes, correlation is slightly higher. Two examples shown in figure 18 are for office buildings in Sheffield (during Winter with poor visibility), and Birmingham (during Spring), scoring 93.3% and 97.4% respectively. Figure 19 - Manual window counting vs. image processing

Figure 20 shows a complete set of profiles for the photographic survey carried out in central Sheffield. Building numbers as shown relate to the areas outlined in figure 15. Some noise exists in profiles for the early hours of Saturday and Tuesday, which was due to a snowstorm, and fog. This would mean that the image processing system, at certain times, may underestimate the number of lit windows visible. Profiles have been sorted by peak numbers of visible windows for ease of viewing. The lighting profiles shown in figure 20 are for offices only, and are presented in a similar format averaged by broader time bands to more clearly show the differences in lighting use between weeknights and weekends.

n u m b e r o f w in d o w s v is ib le








12h 18h


06h 12h 18h 00h

time o f da y ( ho ur )


sa tur d ay

06h 12h 18h

sun da y

00h 06h


mo nd ay 18h 00h


tu esd ay

1 16d

3 10



bu ildin g nu mbe r ( d = d omestic)

2 14


9 12d


11 6 18d 15 17 13

Figure 20 – Lighting profiles for buildings shown in figure 15

Figure 21 - Week vs. weekend lighting averaged by time bands

Discussion and future work Imaging buildings from further away enables larger scenes to be surveyed during each time-lapse session, and consequently more buildings. Performance was affected by uneven image brightness across each image due to moonlight, cloud cover, and the intensity and clustering of illuminated windows themselves. The algorithm described in figure 3 was found to be greatly improved though the use of spatial filtering as described in the code shown in figure 12. Software run-times have been improved through the manual selection of the region surrounding each building, such that the whole scene does not need to be processed to find, e.g. image brightness. Typically the processing of 800 images takes one minute. The accuracy of the system was found to be 93% in a test with the basic algorithm (for close-up shots of around 200 metres). For tests on large scenes, with a range of up to 2 km, the lower result for accuracy was 93.3% and the higher was 97.4%. The time lapse method means that a surveyor is not required to work at night, with installation of the cameras taking around 30 minutes, and 15 for retrieval. The use of the post processing software offers significant time savings. Manually counting windows from images was found to take up to 10 seconds per frame, per building. The current image processing software counts at slightly higher than 130x this speed. Weather effects such as fog and snow can be seen to affect the performance of the imaging system, whereby the number of visible windows in a particular image may drop suddenly. A solution is to take images over a longer period. Using the equipment described, it was found that camera battery life and memory was adequate for surveys lasting at least 6 days. The use of a micro hard drive to extend camera memory capacity, and construction of an electronic controller to limit image taking to night time only may also be considered. What becomes apparent is that many buildings begin an evening with illumination approaching a peak, and in most cases, lighting is gradually reduced overnight, but not by a large margin. Referring to figure 21, the low rate at which lighting is reduced for offices during the course of the night is notable. It also becomes clear that lighting use has not dropped significantly at weekends between 24:00 and 02:00. Preliminary results are broadly similar to those suggested in [1], whereby drops in night time lighting use are low. Further time lapse photographic surveys are planned in order to build up a more accurate picture of night time and weekend lighting use, which will be the subject of a further paper.

Conclusions The image processing system performance is comparable to that of a human observer, in its current iteration, with peak accuracy approaching 98%. Since the image processing system tends to underestimate the amount of lit windows, it is clear that data generated will not exaggerate the case for a reduction in unoccupied office lighting. Time savings are considerable, with no human intervention required at night, and off-line counting of illuminated windows enabled at greatly improved speeds. The system developed can give a broad picture of night time and unoccupied lighting use far more cheaply than, for example, sub metering of feeds for lighting circuits. Results validate manual surveys that show lighting use not to drop significantly outside business hours. The system described in this paper will be used to further explore the issue of lighting use in unoccupied buildings.

Acknowledgements The original idea for surveying buildings externally to quantify unoccupied lighting came from Harry Bruhns, and early stages of the work formed part of Carbon Reduction in Buildings (CaRB) Consortium. CaRB has 5 UK partners: De Montfort University, University College London, The University of Reading, University of Manchester and The University of Sheffield. CaRB is supported by the Carbon Vision initiative which is jointly funded by the Carbon Trust and Engineering and Physical Sciences Research Council, with additional support from the Economic and Social Research Council and Natural Environment Research Council. The partners are assisted by a steering panel of representatives from UK industry and government. See for further details. Apparatus was purchased as part of a grant from the Science Research Investment Fund (SRIF) from The Higher Education Funding Council for England. The author would like to thank Birmingham University Estates Department and Sheffield University School of Architecture for access to the viewpoints from which pictures were taken.

References: [1]

Bruhns, H.R. Quantifying national electricity waste from overnight office lighting. Improving Energy Efficiency in Commercial Buildings (IEECB’08) Frankfurt, Germany, 10-11 April 2008.


DEFRA, DEFRA, e-Digest Statistics about: Climate Change. 2006. Department for Environment, Food and Rural Affairs, Customer Contact Unit, Eastbury House, 30 - 34 Albert Embankment, London, SE1 7TL, UK


Anon. International Survey of Building Energy Codes. Available from: 2008. Department of Climate Change, 2 Constitution Ave, Canberra, ACT 2600, Australia.


Persson, M.-L. Windows of Opportunities: The Glazed Area and its Impact on the Energy Balance of Buildings. Available from: 2006. Doctoral thesis, Uppsala University, P.O. Box 256, SE-751 05 Uppsala, Sweden.


Webber, C. A. Roberson, J. A., McWhinney,et al., After-hours Power Status of Office Equipment and Energy Use of Miscellaneous Plug-Load Equipment. Energy, 31, 2823-2838, Nov 2006.


Carbon Reduction in Buildings (CaRB) project, EPSRC Grant Reference: GR/S94377/01, IESD, DeMontfort University, Leicester, LE1 9BH, UK.


Ross, C. Lights out, conservation on for city's tall towers. Boston Globe, September 3, 2008. Boston Globe,135 Morrissey Blvd. Boston, MA 02125, USA


Belson, K. Efficiency’s Mark: City Glitters a Little Less. New York Times, November 1, 2008.


Smith, L. Reporting the Global Action Plan study. The Times, November 18, 2006. Times Newspapers Ltd, 1 Virginia St, London E98 1XY, UK


Directive 2005/32/ec of the European Parliament and of the Council Establishing a Framework for the Setting of Ecodesign Requirements for Energy-using Products Amending Council Directive 92/42/eec and Directives 96/57/ec and 2000/55/ec. PE-CONS 3618/05, Strasbourg, July 6th, 2005.


Van Tichelen, P., Jansen, B. , Geerken, T., Vanden Bosch (Laborelec), T., VanHoof, V., Vanhooydonck (Kreios), L., Vercalsteren, A. Final Report Lot 8: Office lighting, Contract- TREN/D1/40-2005/LOT8/S07.56452) Preparatory Studies for Eco-design Requirements of EuPs. 2007.


House of Commons Science and Technology Committee. Light Pollution and Astronomy Seventh Report of Session 2002–03. HC 747-I, Published on 6 October 2003, by authority of the House of Commons, London: The Stationery Office Limited


Anon. Light Awareness Program to display birds killed in collisions with buildings. Royal Ontario Museum, 100 Queen's Park, Toronto, Ontario, M5S 2C6 Canada.


Morgan-Taylor, M. P., Light Pollution and Nuisance: The Enforcement Guidance for Light as a Statutory Nuisance. Journal of Planning and Environment Law, 1114-1127, 2006.


Pauley, S.M., Lighting for the Human Circadian Clock: Recent Research Indicates that Lighting has Become a Public Health Issue. Medical Hypotheses, 63, 588-596, 2004.