Very short-term solar forecasting using inexpensive ...

Very short-term solar forecasting using inexpensive fisheye camera sky-imagery Saad Sayeef1, Sam West1 1 Commonwealth Scientific and Industrial Research Organisation (CSIRO), Newcastle, Australia [email protected] Keywords: cloud motion, solar forecasting, sky camera Abstract Very short-term forecasting of solar irradiance is important for a number of applications, including control of solar thermal power levels and scheduling of fossil fuel powered generators or storage solutions in hybrid renewable minigrids. Accurately forecasting direct normal irradiance or global horizontal irradiance in the seconds-to-minutes time-frame enables such applications to leverage fine-tuned dynamic operational schedules that can reduce fuel costs, increase network stability or maximise system lifetimes. Traditional short-term solar forecasting has relied upon expensive sky cameras equipped with sun-tracking shadow-bands to provide averaged cloud movement and irradiance estimates [1]. More recently, relatively inexpensive fisheye security cameras have been repurposed for the task, providing unshaded, but much higher quality images [2], though very little published research uses these images for forecasting. This paper explores the development and performance of irradiance forecasting algorithms using sequences of sky images from inexpensive cameras at two sites in Newcastle and Canberra, Australia. Using a database of images and irradiance measurements at these sites, this paper presents novel techniques for applying per-pixel optical flow algorithms to determine cloud movement vectors, and extrapolating cloud movement paths to identify future shading timing and duration. Forecast accuracy is evaluated by comparison with co-sited DNI and GHI measurements. This paper also presents a binary cloud/sky classifier approach using a neural network, trained using a pre-labelled multi-day image set encompassing a variety of cloud conditions and more than ten distinct features extracted from raw pixel-data. This classifier is able to classify cloud edges and bodies achieving greater than 95% accuracy on a large test set. The cloud classifications produced are used to improve the accuracy of the shade timing forecasts, and can be easily expanded to include multiple cloud types and thicknesses.

Solar2014: The 52nd Annual Conference of the Australian Solar Council

1.0 Introduction The solar resource available to the Earth’s surface is considerable but highly variable due to atmospheric conditions. Variations occur commonly due to diurnal patterns, physical shading and atmospheric obstructions such as clouds and particulates. The characteristics of each event and their corresponding effect on the ground can be quite different. Atmospheric obstructions such as scattered cloud can occur quickly and without warning, but their effect may be transient and only affect a small area depending on the cloud’s size, altitude and position relative to the sun. Characteristics such as these make accurate solar forecasting at high temporal and spatial resolution difficult to model. Thus solar forecasting can be split into a “macro” picture and a more detailed “micro” picture, where the former provides an overall forecast and average range of conditions using atmospheric data from satellites and meteorological stations, and the latter can be obtained using ground-based sky imagers which are location-specific and provide data allowing individual impact to be assessed. Predicting the solar future in this way can benefit the cost-effective management of intermittency in individual grid operation, market operation, network planning and capital investment. For example, in ‘town size’ off grid systems, predictions may provide advanced warning of reductions in solar power production and allow additional capacity to be prepared, or identify periods of excess generation and allow electrical loads to be brought forward and storage devices to be recharged. More accurate generator unit commitment scheduling and generation and demand profile matching may also be achieved. Solar variability can be characterised in the temporal or spatial domain. The temporal domain describes the “time horizon” of the forecast which describes the maximum reach into the future possible with a given system. Fluctuations over these timescales also give rise to intermittency effects on power systems, as indicated in Table 1. More detail on solar intermittency and its grid impacts can be found in CSIRO’s 2012 report on solar intermittency [3]. Table 1 Potential power system impacts of intermittency over various intermittency timescales

Timescale of Intermittency Seconds

Potential Power System Impact Power quality (e.g. voltage flicker)

Minutes

Regulation reserves

Minutes to hours

Load following

Hours to days

Unit commitment

Short-term irradiance forecasts are aided by the fact that clouds can be observed visually. Forecasting irradiance and power production using sky imaging represents a key opportunity to manage both solar power intermittency directly and manage the uncertainty surrounding solar


generation. This has clear outcomes for improved electricity system planning, supply and demand matching and the operation of commercial solar installations. This paper provides an overview of how different methods can be used to analyse sky images to produce accurate short-term solar irradiance forecasts. 2.0 Current state-of-the-art short-term solar forecast models Current solar resource forecasting methods include using numerical weather prediction (NWP) and satellite cloud observations. NWP is capable of providing information up to several days ahead but with several biases and random errors in the irradiance estimates [4]. Information on the state of the atmosphere and Earth’s level of cloud cover have also been used to generate time/site-specific irradiance data and maps of solar radiation. Using imagery from geostationary satellites, forecasts using these images have been able to provide accurate solar irradiance predictions at a resolution of 1 square kilometre up to six hours ahead [5]. Both the NWP and satellite-based forecasting methods are, however, inadequate to achieve high temporal and spatial resolution for intra-hour forecasts [6]. Ground observations using a sky imager have demonstrated the ability to fill the gap - obtaining accurate short-term solar irradiance forecasts – however the technique is not widely used commercially, mainly due to the high cost of specialised cameras, the complexity of image-processing algorithm development and the relatively small-area forecast provided. The recent availability of suitable inexpensive cameras and high performance image processing libraries has made this approach more technically feasible, while integration into a wide-area forecasting system makes the small forecast area much more useful in a broader context. Sky imaging techniques also show potential to augment the spatial and temporal resolution provided by satellite and numerical forecasting methods, making their use even more attractive in solar irradiance forecasting systems. An essential component of short-term solar forecasting is the detection of clouds and their motion in the sky. The following subsections provide an overview of the various techniques that have been used for obtaining cloud motion vectors to track the speed and direction of clouds. 2.1 Motion vectors from skycam images A rooftop-mounted total sky imager TSI 440A has been used to take images of the sky over the University of California, San Diego (UCSD) since August 2009 [6]. This sky imager has a downward pointing camera observing a spherical mirror which reflected the sky image. Images were taken by the system every 30 seconds when the sun was above an elevation angle of three degrees. The resolution of the images provided by the camera was 640 by 480 pixels and the mirror occupied 420 by 420 pixels. The cloud detection technique used by [6] is based on the concept of ratio variation between the red and blue wavelength channels developed at the Scripps Institution of Oceanography [7]. Cloud velocity and direction of motion was then


determined using a cross-correlation method (CCM) applied to two consecutive images of the sky. In this method, the movement of pixel blocks between two images is determined by searching for the area in the second image which has the highest correlation to each block from the first image. This process produces a grid of movement vectors, containing information about the direction and speed of observed movement. By assuming spatial homogeneity in the cloud velocities, the movement vectors are postprocessed to obtain an average cloud velocity across the image. Cloud formation, deformation and evaporation can result in reduced correlation. Chow et al also found that the appearance of a cloud may change over time due to different camera white balancing, lighting on the cloud and different viewing geometries [6]. One of the disadvantages of the TSI 440A sky imager used in this study is the loss of part of the data in each image due to obscuration by the camera arm and shadow-band which permanently obscures parts of the sky traversed by the sun. This rigid motion model proposed and developed by Chow et al employed a block matching strategy to detect cloud movement and did not incorporate cloud formation and deformation monitoring capability. Bernecker et al used non-rigid registration to model complex dynamics of cloud motion [8]. Two different approaches to non-rigid registration that employed different matching strategies were investigated. One of these approaches is known as the Demons algorithm and is based on optical flow [9]. This algorithm iteratively calculates forces that deform the template image to match it to a reference image. The other approach investigated is based on a variational formulation by Fischer et al. [10], based on the principle of minimising an energy function. Bernecker et al compared the two approaches with the block matching technique used by Chow et al and a 19% improvement over the block matching strategy was reported. Urquhart et al deployed two TSI 440A sky imagers at a 48MW PV power plant in Henderson, Nevada [11]. The images obtained from the two sky imagers were analysed to generate cloud maps over the PV plant which covered an area of 1.3 square kilometres. A binary map showing cloud shadows with a resolution of 6.25 square metres was forecast for up to 15 minutes. Cloud height was estimated by employing stereography using images from two sky imagers. The cross-correlation process employed by Chow et al was used to yield cloud motion vector fields at consecutive instants in time, which were then used to generate cloud position forecasts. The testing of the forecast performance on three different days showed that the day with the least clouds had the lowest error while the day with the most clouds produced the highest error. Cloud tracking work was also undertaken by Wood-Bradley et al [12] using the Lucas-Kanade algorithm for optical flow. This algorithm was reported to work best with low displacement of pixels between subsequent images. 2.2 Motion vectors from satellite images


A body of research exists regarding the analysis of cloud image sequences from satellite data streams. The algorithms used in this scenario may prove useful for evaluating skycam photos as well as potentially identifying areas where satellite and skycam techniques can be combined to augment forecasts. Hammer et al applied a statistical method to detect the motion of cloud structures from satellite images [13]. Cloud images were used to estimate surface irradiance for timescales of 30 minutes to two hours by extrapolating the temporal development of cloud information. The motion of clouds was determined from two consecutive cloud index images and a motion vector field was estimated based on a model developed by Konrad et al [14]. Hammer et al introduced an estimation criterion for the quality of the motion vector field then applied a probabilistic model of motion before using a Monte Carlo algorithm to search for the most probable vector field. The calculated motion vector fields were used to forecast solar irradiance, however, the challenge of taking into account cloud formation and dissolution was not addressed. Hamill and Nehrkorn also proposed a cloud forecasting technique using satellite images. Cloud motion displacement vectors were derived using cross-correlation analysis of an area in two sequential frames of satellite data with surrounding areas in the second frame [15]. The developed scheme was tested using two satellite images of the same scene half an hour apart. It has been seen that the methods used to derive cloud motion vector algorithms to date using images from sky and satellite imagers are similar in nature. Though the task may be similar, the spatial and temporal resolution currently achievable using ground-based cloud trackers is significantly higher than satellite imagers, though only over a relatively small area. Overall these technology traits will likely lead to the development of different approaches depending on the application and will likely promote fusion of the forecasts from high resolution, highly-localised sky cameras and lower resolution, wide-area satellite data, to give as complete a picture as possible for the near, middle and longer time horizons. The work presented in this paper is differentiated from prior studies mainly due to the use of a relatively inexpensive IP-camera sold for off-the-shelf security applications. The motivation for this selection was to investigate the performance possible using simple, low-cost sky camera systems. 3.0 Inexpensive skycamera for short-term solar forecasting The hardware selected for the work presented in this paper can be purchased for AU$600-800, is weatherproof and installation is simple and, requires only physical mounting and the connection of one Ethernet cable supplying data connectivity and power. Almost all previous research has been performed using more expensive imaging and shading systems, generally costing


upwards of AU$20,000. Two CSIRO facilities currently host instruments well suited to sky camera research - the CSIRO Energy Centre in Newcastle and the CSIRO Marine and Atmospheric Research Division’s solar monitoring facility at Black Mountain, Canberra. Figure 1 shows the two models of security camera currently in use as sky cameras at Newcastle and Black Mountain. Specifically, hardware available at these sites relevant to sky camera forecasting research include: At Newcastle:        

1 x Mobotix Q24 skycam 6 x Vivotek FE171V skycams 1 x Tracking Pyroheliometer 1 x LiDAR Ceilometer 1 x Pyranometer 1 x Weather station Multiple Photovoltaic Arrays 2 x Concentrating solar research fields

At Black Mountain:     

1 x Mobotix Q24 skycam 1 x Pyranometer 1 x LIDAR Ceilometer 1 x Weather station Multiple Photovoltaic Arrays

Figure 1: The IP-cameras in use as inexpensive sky cameras: The Mobotix Q24 (left) and the Vivotek FE8172V (right)

4.0 Early System Development A library of 24 months of sky camera images at 10 second intervals has been collected from the Newcastle site, and a similar dataset of 12 month from the Canberra facility (as of May 2013). Early work on cloud motion forecasting research performed by CSIRO includes development of a prototype system incorporating algorithms for sun position tracking, cloud identification, classification and edge detection, cloud movement tracking, distortion correction, motion projection and a simple scheme for predicting direct/diffuse/total horizontal irradiance. Figure 2 shows some examples of the results of several stages of the prototype’s image processing pipeline, showing a) the raw sky image, b) a false-colour version of the cloud classification using a simple red:blue ratio (RBR) threshold, c) a classified and masked cloud image, and d) gridded Solar2014: The 52nd Annual Conference of the Australian Solar Council

cloud motion vectors extracted from two sequential masked cloud images using a cross-correlation search.

a)

c)

b)

d)

Figure 2: Examples of a) an unmodified sky camera image (top left), b) a false-colour cloud classification image (top-right), c) masked cloud classification (lower left), and d) cloud motion vector extraction (lower right)

Figure 3 shows the interface for a fisheye distortion motion projection algorithm which allows arbitrary velocity vectors to be extrapolated and mapped onto the distorted fisheye image. This mapping indicates the apparent path a moving object will take across the image and allows for lens distortion to be accounted for when generating motion vectors.


Figure 3: The lense distortion motion projection test application which allows the system to account for lense distortion in cloud vector mapping

Figure 4 shows the results of a count of the ‘cloudy-pixels’, i.e. the number of pixels classified as part of a cloud across a single day using the prototype’s cloud classification engine. Simply counting the pixels in the sky is not a forecast per se, but could be a useful metric for informing very simple controls schemes for scheduling backup generators, for example. This represents a useful control signal for the most conservative approach to generator control – simply starting it whenever clouds are detected in the sky image, minimising the risk of blackouts caused by solar panel shading.


Figure 4: An example of the graphical interface for the sky/cloud classification system showing a cloudy day with early morning and some afternoon sun.

5.0 New Forecasting Developments More recent development has produced a system with a number of improvements from the simpler forecasting approach based on other efforts in the literature, as outlined in the previous section. Other recent skycam-based solar forecasting systems, such as [16], [17], extract motion vectors using the block-matching cross validation approach discussed previously, and subsequently take the average velocity and apply it to the whole sky image. This method, however, was found to be unable to produce accurate forecasts when complex weather conditions exist, producing cloud formation and dissolution, or multiple cloud layers moving with differing velocities. Instead, a motion vector extraction technique based on a common technique from computer vision, called dense optical flow (also proposed in [8], [12]), is proposed which estimates the motion of each pixel from one image to the next. This allows a more general solution which has been found to perform well under complex cloud movement conditions. Traditional approaches to cloud identification have involved simple thresholding of pixel values based on the ratio (RBR) and difference (RBD) of red to blue channel ratio values [12], [18], [19], and comparisons with clear sky libraries [17]. When testing implementations of these schemes, the Solar2014: The 52nd Annual Conference of the Australian Solar Council

authors found them to perform well under most conditions, but poorly with very dark clouds, near the sun when haze or lens flares were present, and close to sunrise and sunset where the sky’s colour temperature changes significantly. These difficulties were limiting the accuracy and usefulness of our forecasting approach. To overcome these problems, the authors have developed an advanced forecasting model using an artificial neural network which incorporates both traditional metrics (RBR and RBD), and uses additional inputs for channels in the RBG and HSV colour spaces, pixel displacement amount, distance from the sun, and the sun’s position. The neural network is trained using a usercreated dataset identifying clouds and sky under a variety of weather conditions and times of day. Using 5-fold cross validation, the network was able to correctly classify 95% of the cloud pixels on a dataset of 500,000 pixels selected from several weeks of data. Figure 5 shows two examples, both the new optical-flow based movement vector extraction technique, and the neural network cloud classifier in action. Both sky images show false-coloured clouds that are likely to shade the sun at some point in the future in white and yellow. Red areas show sections of clouds in the sky that are moving towards the sun, but have been classified as sky. Thin clouds and cloud edges are shown in yellow, while pixels which have a high probability of being clouds are shown in white. Multiple cloud layers moving toward the sun from different directions can be seen in both images, highlighting the advantages discussed previously. Irradiance forecasts which consider all visible cloud movement are being developed by modelling the relationship between cloud amount and thickness of the highlighted areas, and observed irradiance changes in historical measured irradiance and solar power output.

Figure 5: Examples of dense optical flow’s ability to identify multiple cloud layer movement in different directions


6.0 Results A binary shading forecast model using the image processing outputs discussed above was developed. This model attempts to, for example, predict 1 minute-ahead drops in direct normal irradiance (DNI) of more than 50% of the expected clear sky DNI by thresholding the amount and thickness of clouds forecast to shade the sun in future 10-second timesteps. This threshold can be adjusted to change the bias and sensitivity of the forecast, and is designed ot match well to applications such as diesel generator scheduling for remote area power systems, allowing very conservative forecasts that avoid false positives (predicting sun when it’s actually shaded) to avoid blackouts, or maximising overall prediction accuracy to minimise generator fuel consumption and emissions. Table 2 shows results from the model, tested using 14 days of sky images in December 2013 at the Canberra installation. Accuracy is measured as the percentage of 10-second timestep images which were correctly predicted to be sunny or shady. Three different bias/threshold settings were used – a low threshold which yielded an 67% overall accuracy, with a very low false positive rate of 0.08%; a high threshold which gave a 96% overall cloud/shade classification accuracy with a slightly higher 2.3% false positive rate; and an intermediate setting which provides a blend between the other two models. Table 2: Summary of binary shading model forecast accuracy with varying threshold settings

Bias Avoiding Blackouts Intermediate Minimising fuel consumption

Threshold 260 700 5000

Correct (%) 67 80 96

False Positive (%) 0.08 0.30 2.30

7.0 Conclusions This paper presented the development of a short-term solar forecasting system using an inexpensive sky camera capable of accurately forecasting shading occurrences due to cloud movements, essential for accurate solar irradiance forecasting. Highlights include:  New approach to using dense optical flow to provide per-pixel cloud movement vectors, producing good results in complex cloud conditions.  Novel approach to accurate cloud classification, with results showing the applied method correctly classifying 95% of clouds in a large dataset  Binary shade forecasting model capable of producing a range of forecasts, from low false-positive rates of 0.08% to high overall accuracy of 96%.


References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19]

C. W. Chow, B. Urquhart, M. Lave, A. Dominguez, J. Kleissl, J. Shields, and B. Washom, “Intra-hour forecasting with a total sky imager at the UC San Diego solar energy testbed,” Sol. Energy, vol. 85, no. 11, pp. 2881–2893, Nov. 2011. A. Kazantzidis, P. Tzoumanikas, A. F. Bais, S. Fotopoulos, and G. Economou, “Cloud detection and classification with the use of whole-sky ground-based images,” Atmos. Res., vol. 113, no. 2012, pp. 80–88, Sep. 2012. S. Sayeef, S. Heslop, D. Cornforth, T. Moore, S. Percy, J. K. Ward, A. Berry, and D. Rowe, “Solar intermittency: Australia’s clean energy challenge,” 2012. P. Mathiesen, J. Kleissl, and C. Collier, “Characterization of Irradiance Variability using a High-resolution, cloud-assimilating NWP,” pp. 1–9, 2008. R. Perez, P. Ineichen, K. Moore, K. Marek, C. Chain, R. A. Y. George, and F. Vignola, “A New Operational Model for Satellite-derived Irradiances: Description and Validation,” vol. 73, no. 5, pp. 307–317, 2003. C. W. Chow, B. Urquhart, M. Lave, A. Dominguez, J. Kleissl, J. Shields, and B. Washom, “Intra-hour forecasting with a total sky imager at the UC San Diego solar energy testbed,” Sol. Energy, vol. 85, no. 11, pp. 2881–2893, Nov. 2011. R. W. Johnson, J. E. Shields, and T. L. Koehler, “Analysis & Interpretation of Simultaneous Multi-station Whole Sky Imagery,” 1991. D. Bernecker and C. Riess, “Towards Improving Solar Irradiance Forecasts with Methods from Computer Vision,” Pattern Recognit., 2012. J.-P. Thirion, “Image matching as a diffusion process : an analogy with Maxwell’s demons,” vol. 2, no. 3, pp. 243–260, 2004. B. Fischer and J. A. N. Modersitzki, “Curvature Based Image Registration,” pp. 81–85, 2003. B. Urquhart, C. Chow, A. Nguyen, and J. Kleissl, “Towards Intra-Hour Solar Forecasting using Two Sky Imagers at a Large Solar Power Plant,” San Diego, 2012. P. Wood-Bradley, J. Zapata, and J. Pye, “Cloud tracking with optical flow for short-term solar forecasting,” in 50th conference of the Australian Solar Energy Society, 2012. A. Hammer, D. Heinemann, E. Lorenz, and B. Lückehe, “Short-term forecasting of solar radiation: a statistical approach using satellite data,” Sol. Energy, no. 1, 1999. J. Konrad and E. Dubois, “Bayesian Estimation of Motion Vector Fields.” 1992. T. Hamill and T. Nehrkorn, “A Short-Term Cloud Forecast Scheme Using Cross Correlations,” Weather Forecast., vol. 8, no. 4, p. 401, 1993. R. Marquez and C. F. M. Coimbra, “Intra-hour DNI forecasting based on cloud tracking image analysis,” Sol. Energy, vol. 91, pp. 327–336, May 2013. J. Shields and M. Karr, “The whole sky imager–a year of progress,” … ARM) Sci. Team …, pp. 1–9, 1998. K. Stefferud, J. Kleissl, and J. Schoene, “Solar Forecasting and Variability Analyses using Sky Camera Cloud Detection & Motion Vectors,” pp. 1–6, 2012. B. Urquhart, C. W. Chow, A. Nguyen, J. Kleissl, M. Sengupta, J. Blatchford, and D. Jeon, “Towards Intra-Hour Solar Forecasting using Two Sky Imagers at a Large Solar Power Plant,” National Renewable Energy Laboratory (NREL), Golden, CO., 2012.
