The application of optimal estimation to lidar
RSPSoc - NCEO - CEOI-ST Joint Conference Centre for Instrumentation (2015)
Abstract:
Lidars are ideally placed to investigate the effects of aerosol and cloud on the climate system due to their unprecedented vertical and temporal resolution. Dozens of techniques have been developed in recent decades to retrieve the extinction and backscatter of atmospheric particulates in a variety of conditions. These methods, though often very successful, are fairly ad hoc in their construction, utilising a wide variety of approximations and assumptions that makes comparing the resulting data products with independent measurements difficult and their implementation in climate modelling virtually impossible. As with its application to satellite retrievals at the turn of the century, the methods of non-linear regression can improve this situation by providing a mathematical framework in which the various approximations, estimates of experimental error, and any additional knowledge of the atmosphere can be clearly defined and included in a mathematically `optimal' retrieval method, providing rigorously derived error estimates. In addition to making it easier for scientists outside of the lidar field to understand and utilise lidar data, it also simplifies the process of moving beyond extinction and backscatter coefficients and retrieving microphysical properties of aerosols and cloud particles. A technique to estimate the lidar's overlap function using an analytic model of the optical system and a simple extinction profile has been developed. This is used to calibrate the system such that the profile of extinction and backscatter coefficients can be retrieved using the elastic and nitrogen Raman backscatter signals. These methods have been used to extract value from compromised data collected with a prototype Raman lidar system. Selected events will be presented, with the hope that others may be inspired to apply the techniques to a more robust system.Parallel retrieval of aerosol and cloud
ATMOS 2015 European Space Agency (2015)
Abstract:
Due to similarities in their radiometric signature, it is not possible to retrieve aerosol and cloud properties simultaneously from satellite imagery. A plethora of filtering techniques have been developed to ensure aerosol and cloud are analysed separately, but this neglects the scientifically interesting regions of interaction between the two. It also limits the spatial coverage of such products, with up to 20% of the planet neglected because it is considered too cloudy to be suitable for an aerosol retrieval but insufficiently so for a cloud retrieval. The Optimal Retrieval of Aerosol and Cloud (ORAC) is a single algorithm that can retrieve the aerosol or cloud properties consistent with a single measurement. By performing radiative transfer calculations via look-up tables, various types of particle can be considered in parallel — such as liquid-phase cloud, different models of ice nuclei, and various clean and polluted aerosols — by simply running the program repeatedly using tables assuming different microphysical properties and vertical distributions. Bayesian statistics can determine the probability that the scene contains a specific species, classifying it as aerosol, cloud, or uncertain. The important but infrequently discussed `uncertain' region can then be used to investigate the impact of contamination and data coverage on existing products by, for example, observing how retrieved aerosol optical thickness varies as a function of the distance from the nearest cloud. It also provides a potential window for the study of aerosol-cloud interactions.Retrieval of aerosol backscatter, extinction, and lidar ratio from Raman lidar with optimal estimation
Atmospheric Measurement Techniques European Geosciences Union 7:3 (2014) 757-776
Abstract:
Optimal estimation retrieval is a form of nonlinear regression which determines the most probable circumstances that produced a given observation, weighted against any prior knowledge of the system. This paper applies the technique to the estimation of aerosol backscatter and extinction (or lidar ratio) from two-channel Raman lidar observations. It produces results from simulated and real data consistent with existing Raman lidar analyses and additionally returns a more rigorous estimate of its uncertainties while automatically selecting an appropriate resolution without the imposition of artificial constraints. Backscatter is retrieved at the instrument’s native resolution with an uncertainty between 2 and 20 %. Extinction is less well constrained, retrieved at a resolution of 0.1–1km depending on the quality of the data. The uncertainty in extinction is >15 %, in part due to the consideration of short 1 min integrations, but is comparable to fair estimates of the error when using the standard Raman lidar technique. The retrieval is then applied to several hours of observation on 19 April 2010 of ash from the Eyjafjallajökull eruption. A depolarising ash layer is found with a lidar ratio of 20– 30 sr, much lower values than observed by previous studies. This potentially indicates a growth of the particles after 12– 24 h within the planetary boundary layer. A lower concentration of ash within a residual layer exhibited a backscatter of 10Mm−1 sr−1 and lidar ratio of 40 sr.The application of optimal estimation retrieval to lidar observations
(2013)
Abstract:
Optimal estimation retrieval is a nonlinear regression scheme to determine the conditions statistically most-likely to produce a given measurement, weighted against any a priori knowledge. The technique is applied to three problems within the field of lidar data analysis. A retrieval of the aerosol backscatter and either the extinction or lidar ratio from two-channel Raman lidar data is developed using the lidar equations as a forward model. It produces profiles consistent with existing techniques at a resolution of 10-1000 m and uncertainty of 5-20%, dependent on the quality of data. It is effective even when applied to noisy, daytime data but performs poorly in the presence of cloud. Two of the most significant sources of uncertainty in that retrieval are the nonlinearity of the detectors and the instrument's calibration (known as the dead time and overlap function). Attempts to retrieve a nonlinear correction from a pair of lidar profiles, one attenuated by a neutral density filter, are not successful as uncertainties in the forward model eliminate any information content in the measurements. The technique of Whiteman et al. [1992] is found to be the most accurate. More successful is a retrieval of the overlap function of a Raman channel using a forward model combining an idealised extinction profile and an adaptation of the equations presented in Halldórsson and Langerholc [1978]. After refinement, the retrieval is shown to be at least as accurate, and often superior to, existing methods of calibration from routine measurements, presenting uncertainties of 5-15%. These techniques are then applied to observations of ash over southern England from the Eyjafjallajökull eruption of April 2010. Lidar ratios of 50-60 sr were observed when the plume first appeared, which reduced to 20-30 sr after several days within the planetary boundary layer, indicating an alteration of the particles over time.Estimation of a lidar's overlap function and its calibration by nonlinear regression
Applied Optics 51:21 (2012) 5130-5143