Calibration of uncooled thermal infrared cameras

The calibration of uncooled thermal infrared (IR) cameras to absolute temperature measurement is a time-consuming, complicated process that significantly influences the cost of an IR camera. Temperaturemeasuring IR cameras display a temperature value for each pixel in the thermal image. Calibration is used to calculate a temperature-proportional output signal (IR or thermal image) from the measurement signal (raw image) taking into account all technical and physical properties of the IR camera. The paper will discuss the mathematical and physical principles of calibration, which are based on radiometric camera models. The individual stages of calibration will be presented. After start-up of the IR camera, the non-uniformity of the pixels is first corrected. This is done with a simple two-point correction. If the microbolometer array is not temperature-stabilized, then, in the next step the temperature dependence of the sensor parameters must be corrected. Ambient temperature changes are compensated for by the shutter correction. The final stage involves radiometric calibration, which establishes the relationship between pixel signal and target object temperature. Not all pixels of a microbolometer array are functional. There are also a number of defective, so-called “dead” pixels. The discovery of defective pixels is a multistep process that is carried out after each stage of the calibration process.


Introduction
In recent years, thermography has had a dramatic development with annual growth rates of over 20 % (Mounier, 2011).This development will have an even more dynamic impact in the future.Enabling this huge market success are infrared (IR) image sensors based on microbolometer arrays, which have excellent thermal and spatial resolution (Kruse, 1997;Vollmer and Möllmann, 2010).Also, because no cooling is required, they have low power consumption and have a low entry-level price.
Mainly due to the rapid development of micro-and nanotechnology, microbolometers have become significantly cheaper and more efficient.While early in 2000 the maximum image size was 320 × 240 pixels, nowadays commercial sensor arrays are available with a maximum size of 1024 × 768 pixels.IR image sensors with full HDTV resolution (Black et al., 2011) are already being advertised.The development trend is towards even smaller pixel grids (< 17 µm), to lower power loss and compact ceramic packages.While up until a few years ago, the stabilization of the temperature sensor was a requirement, nowadays current arrays manage without this.The use of microbolometers without temperature stabilization allows for compact, energy-efficient camera designs.
Both these components and the calibration process play an especially crucial role in ensuring the quality of an IR camera.The calibration is implemented both in the hardware and the software (firmware).For the two device concepts -vision display device and -temperature-measuring image device there are different calibration concepts (Budzier, 2014).
Vision devices display the measured radiation distribution of the scene qualitatively.They serve primarily as a night Published by Copernicus Publications on behalf of the AMA Association for Sensor Technology.vision device.This class of devices is produced in very large quantities.They are used in military applications, security technology and increasingly in automotive technology (night driving aid).The aim of calibration is to produce as closely as possible an optically flawless image.This process is referred to generally as "smooth out".
Temperature-measuring IR cameras also display a temperature value for each pixel.Here, after the "smooth out", a radiometric calibration must also be carried out.Practically every pixel of an IR camera is a separate pyrometer.The main problem is that the calibration parameters of microbolometers depend on both the ambient temperature and the camera temperature (Budzier and Gerlach, 2011).
In the following text the calibration of temperaturemeasuring, uncooled IR cameras will be described for microbolometer arrays1 .This is based on a radiometric camera model, which is described in Sect. 2. The individual steps of the calibration process will be discussed in the following sections.

Radiometric camera model
The theoretical basis for the calibration is based on a radiometric model of the thermal uncooled IR camera.In this case, the sensor array including the sum of radiant fluxes from the object and from inside the camera will be considered (Fig. 2).
A pixel "sees" inside the camera essentially the edge and the bracket (front panel) of the optics and its own sensor housing.In addition, the pixel gives out emissions in the entire half-space.The irradiance E P of the pixel (m, n) is with the radiance L O,mn of the object, L C,mn of the camera interior and L S of the pixel and the reduced solid angle of the object ω FOV,mn and the camera interior ω C,mn (= ω case,mn + ω Front,mn ).
The reduced solid angle ω FOV,mn of the object depends on the position of the pixel in the microbolometer and the f-number k of the optics used.For the central pixel and an f-number k = 1 applies (Budzier and Gerlach, 2011).
Since the pixel receives radiation from the entire halfspace, the reduced solid angle ω C for the central pixel of the camera is Thus, the centre pixel receives 4 times as much radiation from the camera interior as from the object!In off-centre pixels, the ratio is even worse (Budzier, 2014).
In order to determine the radiance L C of the object from Eq. ( 1), the radiance L C of the camera interior must also be known.Since each pixel sees different elements in the camera interior, the radiance L C must be measured with an optical shutter.For this purpose, the shutter, which has the camera temperature ϑ C , is closed.The irradiance E Sh with closed shutter now is (4) Assuming that the camera shutter represents the interior camera space (same mean temperature and simplified: L C = L Sh ), then from Eqs. ( 4) and (1) the irradiation intensity of the pixel through the object is obtained for the deviation between the shutter image and the measured object: For calculating the object temperature the sensor temperature is not required.Equation ( 5) is valid as long as the camera internal temperature and the sensor temperature are constant.
If there is a change in any one of these temperatures then the shutter must be activated again.The raw image V P,mn is then obtained by multiplying the irradiance E O,mn with the voltage sensitivity R V and the pixel area A P : In the above considerations, a single DC (direct current) bias (offset mn ) and a uniform sensitivity (gain mn ) were assumed for all pixels.Due to the complex manufacturing technology, that is not the case.While the sensitivity can vary by up to ±20 %, the deviation of the offset lies in the range of the signal.Due to the large offset differences between the pixels it is often the case that no signal is visible in the raw image.These differences are eliminated by a conventional two-point correction and apply to the IR image V O,mn : V O,mn = V P,mn − V Sh,mn − offset mn * gain mn , with the pixel-specific variables offset mn and the two-point correction gain mn .For this purpose, when the shutter is closed, an image is stored in an image memory and then subtracted from the current online IR image (Fig. 3).In order to save an arithmetic operation per pixel in real time, the offset and shutter correction are thereby summarized.Modern microbolometer arrays do not contain Peltier elements in a vacuum housing.These are called TEC-less microbolometers (TEC: thermo-electric cooler).The microbolometer is no longer stabilized to a constant temperature, i.e. the sensor temperature ϑ S varies with the temperature ϑ K of the camera.However, since the sensitivity and the operating point (offset) of a pixel are dependent on the sensor temperature, this must therefore be measured and taken into account in the camera model, as opposed to temperaturestabilized microbolometers.
The dependence of the offset and the sensitivity of a microbolometer array cannot be derived from the physical properties of a bolometer resistance without information concerning the signal processing.Since the internal signal processing of a microbolometer array is not known in detail for reasons of company in-house security, the array must be regarded as a black box.In general, the following polynomials can be assumed for the temperature dependence of the offset O V and the sensitivity G V : The sensitivity and the offset of the pixel must therefore be corrected using the temperature sensor ϑ S (Fig. 4): (10)

Calibration
A radiometer, where each individual pixel of an IR camera can be determined, measures the radiant flux of the object and generates an output signal which, as a result of the calibration, is proportional to the temperature of the object (DeWitt and Nutter, 1989).A radiometric IR camera displays as accurately as possible the true temperature of a black body.The calibration is used here to calculate a temperatureproportional output signal (IR image) from the measurement signal (raw image) taking into account all technical and physical properties of the IR camera.The steps necessary for this are summarized in Fig. 5.
In the following section, the non-uniformity correction (Sect.3.1), the temperature-dependent correction (Sect.3.2), the defective pixel correction (Sect.3.3), the shutter correction (Sect.3.4) and the radiometric calibration (Sect.3.5) will be presented in detail.There will be no further discussion of the operating point setting which depends significantly on the microbolometer used and would correspond to the example provided by the manufacturer's procedure.

Non-uniformity correction
Because of the technology, the individual pixels of a microbolometer have uniquely different operating points (DC bias values) and sensitivities and, thus, differing characteristics.During the correction of this non-uniformity all pixels are converted onto a single characteristic curve, the so-called standard characteristic curve.This process is referred to as "smooth out" because the IR image now with uniform illumination has no structure and so is smooth.According to this characteristic adjustment, all pixels behave the same and subsequent calibration steps can be exemplarily performed on any pixel or on any group of pixels.The description of the pixel characteristic curve is often a function of the object temperature ϑ O .The function U mn (ϑ O ) of the pixel at location (m, n) is not linear.Therefore, the characteristic curve is generally described with a secondorder polynomial (Schulz and Caldwell, 1995).For radiometric IR cameras this regression is not sufficient.Here, an exponential regression (Horny, 2003) is provided, which will be described in Sect.3.5.
The characteristic curve corrections described in the literature (Schulz andCaldwell, 1995, andWallrabe, 2001) also refer to photon sensors, whose function U mn ( O ) is often not linear, as well as IR vision equipment without radiometric adjustment.In contrast, the relationship between the radiant flux and the pixel voltage U mn of a microbolometer is linear and can be used for describing a characteris-Figure 6.Schematic of the two-point correction procedure, using the example of two pixels.Dashed curves: the value "offset" is shifted parallel to the standard curve.tic curve.A significant simplification of the correction algorithm is achieved by the subsequently described separation of non-uniformity correction and radiometric calibration.This means that the smooth out of the image is performed first and then subsequently the temperature of connectivity of the now common pixel characteristic curve.This approach also leads to a reduction of the computational effort and correction allows for ease of data processing in real time.
Firstly, it is assumed that both the sensor temperature and the ambient temperature are constant.
The linear relationship between the voltage of the pixel U mn and the radiant flux can be described with a linear equation: with the slope of the straight line represented by a mn and the intercept by b mn .Slope and intercept behave pixel-specific and must be calculated so that a standard characteristic curve is obtained for all pixels: where a norm is the slope and b norm is the intercept.For each pixel (m, n), a constant pair offset mn and gain mn must be determined so that U korr,mn = (U mn − offset mn ) * gain mn + U normoffset (13) applies.The voltage U korr,mn is then corrected, i.e. on the standard curve traced back to the voltage value of the pixel.
Figure 6 shows the principle of this two-point correction procedure.First, the pixel graph is shifted in parallel (−offset mn ) and then the slope is corrected (*gain mn ).Finally, for all pixels a valid constant voltage can be added.The standard curve is thus shifted back (+U normoffset ).This last step is not necessary in every case but guarantees that the corrected pixel voltages are in the same range of values as the measured pixel values.This is important, for example, if the correction is implemented in hardware (16 bit fixed-point arithmetic) and the range of values is limited.
In order to reduce the calculation during real-time correction, Eq. ( 13), can be simplified to To determine the standard curve and the pixel-related coefficients, all pixel values U mn must be measured with T 2 > T 1 on two radiant fluxes: (T 1 ) and (T 2 ) (Eq. 12).Then the coefficients of the standard curve (Eq.12) may be initially calculated: with the mean pixel voltage U k (k = 1, 2) as the average value for all pixels J : The standard curve is now the mean curve for all pixel characteristics.Each pixel now deviates at the measurement point k with U mn,k from the mean: Using the method of least squares, a function, can now be determined so that all voltage differences U mn,k are minimal.Using a simple regression the pixel-specific coefficients from Eq. ( 20) can now be obtained: The corrected pixel voltage U korr,mn is then calculated from the current measured value U mn of the pixel and the differential voltage U mn : The values for Eq. ( 15) result from a comparison of the following coefficients: offset * mn = −d mn . (25) Due to various measurement errors, in particular the temporal noise of the pixel voltage, the determination of the coefficients is faulty.This shows that a small deviation remains between the pixels in the corrected image.This can be interpreted as spatial noise.Now, in order to assess the quality of the correction, the corrigibility parameter C was introduced by Gross et al. (1999) and Horny (2003): with the noise voltages U TN and U SN for the temporal noise and spatial noise, respectively.The corrigibility C is ideal for correcting the characteristics of 0. It is equal to 1 if the spatial noise has the same value as the temporal noise.If the corrigibility is greater than 1, then the spatial noise dominates and the non-uniformity is clearly visible in the IR image.

Temperature dependence correction
If the temperature of the microbolometer is not constant, as is the case for TEC-less microbolometers, the sensor sensitivity and offset parameters change with temperature (Eqs.8-10).
To correct this temperature dependence, the polynomials of Eqs. ( 8) and ( 9) must be determined.For this purpose, the pixel voltages U P must be measured at various ambient temperatures ϑ A .The pixel voltage is the sum of the pixel offset U S , the signal from the camera interior U C and the constant object voltage U O : It is important that both the camera temperature ϑ C and the sensor temperature ϑ S can be distinguished from the ambient temperature ϑ A .They are always higher by a few kelvin.If the ambient temperature changes, then the camera and sensor temperatures change, in fact, with different time constants.This makes them distinguishable in the output signal.The temperature dependence of the pixel offsets U S (ϑ S ) is given by Eq. ( 8).The signal voltage U C , resulting from the temperature ϑ C of the camera interior, is calculated with a quadratic polynomial: where the polynomial coefficients a C , b C and c C are initially unknown.If a measurement is made at the point in time t i with the temperatures ϑ C and ϑ S i , then, by superposition of Eqs. ( 27) and ( 28), By making a series of measurements with i = 1, 2, . .., i.e. measurements under varying ambient temperatures, then the following vectors are obtained: This series of measurements is taken with three different object temperatures, but they are constant in each case.In the IR image three emitters are thus shown with different temperatures ϑ O1 , ϑ O2 and ϑ O3 : with n = 1, 2, 3, and the vectors The series of measurements can now be represented in matrix notation: with The vector X includes the desired polynomial coefficients.The solution to this over-determined system of equations with I 3 can be calculated with the Gaussian standard equation: So now the desired polynomial coefficients are known, in particular the sensor offsets o 3 , o 2 and o 1 .The missing constant o 0 can be chosen arbitrarily.When the calculated value is subtracted from the sensitivity corrected signal according to Eq. ( 8), the camera will behave like a camera with temperaturestabilized microbolometers.

Defective pixel correction
Due to the difficult manufacturing process for microbolometer arrays, all pixels have different parameters such as operating points, characteristic curves and noise.Pixels that either do not work or whose parameters vary greatly from the mean are defined as non-functional or defective.Defective pixels are generally referred to as "dead" pixels.Pixel defects manifest themselves as defective pixels in the IR image.Their actual value can only be estimated with the help of neighbouring pixels.The measured value at this point of the IR image is not reconstructable.Therefore, the number of dead pixels is an important quality characteristic of microbolometers.Normally not more than a maximum of 1 % of all pixels should be defective.
A pixel is considered defective if any of the following conditions is met.
-The operating point is outside of the previously defined voltage range U AP of the offset value dispersion.
-The sensitivity differs more than ±10 % from the mean value.
-The noise voltage is 1.5 times greater than the average noise voltage of the array.
In addition, a group of defective pixels exist, which, although they do not meet the above criteria, behave differently and are classified as defective.These are, for example, short circuits between adjacent pixels or non-linear characteristics of individual pixels.Figure 7 shows a raw image with a plurality of defective pixels (black dots).Defective pixels occur not only individually but also in clusters.A cluster of dead pixels is a group of at least two defective pixels that are adjacent or gather together in a corner.Clusters are characterized by their size, that is, by the number of defective pixels.In the image section in Fig. 7b clusters are clearly visible.Particularly critical are defective rows or columns, because, despite a correction in the IR image, they are always conspicuous.A column or row is usually considered defective if more than 50 % of the pixels do not work.
Since the number of defective pixels of a microbolometer is an important quality attribute, the manufactures will  always indicate in their specifications the maximum permissible number of defective pixels.In the centre of an IR image defective pixels are particularly noticeable.Therefore, the image area is divided into at least two zones (Fig. 8).
In the central region (zone A) higher demands are placed on the functionality of pixels than of those on the edges.Table 1 shows a specification of permissible defective pixels.The detection of defective pixels proceeds in three steps (flow chart in Fig. 5).
1.The first defective pixel detection must be done before the uniformity correction.Here all defective pixels are detected that are located outside of the previously defined range of variation of the pixel operating points.These are primarily pixels which are outside the con-trol range and, thus, affect the position of the standard curve.Furthermore, all pixels which are too noisy are eliminated.
2. The second defective pixel detection is performed using the calculated gain and offset values according to the characteristic curve correction.Here defective pixels are identified that have too great a deviation from the standard curve.
3. The third defective pixel detection is carried out at the end of the calibration.In this case, all defective pixels which have not yet been detected are recorded.This is done by considering the IR image with different adjustments and richly contrasting scenes.
While the first two defective pixel detections can be performed computationally, the final detection is carried out manually.This also means to verify the correction of the cluster and, if necessary, to change the correction method.
There is no reading at the location of a defective pixel.This can only be estimated from the surrounding area.The aim of the recalculation of the pixel value is always to produce a high-quality visual image, i.e. so that an observer of the IR image may not notice any defective pixels.The calculation of the pixel value is carried out by methods of image preprocessing, such as with median operators.

Shutter correction
In order to calculate the radiance and the object temperature from Eq. ( 5), the radiance L C of the camera interior must be known.Should the camera temperature ϑ C vary as a result of a change in the ambient temperature, the radiance value of the camera interior needs to be adjusted accordingly.This process is called shutter correction.
The starting point for consideration is that the radiance of the camera interior space was determined using a known ambient temperature and thus a known camera temperature ϑ C0 (ϑ C0 = ϑ Sh0 ); therefore, according to Eq. ( 5) the object radiance and the object temperature can be calculated.
with the voltage equal to When ω C U C is known, then the ambient temperature correction can be carried out: The voltage U O in Eq. ( 42) is now independent of the ambient temperature as well as the camera temperature and is used to calculate the object radiance L O .
The projected solid angle ω C is known theoretically.With an f-number k = 1, then, for the centre pixel is obtained.
For each pixel the voltage U C is determined with the aid of shutter correction.For this purpose, the shutter is closed and the pixel voltages (shutter image U Sh ) are measured.When the shutter is closed, the pixel sees the entire half-space (ω c = π ): Therefore, the shutter signal U Sh must be multiplied by a factor a sh .This results from the so-called shutter characteristic curve.The shutter characteristic curve is the ratio of the shutter-open signal U open to shutter-off signal U Sh : It follows that with The projected solid angle ω C of the camera interior can be read from Eq. ( 46): with the increase a Sh in the shutter curve.Now the signal voltage of the object can be calculated independently of the ambient temperature using the shutter image:

Radiometric calibration
The previous corrections in Sect.3.1 and 3.4 lead to all pixels of the IR image having the same behaviour and the IR image not being dependent on the ambient temperature.Finally, the radiometric calibration calculates the temperature of the object to be measured from the grey values of the pixel.It works with a voltage object temperature characteristic, so for each grey value U D a temperature value T O is assigned.According to Horny (2003) it is possible to approximate the sensor output signal with a Planck curve: where in B, F , O and R are the regression coefficients to be determined.Then the inverse function from Eq. ( 50) serves to calculate the object temperature T O : The coefficients allow for a physical interpretation.The value of O is a general offset.Using this value, the characteristic curve along the ordinate can be displaced in parallel.The coefficient R represents the system response of the IR camera and is the counterpart to the system sensitivity.Planck's radiation law can be applied for the coefficient B: It thus describes the spectral behaviour of the system.The wavelength λ B is the effective wavelength of the IR camera.The coefficient F allows for an alignment of the non-linearity of the system.The coefficients can be determined using a non-linear regression analysis.

Example of application
In the following, the calibration process will be illustrated by an example.The calibration process begins with the setting of the operating point.Figure 9 shows a histogram of the raw image with optimized operating point (dynamic range of −32 000 to +32 000 grey values).To demonstrate the modulation, a black body with a temperature of 150 • C has additionally been shown.
After the operating point has been fixed, in the first defective pixel detection procedure, all pixels which lie outside of the dynamic range are defined as defective.Figure 10 shows the effect of a two-point correction of the pixel graph, in which the variation of the values of the individual pixels occurring in the raw image (Fig. 10a) is eliminated.The recognizable characteristic stripe structure is formed by the column-wise arrangement of "blind" bolometers.The optical image resulting from the natural vignetting of the optical signal and delivered to the image edge is also always present in raw image where in Fig. 10a it is hardly recognizable by the content of the thermal image.
By analysing the offset and gain values calculated in the two-point correction (Fig. 11), in a second defective pixel detection procedure more abnormal pixels that lie outside a defined scatter band can be sorted out.Fixed limits of ±20 % have been proven here by the gain values.
For TEC-less microbolometers, the temperature dependence of the offset values and the sensitivity has now to be determined (Fig. 12).The regression analysis includes the known relationships between camera or sensor temperature and the respective measured variables.These are for the sensitivity of a polynomial of the second order (see Eq. 9) and the offset values for a third-order polynomial (see Eq. 8).
After correction of the non-uniformity, the correction of the ambient temperature dependence is carried out.The socalled shutter characteristic curves are recorded, which represent the ratio of the shutter-open signal to shutter-off signal as a function of the temperature of the camera (Fig. 13).For this, the camera temperature (in Fig. 13 from 3.9 to 47.3 • C) varies and in each case the signal V open of the object and the shutter V sh is measured in a temperature chamber.Both signals are directly proportional to each other.
Subsequently, the radiometric calibration is carried out by determining the signal voltage V D object temperature characteristic (Fig. 14).The inclusion of this characteristic is done with a black body as a measuring object.Its temperature T O will vary within the measurement range, e.g. from −10 to 150 • C. With the recorded measurement points, a regression is carried out in accordance with Eq. ( 50).
Finally, a third defective pixel detection takes place during the final check.Here visually conspicuous pixels are rejected by visual inspection.

Summary
The calibration of an uncooled IR camera is a complex and lengthy process, which significantly affects the cost of an IR camera.Depending on the measurement technology used, such as black bodies, references and climatic chambers, the proposed calibration allows for the measurement of absolute temperatures with a maximum measurement uncertainty of about ±1 K.This is only true for steady ambient temperatures, i.e. when the camera is at a constant temperature.When changing the ambient temperature a shutter cycle is always required.With sudden changes of the camera temperature between two shutter cycles, e.g. as a result of a jump in the ambient temperature, strong variations in measurements can appear.The behaviour of the IR camera is not ergodic.An additional measurement of the ambient temperature is not usually possible.In commercial IR cameras, this problem is solved by predictive models which estimate an expected camera internal temperature from the previous temperature change of the camera.Solutions to these problems are not known or published.
For rapid changes in temperature of the camera, the shutter needs to be frequently operated, e.g.several times within 1 min.Since the operation of the shutter always causes an interruption of the measuring process, the user should operate the shutter as little as possible or even completely avoid using it.For the shutterless operation of IR cameras, however, much more complex calibration algorithms are required, which build on those described here.Such an approach is pursued by Tempelhahn et al. (2014).
Edited by: R. Morello Reviewed by: three anonymous referees

Figure 1 .
Figure 1.Structure of an uncooled IR camera with microbolometers.

Figure 3 .
Figure 3. Camera model for the calculation of the IR image (explanation in text).

Figure 4 .
Figure 4. Extension of the camera model from Fig. 3 for the calculation of the IR image for TEC-less microbolometers.

FormatFigure 8 .
Figure 8.Typical subdivision of infrared microbolometer arrays in the mid (zone A) and marginal zones (zone B).

Figure 9 .
Figure 9. Histogram of the raw image at the operating point showing a black body with a temperature of 150 • C (microbolometer with 384 × 288 pixels).

Figure 10 .
Figure 10.Effect of two-point correction.(a) Raw image (colour bar chart in grey values) and (b) thermal image (colour bar chart in • C).

Figure 11 .
Figure 11.Histograms of (a) the offset values and (b) the gain values of the microbolometer arrays under consideration.

Figure 12 .
Figure 12.Temperature dependence of (a) the offset values and (b) the sensitivity of the observed TEC-less microbolometer arrays.

Figure 13 .
Figure 13.Shutter characteristic curve.Parameter: camera temperature of the respective measurement points.

Table 1 .
Specification of allowable defective pixels.