Fusion-based simultaneous estimation of reﬂectance and illumination for low-light image enhancement

Low-light image enhancement is a challenging ﬁeld in image processing. Retinex-based methods perform well for low-light images. However, reﬂectance and illumination estimation is an ill-posed problem. This paper presents a new framework for the simultaneous estimation of reﬂectance and illumination for low-light image enhancement. The algorithm estimates multiple instances of illumination and reﬂectance and blends them to estimate the ﬁnal components. The proposed approach uses multi-scale fusion for illumination estimation and naive fusion for reﬂectance estimation. Extensive experimentation and analysis with a large set of low-light images validates the performance of the proposed approach. The comparison shows the superiority of the proposed approach over most of the existing low-light image enhancement methods. The proposed method provides colour constancy in low-light image enhancement and preserves the naturalness of the image.


INTRODUCTION
Image enhancement (IE) is crucial in designing robust and efficient vision-based systems. The quality of input image may have a notable outcome on the performance of a visionbased system. IE techniques try to ensure satisfactory quality of the inputs for vision-based systems. If a vision-based system is operating in a weakly illuminated environment, IE algorithms become a necessity. Histogram equalisation (HE) [1] is a straightforward approach to IE. HE performs IE using frequency of intensities of the input image. HE-based approaches focus on the improvement of the dynamic range in an image. Abdullah et al. [2] developed dynamic HE (DHE), which divides the histogram based on local minima. DHE focuses on maintaining the dynamic range of output proportional to the input image. Parihar and Verma [3] proposed entropybased DHE, which divides histogram optimally with the help of entropy. Celik et al. developed an approach named contextual and variational contrast enhancement (CVC) [4]. CVC uses 2D-histogram to integrate local properties of an image for the preservation of natural characteristics. Parihar et al. [5] developed fuzzy contextual contrast enhancement (FCCE), which constructs fuzzy-difference histogram using fuzzy characteristics of an image. However, these algorithms deal with the con- trast of the images instead of their illumination. Thus, these algorithms fail to enhance low-light images. Land [6] introduced the Retinex theory, which considers an image as the product of reflectance and illumination. Retinexbased approaches deal with the illumination of the image. Thus, Retinex-based approaches perform reasonably superior to other methods for enhancement of low-light images. The conventional Retinex-based approaches [7][8][9] consider the centre/surround function for illumination estimation. Jobson et al. [7] presented single-scale Retinex (SSR), which considers Gaussian-filtered image as illumination. In SSR, there is a trade-off between compression of dynamic range and colour constancy. Barnard et al. [10,11] have shown that colour constancy can resolve illuminant variant problem. To overcome the drawback of SSR, Jobson et al. [8] proposed multi-scale Retinex (MSR). MSR uses numerous scales of Gaussian to provide compression of dynamic range and colour constancy simultaneously. MSR performs reasonably well while dealing with greyscale images. Jobson et al. [9] developed MSR with colour restoration (MSRCR) to provide reasonably superior results for colour images. In [12,13], Parihar and Singh discussed various Retinex-based IE approaches. However, the algorithms mentioned above contemplate reflectance to be the enhanced image, which sometimes generates unnatural results erratically.
Various researchers endeavoured the estimation of reflectance and illumination simultaneously. It is ill-posed to estimate reflectance and illumination from an image. Researchers composed priors for estimation of reflectance and illumination. Fu et al. [14] formulated probabilistic prior and presented an approach for the computation of reflectance and illumination simultaneously for image enhancement (PIE). Moreover, Fu et al. [15] proposed a weighted simultaneous estimation (WSE) approach to compute the reflectance and illumination components. Both PIE and WSE perform well in estimating the reflectance and illumination. However, both methods are unable to achieve a significant enhancement for low-light images. Li et al. [16] presented an approach to estimate reflectance, illumination, and noise simultaneously from an image (SRIE). Hao et al. proposed semi-decoupled decomposition (SDD) [17] that considers the inherent noise in the reflectance. SRIE ans SDD provides appropriate colour constancy. However, they generate over-smoothing in the results at times. The computational cost of the above algorithms for simultaneous estimation is high. Thus, these algorithms may not be effective in real-life applications.
Ren et al. [18] presented CRM method for low-light IE using camera response model. Ghosh and Chaudhury [19] developed a low-light IE using fast bright pass filter. Srinivas and Bhandari proposed ATSF [20] for low-light IE, which uses an adaptive sigmoid function for improving the lightness of the image. The researchers [14][15][16] achieved low-light IE with the help of estimation of reflectance and illumination from an image (i.e. simultaneous estimation). However, the majority of the simultaneous estimation-based methods experience one or more of the following limitations: • improper enhancement in darker regions of low-light images; • information loss in the enhanced image, due to oversmoothing; • high computational complexity, thus ill-suited for preprocessing in vision-based systems; • inefficient stopping criteria for iterative algorithms.
This work presents a new algorithm for enhancement of lowlight images. The approach endeavours the simultaneous estimation of reflectance and illumination. The proposed algorithm estimates illumination using a novel approach of multi-scale fusion. Further, it computes the reflectance using naive fusion. The notable contributions of the proposed algorithm are as follows.
• Proposed a new fusion-based framework for simultaneous estimation of illumination and reflectance components. To the best of our knowledge, it is the first approach which performs fusion-based simultaneous estimation of both illumination and reflectance components for enhancement of low-light image. • Proposed an approach for estimation of illumination utilising bilateral filtering and multi-scale fusion. Leveraging the concept of the bilateral filtering, we propose a new illumination component with structural details. Further, performed multi-scale fusion with appropriate instances of the illumination which helps in maintaining the natural variation in the illumination. • Proposed a new naive fusion-based approach for reflectance estimation. To the best of our knowledge, the work proposed a new method of reflectance estimation leveraging fusionbased approach. In addition, naive-fusion is used for the first time for reflectance estimation. • In fusion-based approaches, selection of appropriate components is an ill-posed problem. We proposed appropriate components for both illumination and reflectance estimation.
In the remaining paper, the content is organised as follows. Section-II incorporates related work, while Section-III presents an elaboration of the proposed approach. Section-IV provides results and analyses, and Section-V concludes the proposed approach.

RELATED WORK
This section contains a brief discussion about the Retinex theory and image fusion. Retinex-based image formation model considers illumination and reflectance for image formation.
On the other hand, fusion-based methods endeavour to blend images based on desired features.

Retinex theory
Land [6] presented the Retinex theory, which considers an image as the product of reflectance and illumination. According to Retinex theory, an image I is given as where R denotes reflectance, T denotes illumination of the image, and • is the element-wise product. The Retinex-based approaches require the estimation of R, and T component, which is an ill-posed problem. In [7], Jobson et al. proposed a prior for illumination estimation, which can facilitate the computation of reflectance. Thus, to estimate reflectance using illumination, (1) can be reframed as where t 0 restricts the lower bound of the illumination. Various Retinex-based approaches [21,22] estimate illumination component, and then compute reflectance by using (2). The enhancement is achieved by refining the illumination and combining it with the reflectance component. Fu et al. [21] developed an algorithm (FWE) for the enhancement of weakly illuminated images by utilising Retinex theory. FWE estimates reflectance with the help of estimated illumination. Further, FWE refines illumination based on the multi-scale fusion of multiple instances of the input image, as discussed in section 2.2. Guo et al. [22] presented an approach for enhancement of low-light images by utilising map estimation (LIME). LIME computes illumination using the pixel-wise maximum operation on an image. Furthermore, LIME refines the illumination with the help of multi-objective optimisation. LIME performs significantly well for low-light images. However, it results in over-enhancement of low-light images. Moreover, LIME fails in providing proper colour constancy.

Image fusion
The utility of image fusion lies in the fact that it effectively combines the properties of different techniques in a way such that the desired properties are included in the result. Image fusion uses different kinds of weighting techniques to perform the selection of candidate pixel values with better features. Fusion can be expressed as where A denotes output, B denotes input, W denotes the weight which can be in scalar or vector form as per requirement, and n denotes the number of instances for blending. Naive fusion uses the direct approach, as shown in (3), whereas multi-scale fusion utilises pyramids to blend the images. Multi-scale fusion involves the use of Gaussian and Laplacian pyramids. Burt and Adelson [23] proposed multi-scale fusion for the encoding of images. Burt and Kolczynski [24] further developed the approach to utilise it for a weighted fusion of images. Various researchers have utilised multi-scale fusion for IE [21,25], high dynamic range (HDR) imaging [26], colour balance [27], and other domains. Mertens et al. [26] developed a method for blending low dynamic range images to generate an HDR image using multi-scale fusion. Liu et al. [25] presented a method for detail-preserving IE based on multi-scale fusion by generating multiple instances from an image. In FWE, Fu et al. [21] attempted to estimate illumination with the aid of multiscale fusion. FWE derives multiple instances from an input image, and for each instance, the algorithm generates a weight map with the help of an energy function. After finding multiple instances and their respective weight maps, the algorithm utilises multi-scale fusion with the aid of Gaussian and Laplacian pyramids. Further, FWE performs upscaling of illumination to improve lightness in the image and combines it with reflectance to generate the enhanced image.

PROPOSED APPROACH
This section presents the proposed framework for low-light IE. The first subsection presents the motivation, and following subsections contain the detailed description of the proposed fusion-based simultaneous estimation of reflectance and illumination for low-light IE approach.

Motivation
Retinex-based approaches perform well for low-light IE using illumination and reflectance. However, the estimation of reflectance and illumination components is a challenging task. The characteristics of illumination of an image depend on the source of luminance. The characteristics of illumination in daylight and low-light images vary due to different sources of luminance. Sunlight provides uniform illumination in most of the daylight images. In general, low-light images suffer from nonuniform illumination due to distinct sources of luminance in different regions. For a good quality image, the illumination should be uniform throughout the image. However, there may be natural variation in illumination due to objects and details present in the image, as is shown in Figure 1(a). If the algorithm ignores the natural variation in illumination, the resultant image may have over-enhanced regions and artefacts, as shown in Figure 1(b). The variation in illumination may be modelled by structural information of an image. Thus, a good estimate of illumination must preserve structural information while performing smoothing of the fine details. Low-light images can have relevant information about illumination in any colour channel based on the illumination of the specific region. In most low-light images, a pixel-wise maximum of all colour channels (max-channel) can help in capturing maximum illumination relevant information. A set of approaches [21,22,28] have presented max-channel as the initial estimate of illumination. However, max-channel may fail to represent actual illumination for images with varying illumination. An image represents a 3D view in a 2D perspective. Hence, the objects appearing adjacent to each other in an image may be way apart in reality. In such situations, illumination of those objects varies in reality. Nevertheless, two vicinal objects may contain similar maximum illumination, and max-channel may therefore not determine their illumination [28]. Researchers [22,28] have presented structure-preserving smoothing as the illumination estimation in low-light images. Structure-preserving smoothing performs comparatively better for illumination estimation. As discussed earlier, simple smoothing of the image details fails to capture natural variation (non-uniformity) in the illumination and thus causes over-enhancement. However, inappropriate structure-preserving smoothing can lead to the generation of artefacts and degraded images. There is a fine boundary between structural details and textural details of an image. Suppressing (or enhancing) one of these may result in suppression (or enhancement) of other, which is undesirable. Thus, it is not easy to achieve appropriate structure-preserving smoothing. The proposed approach employs bilateral filter [29] for nonuniform illumination estimation. Bilateral filtering performs smoothing while preserving image details. However, a bilateral filter sometimes retains few textural details while attempting structural preservation. Thus, estimated illumination using bilateral filter deviates from a good estimation. The importance of textural details suppression inspires us to use the inverse gradient [28] for illumination estimation. The inverse gradient performs smoothing by suppressing the image gradient information, i.e. textural and structural details. However, the quality of estimated illumination may be compromised due to the suppression of structural information. As we discussed, since there is a fine boundary between textural and structural details, an effective suppression of textural details becomes an ill-posed problem. Therefore, the notion of fusion-based illumination estimation may provide an estimation, which is close to actual illumination. In fusion, it is important to select the right instances in adequate number. Fusion of improper instances affects enhancement and provides poor contrast. Figure 7 shows that result of FBE which is unable to achieve appropriate lightness and contrast in the results. The proposed approach achieves textural suppression with the help of inverse gradient. To further enhance the smoothing of image details, we use Gaussian filtering of the max-channel. After estimation of the instances, the proposed approach performs multi-scale fusion to generate illumination by smooth blending of all instances. Multi-scale fusion blends images smoothly and effectively with almost no artefacts.
The IE depends on the estimation of both reflectance and illumination components. Thus, reflectance estimation is again a key task. The characteristics of reflectance depend on the objects and their properties. The reflectance provides information regarding colour, contrast and structure of the objects in an image. However, the reflectance component does not have any established priors. Many researchers [21,22] endeavoured to estimate reflectance with the help of estimated illumination using (2). Intuitively, the reflectance component obtained using (2) should give a good estimate as it maintains the relation among reflectance, illumination and the image as per Retinex theory. However, the reflectance component may not be suitable for obtaining a good quality image because the relation in (2) involves a low-light image. Therefore, it is required to have a prior for reflectance. Based on the analysis of a large set of images and various Retinex-based algorithms, we have established the following prior for reflectance component of good quality images. Reflectance component should preserve image details, i.e. textural and structural details, especially in low-light regions. It should be able to retain natural colour information throughout the image. In low-light images, some details may be missing in direct estimation of reflectance using illumination. Thus, to improve the reflectance estimation, the notion of fusion may be used again. We could use multiple instances of estimated reflectance. However, the choice of these instances plays a defining role. The reflectance component achieved using the proposed illumination estimation, and Retinex model is one good alternative because of two reasons: good illumina-tion estimate should result in good reflectance estimate, and it retains original pixel-wise relation (thus, avoids any distortion). To further improve reflectance estimation, we may use other components which can contribute towards the colour, structure and texture of the reflectance. The illumination estimation (T 1 ) achieved by bilateral filtering contains structural details along with smoothing of textural details. If we use T 1 in obtaining the reflectance estimation as per the Retinex model, it will enhance textural details. Along with structural and textural details, reflectance also requires colours without any distortion. As Gaussian filtering provides uniform illumination, when we divide the image with uniform illumination, it provides distortion-free colours for instance. This is because it will maintain the relative relation among the pixels. As we have previously discussed, the reflectance must have structural and textural details with no distortion in the colour of objects. To achieve these priors in reflectance, we require to fuse adequate instances in the correct manner.
Unlike illumination estimation, the multi-scale fusion of reflectance causes artefacts, since it employs a smooth blending of the images. It can lead to a loss of reflectance properties. Thus, the approach of naive fusion can achieve better reflectance estimation.

Fusion-based simultaneous estimation of reflectance and illumination for low-light image enhancement
The proposed algorithm performs low-light IE using simultaneous estimation of both components (i.e. reflectance and illumination) of an image. The algorithm consists of three major steps for the estimation of each component-first, estimation of multiple instances for both reflectance and illumination; second, the estimation of the respective weight maps of each instance; third, the fusion of instances for the estimation of components. After estimation of each component, the proposed algorithm combines the estimated reflectance and gamma-corrected illumination to achieve enhancement. Figure 2 shows framework of the proposed algorithm. To achieve IE, the approach of illumination estimation is discussed below.

Components estimation for illumination
In low-light images, max-channel of the image captures most of the illumination relevant information [21,22,28]. Thus, the proposed algorithm uses max-channel of the input image to estimate multiple instances of the illumination. The max channel of an image can be estimated as where T 0 denotes the max-channel of the image, c denotes the colour channel and R,G ,B denote the red, green, and blue colour channels of the image respectively. As discussed earlier, Illumination requires smoothing of texture along with the preservation of structure (i.e. strong edges [5] that define the broad structure of the objects in images). We proposed the use of a bilateral filter to estimate an instance of illumination. Bilateral filtering provides smoothing while preserving the edges. The smoothing provides an estimate of illumination, whereas the preservation of edges helps in maintaining the natural variation in the illumination of the image. The estimation of bilateralfiltered illumination instance is given as where T 1 denotes the first instance of illumination (i.e. bilateralfiltered instance of max-channel), P denotes the neighbourhood of the pixel x, G s and G r denote Gaussian kernels for space and range respectively. K x is the sum of the weights applied to each of the pixels in the neighbourhood. Bilateral filtering performs smoothing of textural details while preserving structural details. As discussed earlier, a clear categorisation of textural details and structural details is nearly impossible. While preserving structural details, bilateral filtering may leave some textural details and vice-a-versa. Textural details are undesirable in illumination. Thus, we proposed the use of inverse gradient to further suppress the tex-tural details in the illumination. The inverse gradient emphasises the non-textural details, which can help in providing textural smoothing. Moreover, the inverse gradient provides an estimate with low textural information and helps in textural suppression. We estimate the second instance of illumination using the inverse gradient, which is expressed as where T 2 denotes the second instance of illumination (i.e. inverse gradient of max-channel), and ∇ h and ∇ v denote the first-order derivatives along the horizontal and vertical directions, respectively. The variable is used to set a lower limit to the denominator. The above two instances focus on the suppression of textural details and preservation of structural details. It may compromise the smoothing of the overall image. To improve smoothing, Gaussian filtering of max-channel is considered as the third instance of illumination estimation. In MSR, Jobson et al. established that multiple Gaussian filters work better than a single Gaussian filter. Thus, the proposed approach uses the concept of multi-Gaussian filtering to estimate the third instance of illumination. The third instance of illumination T 3 is estimated as the average of multiple Gaussian filters on T 0 , which is expressed as where T 3 denotes the third instance of illumination (i.e. average of Gaussian-filtered instances of max-channel) and i denotes the scale of ith Gaussian filter. The multiple scales help in providing dynamic range compression and colour constancy simultaneously. The values are considered as per the empirical analysis presented in MSR [8].

Components estimation for reflectance
Reflectance estimation plays a vital role in IE. As discussed earlier, the reflectance component should preserve textural details, structural details, and natural colour information. The fusion of appropriate instances of estimated reflectance leads to a superior estimation of reflectance. The proposed algorithm uses three instances of estimated reflectance to achieve the desired characteristics of the reflectance, as discussed in Section 3.1.
The algorithm estimates the first instance of reflectance using the estimated illumination (T est ) and the relation (2). The reason for selecting the above instance is two-fold: a good illumination should result in a good reflectance, and it retains image properties. The first instance of reflectance estimation can be expressed as where R 1 denotes the first instance of reflectance, T est denotes the estimated illumination, and c denotes the colour channel. The estimation of reflectance using final estimated illumination (T est ) and the Retinex model (2) helps in maintaining the relationship between the image and its components. It prevents any distortion in the image quality. However, despite good illumination, the reflectance estimation may not be proper due to the involvement of a low-light image. This may result in loss of textural details, especially in dark regions. Thus, to further improve reflectance estimation, the proposed approach uses illumination achieved by bilateral filtering (T 1 ). T 1 contains textural smoothing with structural preservation. The estimate of reflectance using T 1 provides enhanced textural details. The second instance of reflectance using bilateral filtering can be expressed as where R 2 denotes second instance of reflectance and c denotes the colour channel.
One of the significant challenges in the estimation of reflectance is to retain colour constancy of the image. The above two instances help in preserving the textural and structural details. The proposed algorithm uses the third instance of reflectance estimated using T 3 . The illumination estimation T 3 is generated by Gaussian filtering, which causes uniform smoothing. Thus, the reflectance estimation using T 3 preserves local characteristics of the image and prevents any colour distortion. The third instance of reflectance can be expressed as where R 3 denotes the third instance of reflectance. It will maintain the relative relation among the image pixels. As discussed earlier, the reflectance estimate must contain textural and structural details without any colour distortion.

Fusion for illumination estimation
Illumination estimation requires an effective blending of the instances. The proposed algorithm uses multi-scale fusion for blending of the illumination instances in order to incorporate the desired features from different instances. The process of illumination blending requires weight maps corresponding to each instance. The algorithm computes Gaussian pyramids for the weight maps of the input illumination instances, and Laplacian pyramids for the input instances of the illumination. Downsampling is performed on the input weight map using a low-pass filter to estimate the next layer of the Gaussian pyramid. The low-pass filter helps in reducing the dimension and density of the current layer of the pyramid. The down-sampling operation can be expressed as where r and c denote number of row and column in current layer, Rd denotes the reduced layer (i.e. the next layer), x and y denote the pixel coordinates. w(⋅) is the weighting function for the generating kernel which is based on Gaussian weighting. Gaussian pyramid is a set of original and (l − 1) downsampled images. It uses each layer to estimate the next layer of the pyramid. Figure 3 shows a Gaussian pyramid. The estimation of Laplacian pyramid for each instance of illumination requires the Gaussian pyramid. The layers of the Laplacian pyramid are the differences of consecutive layers of Gaussian pyramid, given by where j denotes the layer of the pyramid,  is the gaussian pyramid,  is the laplacian pyramid, and Ep(⋅) denotes the expand operation. Expand operation interpolates the intermediate values of input layer to perform up-sampling. The expand operation is reverse of the reduce operation, it can be given as The proposed algorithm utilises these pyramids for illumination estimation, which can be represented as where T est denotes the estimate of illumination, T i denotes the illumination instance, W i denotes the corresponding weight map, n denotes the number of instances for fusion, l denotes the l th layer of the pyramid. The resultant pyramid {T est } is collapsed to form T est .
To estimate illumination using multi-scale fusion, weight maps play a vital role in incorporating desirable features. An appropriate weight map helps in avoiding the overexposed and underexposed pixels. Moreover, it provides the appropriate contrast in the illumination to capture the structural details. To deal with the smooth blending of illumination, Fu et al. [21] presented an appropriate weight map calculation based on the combination of brightness and contrast of the components for illumination. Brightness-based weight map helps in reducing the effect of overexposed and underexposed pixels. Fu et al. presented the brightness-based weight map as where W b i denotes the brightness based weight map for the ith component of illumination. The contrast-based weight map helps in capturing appropriate structural details from the components of illumination. Fu et al. presented the contrast based weight map as where W ct i denotes the contrast based weight map for the ith component of illumination. H and S denote Hue and Saturation channel of image in HSV colour space, respectively. The values of parameters and are 2 and 250 • , respectively, as these values perform well for the proposed algorithm. The above mentioned weight maps are combined to extract the desired features from the instances of illumination. Thus, the combined weight map can be expressed as where W i denotes the weight map for the ith component of illumination, and • denotes the element wise multiplication. To perform feature extraction uniformly from all the components of illumination, W i requires normalisation, which is given as After the estimation of weight maps for each components, the proposed algorithm performs multi-scale fusion for estimation of illumination, as described previously in Equation (15).

Fusion for reflectance estimation
Each of the instances of reflectance contains their own specific desired properties which are essential for reflectance estimate. Therefore, the proposed method uses fusion to estimate reflectance having desired features. However, multi-scale fusion can lead to a smooth blending of the reflectance components, which can generate artefacts in reflectance. As it can be noticed from the priors, reflectance does not require smooth blending. Thus, the proposed algorithm utilises naive fusion to prevent smooth blending. Naive fusion uses uniform weights to blend the instances of reflectance. It blends the instances of reflectance to generate the estimated reflectance with almost no artefacts, which can be expressed as where W r denotes the weights for reflectance instance. The value of W r for each instance of reflectance is chosen empirically. The weights W r 1 , W r 2 , and W r 3 takes the value 0.5, 0.25, and 0.25, respectively. The first instance of reflectance achieves appropriate detail-preservation. Thus, it requires slightly higher weights. The other two instances of reflectance provide better textural details and colour constancy. Thus, the algorithm balances both textural details and colour constancy by assigning equal weights to them. Naive fusion provides the estimation of reflectance. After the computation of both components from an image, the proposed algorithm generates an enhanced image by pixel-wise product of the estimated reflectance with gamma-corrected illumination. The low-light enhanced image is expressed as (21) where I c f denotes the enhanced image, while c denotes the colour channel. The value of parameter is chosen empirically. The parameter takes the value 0.2 for low-light IE.

EXPERIMENTAL RESULTS AND ANALYSIS
The proposed approach is experimentally analysed on 350 images from various data sets: Guo et al. [22], Berkeley Segmentation database [30], Kodak Database [31], NASA Dataset [32], HDR Dataset [33], and ExDark Dataset [34]. The qualitative and quantitative analyses the performance of proposed method with state-of-the-art strategies: PIE [14], WSE [15], FWE [21], LIME [22], SRIE [16], CRM [18], and SDD [17]. The implementations (i.e. parameters and code) of other algorithms are taken from the authors' provided source to ensure a fair comparison. Moreover, we discuss the parameter analysis to ensure the superior performance of the proposed approach. Another important aspect of performance analysis for the proposed approach is computational complexity. Thus, we also discuss the analysis of computational complexity for the proposed approach in comparison to other methods.

Quantitative analysis
To validate the quality of resultant images, a quantitative analysis of the performance of the method is required. Since visual analysis is often subjective, a quantitative analysis gets necessitated. The quantitative analysis of the proposed algorithm uses structural similarity index (SSIM) [35], visual information fidelity (VIF) [36], and spatial correlation coefficient (SCC) [37]. SSIM [35] measures the structural similarity of two images. SSIM uses mean, variance and standard deviation to compute similarity between two images. SSIM measures the amount by which an image gets degraded as a result of an image processing algorithm. A higher SSIM value shows less degradation in the enhanced image. Thus, a higher SSIM value represents improved image quality.
VIF [36] measures the naturalness of the enhanced image. VIF measures the loss of information incurred. VIF provides a relationship between visual quality and information in an image. It uses natural scene statistics (NSS) model, distortion model, and HVS model. Moreover, VIF has a unique characteristic that it gives a value higher than one if there is an improvement in the quality of an image.
Another measure used in the quantitative analysis is spatial correlation coefficient (SCC) [37]. SCC provides a measure of spatial correlation between the input and enhanced image. SCC considers the high-pass filtering of the original and enhanced image to estimate correlation coefficient.
We performed exhaustive experimentation for 350 low-light images with varying illumination. The results of quantitative measures for sample images are shown in Table 1. It can be noted from Table 1 that the proposed method achieves the best in most of the cases. The proposed algorithm achieves best SSIM values for most of the images except 'Girl', 'Shop' and 'Cup' images. However, it is evident from the visual analysis that the proposed method provides better enhancement than the algorithm with best value. It may be also noted from Table 1 that the proposed algorithm provides best VIF values for all images. It shows that the proposed algorithm restores natural colors in the enhanced image. Further, SCC value of the proposed algorithm is best for most of the cases except the 'Statue' image. It can be observed from the visual analysis that the proposed algorithm achieves better enhancement than WSE.
The manual analysis of large set of results is impractical and may infer erroneous conclusions. Thus, we perform statistical analysis of the results of the proposed algorithm and other algorithms. We use the pair-wise z-test for statistical analysis as shown in Table 2. The null hypothesis (H 0 ) and alternate hypothesis (H 1 ) for the z-test are the following.
H 0 : The performance of proposed and other algorithm is same.
H 1 : The performance of proposed algorithm is better. We perform one tail testing to evaluate the enhancement. The probability for rejection (i.e. p-value) of H 0 is obtained from zdistribution table [38].

Visual analysis
The quantitative analysis presented in 4.1 shows the superiority of the proposed method. However, due to unavailability of any universally accepted quantitative measure, selection of an appropriate quantitative measure for IE is a challenging task. Thus, it is essential to analyse the enhanced images visually as well. An exhaustive visual analysis of the results is performed. The paper contains a few samples of visually analysed images, as shown in Figures 4 to 9.
The 'Girl' image having non-uniform illumination, is shown in Figure 4 along with results of several algorithms. PIE and WSE algorithms produce natural-looking results as shown in Figure 4. The patch of PIE and WSE show limited enhancement of lightness. Patch of FWE highlights limited improvement of lightness and contrast. LIME is unable to produce appropriate colour constancy. Figure 4 highlights a patch of LIME which produces artefacts in the enhanced image. It fails to provide appropriate results in regions with normal lighting. The patch of SRIE presents the over-smoothing effect in results of SRIE, which leads to information loss. It can be observed from the highlighted region that SRIE provides low contrast results. It can be observed from the results of CRM as shown in Figure 4 that colours are faded and poor contrast. For readers' assistance, a portion of image is highlighted in red coloured box. It may be noted from the highlighted region that colour of legs are faded which results in poor contrast. The results  The patch of proposed method shows improved lightness without any artefacts. Besides, the proposed method achieves colour constancy. Figure 5 shows 'Landscape' image having non-uniform illumination and its results for several algorithms. PIE and WSE are unable to achieve significant enhancement in regions with low-light. Further, it can be noticed from Figure 5 that FWE provides limited enhancement of lightness with low contrast. Figure 5 show that LIME attempts to improve the lightness. LIME fails to provide suitable colour constancy as shown in results of LIME in Figure 5. Further, LIME achieves over-enhancement in light dominant regions. The details near the boat are barely visible in patch of LIME as shown in Figure 5. SRIE result achieves an over-smoothing effect and lightness improvement is also limited. One may notice the blurred details on the ground as shown in patch of SRIE in Figure 5. CRM provides faded effect in the colors which results in poor contrast as shown in  [14]; WSE [15]; FBE [21]; LIME [22]; and Bottom row: SRIE [16]; CRM [18]; SDD [17]; Proposed method. The highlighted region of results is shown adjacent to each image FIGURE 5 'Landscape'; Top row: Input image; Results of PIE [14]; WSE [15]; FBE [21]; LIME [22]; and Bottom Row: SRIE [16]; CRM [18]; SDD [17]; Proposed method. Highlighted region of results is shown adjacent to each image Figure 5. SDD provides limited improvement in lightness with less details in darker regions as shown in highlighted region. It can be observed from Figure 5 that the proposed approach provides appropriate illumination enhancement in all regions of the image without any artefacts. Figure 6 shows the 'Birds' image and its results for several approaches. PIE and WSE provide limited improvement in lightness, as shown in Figure 6. FWE generates comparable enhancement of illumination, as shown in Figure 6. However, FWE fails to achieve an appropriate contrast. The patch in Figure 6 highlights the result of LIME which generates an overenhanced image with poor colour constancy. SRIE provides a result with an over-smoothing effect which causes loss of information, as shown in Figure 6. The result of CRM contains poor colour preservation. SDD gives smoothing in few regions of the result. The proposed approach generates a natural-looking result, as shown in Figure 6. The proposed approach provides colour constancy with significant lightness while preserving the naturalness.
The 'Shop' image and its results for several approaches are shown in Figure 7. In Figure 7, PIE and WSE achieve limited improvement in lightness. It can be observed from FWE in Figure 7 that the result looks natural. The patch of FWE highlights that FWE fails to provide appropriate lightness enhance-ment. LIME generates images with comparatively better illumination, as shown in Figure 7. However, the patch of LIME highlights that it achieves poor colour constancy. The result of SRIE contains good colour constancy, although lightness improvement is limited. The results of CRM and SDD contain limited improvement in lightness. It can be noted from the highlighted region that CRM provides faded colours and SDD provides darker results. The proposed approach produces resultant image with suitable colour constancy. It can be noted from Figure 7 that the proposed approach provides better colour constancy while enhancing the lightness. In the highlighted patch, it can be observed that the proposed approach generates superior results.
The image 'Cup' and its results for several approaches are shown in Figure 8. In the image 'Cup', a gradient of illumination may be noticed due to light source focusing in the centre of the image. It has large variation in light across the width of image i.e. left and right corners are dark while center of the image is illuminated. PIE, WSE, and FWE fail to attain significant improvement of lightness in the darker regions, as shown in Figure 8. SDD performs slightly better than PIE, WSE, and FWE, but improvement is not significant. CRM performs better than all above algorithms. It can be noted from Figure 8 that the result of LIME achieves over-enhancement in the illuminated regions  [14]; WSE [15]; FBE [21]; LIME [22]; and Bottom row: SRIE [16]; CRM [18]; SDD [17]; Proposed method. Highlighted region of results is shown adjacent to each image  [14]; WSE [15]; FBE [21]; LIME [22]; and Bottom row: SRIE [16]; CRM [18]; SDD [17]; Proposed method. Highlighted region of results is shown adjacent to each image of the image. The results of SRIE and the the proposed algorithm are comparable. However, one may notice form Figure 8 that the proposed approach gives better lightness in the both corners (left and right) of the image. Thus, proposed approach provides superior enhancement for images with varying illumination. Figure 9 shows few sample input images with their corresponding results generated by the proposed method. It can be noted from the enhanced images that the proposed algorithm yields with appropriate lightness and contrast. The proposed approach is capable of dealing with non-uniform and varying illumination conditions.

Parametric analysis
The proposed approach uses multiple instances of illumination and reflectance for their simultaneous estimation. The instances used for estimation of illumination and reflectance have various parameters that control the quality of results of the  [14]; WSE [15]; FBE [21]; LIME [22]; and Bottom row: SRIE [16]; CRM [18]; SDD [17]; Proposed method proposed approach. The effect of these parameters on the quality of results is analysed with the help of exhaustive experimentation. Since SSIM is one of the most used IE measures, and it gives a good measure of structure preservation in the enhanced image, we selected SSIM to perform parametric analysis. The proposed algorithm uses a bilateral filter for estimation of one of the three instances of illumination (as discussed in Section 3.2.1). The bilateral filter involves three parameters: diameter d of neighbourhood P, sigma for colour space colour , and sigma for coordinate space coordinate . The value of parameter d is always greater than or equal to 1 as the diameter of the kernel cannot be 0. The upper bound of d is not defined for a bilateral filter. We have performed the experimentation on different sizes of images to analyse the effect of the value of d on the image quality. It is observed that the mean SSIM value for the images starts decreasing after d = 5, as shown in Figure 10. Therefore, the best value of SSIM can be achieved at d = 5. To analyse the effect of colour and coordinate , we perform exhaustive experimentation for both parameters simultaneously. It is observed that for values greater than 60, both colour and coordinate produce insignificant improvement in the final image. Thus, the analysis is performed in the range 0 to 60 for both colour and coordinate , as shown in Figure 11. The colour of a pixel depends on the value of the colour . As a larger value of colour will give higher weights to pixels in the neighbourhood. Initially, SSIM value improves by increasing colour then it starts decreasing. SSIM value decreases due to colour distortion of pixels, which is caused by a greater effect of the pixels in the neighbourhood. The parameter coordinate helps in capturing the global properties from the neighbourhood. An increase in the value of coordinate after a certain point can cause distortion due to an increase in weights of the neighbourhood pixels. An increased distortion causes loss of structural information which is visible as blurring of edges and details in the final image. It is found empirically that the best values for d , colour , and coordinate are 5,12, and 44, respectively.

Computational complexity
It is imperative that the algorithm be computationally fast without compromising on the quality of results. Therefore, one of the essential objectives behind the proposed methodology is to improve the computational efficiency of simultaneous estimation for IE. Akin to the quantitative and visual analyses, the computational complexity analysis of the proposed approach is performed on a set of 200 images. We created two sets of 200 images each, one with image size 350x229, and other with image size 700x458. The first set of 200 images is created by decreasing the size of the original image from 700x458 to 350x229. Table 3 shows the average time of computation per image for different methods applied on both sets of images. It can be observed from Table 3 that SRIE and WSE take significantly higher time than other algorithms for both sets of images. The algorithms PIE, LIME, and CRM take lesser time for the smaller images as compared to the proposed algorithm. However, these algorithms take a longer time for larger-sized images. This may be attributed to higher time complexity of PIE and LIME as compared to the proposed algorithm. FWE, on the other hand, takes lesser time as compared to the proposed method for both sets of images. However, the proposed method has a superior performance in comparison to FWE on both visual and quantitative terms (see 4.1 and 4.2), which compensates for the slight increase in the time taken. The time complexity of the algorithms is also analysed for the discussed algorithms for an input image with N pixels. In PIE, simultaneous estimation requires (N ) time per iteration. It is noticed that in the worst-case PIE requires (log N ) iterations for most of the images. Thus, the time complexity of PIE is  (N log N ). In WSE, simultaneous estimation requires (N ) time per iteration. It is noted that WSE requires (N ) iterations for estimation of reflectance and illumination. Thus, the time complexity of WSE is (N 2 ). FWE estimates illumination in a single iteration as it is not an iterative approach. FWE requires multiple steps in an iteration. Thus, the time complexity of FWE is (cN ), where c is constant. Hence, the time complexity of FWE is (N ). LIME estimates illumination in (log N ) iterations, and it requires (N ) time per iteration. Thus, the time complexity of LIME is  (N log N ). SRIE requires (N ) time per iteration for simultaneous estimation. It is noted that SRIE requires (N ) iterations. Thus, the time complexity of SRIE is (N 2 ). CRM uses sped-up solver to estimate illumination thus, CRM requires (N ) time. SDD requires (N ) time per iteration of estimation, and it converges in (N ) iterations. Hence, the time complexity of SDD is (N 2 ). The proposed approach is not an iterative algorithm, and it uses only one iteration for the simultaneous estimation of both components. The algorithm uses multiple instances for the estimation of reflectance and illumination. It causes a linear increase in the time taken. Thus, the complexity of the proposed algorithm is  (c 1 N ), where c 1 is constant. The constant c 1 is slightly greater than constant c of FWE. Although the proposed algorithm takes a little higher time, the complexity of both FWE and the proposed strategy is (N ). However, as discussed before, it can be noted from the quantitative and qualitative analysis that the proposed strategy is superior to FWE.

CONCLUSION
In this paper, a new algorithm for simultaneous estimation of reflectance and illumination is presented. The algorithm performs low-light IE with varying illumination. Along with lowlight enhancement, the algorithm preserves the naturalness of the image. It is evident from the experimental analysis that the proposed algorithm provides superior enhancement while dealing with low-light images having non-uniform illumination. The major advantages of the proposed approach are as follows.
• It achieves appropriate enhancement of lightness in an image while avoiding any over-enhancement. • It performs well on images having non-uniform illumination while preserving naturalness. • It has less time complexity than other approaches for simultaneous estimation. • It reduces the trade-off between lightness, contrast, and naturalness of an image.
An extensive experimentation and analysis over a large set of images validated the performance of the proposed approach. The visual analysis validated that the proposed approach generates images with suitable colour constancy for objects not having uniform illumination. The proposed approach outperforms the state-of-the-art approaches for low-light IE.
In the future, simultaneous estimation-based approaches can be improved with the help of proposed priors. Moreover, the efficiency of the algorithm can be improved by optimising the implementation.