Conservative scale recomposition for multiscale denoising (the devil is in the high frequency detail)

In this paper we reconsider the class of patch based denoising algorithms and observe that they underperform at lower image frequencies. We solve this problem by operating them within a multiscale structure. Our main observation is that denoising algorithms cannot be trusted with the restoration of high frequency details in the image. Indeed, since denoising algorithms must impose their image prior, the fine details are either smoothed or sharpened in the result. In any case the high frequency properties of the images are altered. This realization has a profound implication on the multiscale approaches, which assume that coarse scale restorations are better denoised and hence are replaced in the finer resolutions. This leads to frequency cut-off artifacts as the coarse restorations are pasted at higher resolutions. We start by studying this phenomenon on a simple Discrete Cosine Transform (DCT) pyramid, for which the artifacts resulting from this process are evident. We propose a simple solution consisting of a “conservative recomposition” of the scales that only retains the lower frequencies of each scale, with the obvious exception of the scale at the highest resolution. This soft fusion eliminates the ringing artifacts and attenuates staircasing artifacts and low frequency bumps. An added benefit of the DCT pyramid is that it allows one to maintain the white noise at the lower resolutions, hence it can be combined with any denoising algorithm without adaptation. This soft fusion recipe can be generalized to any other pyramid structure. We apply it to a Laplacian pyramid as an example. Our proposal merges and operates any denoising algorithm into a multiscale method, with improvements both in visual quality and Peak Signal to Noise Ratio (PSNR), and with little additional complexity. The method is demonstrated on several classic or state-of-the-art denoising algorithms.