Overview:

To ensure that extremely noisy and unclear images could be effectively used by both algorithms and human observers, our team developed a noise reduction algorithm.

This algorithm integrates Singular Value Decomposition (SVD), Principal Component Analysis (PCA), Gaussian filtering, and mean filtering. As a result, the processed images became significantly more recognizable and suitable for further analysis.

Team Members: Kefan Wu, Enzo Smajlaj

Noisy Image we chose as the exmaple

Why does noise exist in images?

Sensors and Noises

The sensor of the cameras(CMOS or CCD) is composed of a matrix of small light sensors, each representing a pixel in the image.

Photons in light don't always stay where they are meant to; they can scatter to other pixels, creating noise.

The red square in the grid shows where the light hits the sensor, while the scattered photons (red dots) cause noise. Higher light sensitivity (ISO) makes this noise more noticeable.

Transmission & Storage

When images are transmitted over wireless networks or cables, signal interference can introduce noise. And sometimes errors in storage or transfer due to hardware issues or electromagnetic interference can also cause pixel-level distortions, which will create noise.

Noise Classification

Gaussian Noise

Usually represented as sensor noise. Values are distributed according to a Gaussian (normal) distribution. Appears as grainy, evenly spread random variations in pixel intensity. Caused by sensor heat, electronic circuitry, or analog-to-digital conversion errors.

Salt-and-Pepper Noise

Appears as random black and white dots scattered across the image, creating a sharp contrast between the noise points and surrounding pixels. Bit errors in image transmission or malfunctioning pixels in sensors.

Poisson Noise

Follows a Poisson distribution; noise level depends on the intensity of the image signal. More noticeable in low-light areas or at long exposure times. Caused by randomness in photon detection due to the quantum nature of light.

Comparison & Limitation

Our algorithm applied a unique combination of SVD and Gaussian smoothing, which allows us to reduce the majority of the noise in the image.

However, at the same time, the fill-in-empty-pixel strategy sacrifices the details of the image and makes it blurry.

Building the Algorithm

We employed a multi-step solution for noise reduction, starting with Singular Value Decomposition (SVD), where each color channel of the image is decomposed into orthogonal components to isolate significant details. This has the effect of reducing noise by removing details, but also results in a more altered noise for us to then reduce.

Next, Gaussian smoothing was applied to each channel, using a Gaussian filter to blur the image and reduce high-frequency noise while preserving important structures. The smoothed image was then downscaled (e.g., by 50%) to simplify the noise distribution and decrease computational complexity. Afterward, the downscaled image was upscaled back to its original size, creating blocks that guide further filtering. These upscaled blocks were subjected to iterative mean filtering, where each pixel was replaced by the average of its 3x3 neighborhood, progressively reducing remaining noise. The mean filtering process smoothed the image while maintaining its overall structure.

To address residual noise, we computed the difference between the original image and the filtered image and applied adaptive thresholding to identify excessively noisy pixels. These noisy pixels were replaced with NaN values, marking them for repair. The next step involved a fill-in-empty-pixel strategy, where NaN pixels were replaced with the average of their neighboring valid pixels. This filling process was repeated multiple times to ensure all noisy regions were seamlessly blended and resulted in a noise-reduced image.

Optimizing the Algorithm

To optimize the noise reduction algorithm, we performed a 2D parameter sweep across different values of SVD truncation rank (k) and noise threshold, generating a heatmap to visualize their combined impact on noise reduction performance.

We used a Peak Signal-to-Noise Ratio (PSNR)-weighted Extreme Value Deviation (EVD) metric, which quantifies noise by emphasizing the deviation of extreme pixel values while incorporating PSNR to balance noise reduction and image fidelity. The block sizes for mean filtering were chosen to be 3x3 because they effectively balance noise reduction and detail preservation, ensuring that local features are smoothed without excessive blurring.

Original Image