author: Alexander G. Balakhnin aka Fizick
category: Misc Plugins
requirements: YV12 or YUY2 Colorspace
FFT3DFilter is 3D Frequency Domain filter - strong denoiser and moderate sharpener.
FFT3DFilter uses Fast Fourier Transform method for image processing in frequency domain.
It is based on some advanced mathematical algorithmes of optimal filtration.
It works not locally, but makes some delocalized (block) processing.
In 3D mode, it results in effect similar to partial motion compensation.
This filter can reduce noise without visible quality loss and artefactes,
even with quite strong settings.
It can greatly improve compression and reduce encoded file size.
Also it has option of limited sharpening without both noise amplifying and oversharpening (haloing).
Fiction? Try it yourself! :)
Overlapped blocks and weighting windows are used to prevent blockiness (and ringing).
The blocks are overlapped by some value along their vertical and horizontal sizes.
The lesser overlap, the faster processing, but with more visible grid artifactes.
The overlapping definition is shown on picture.
Some analysis and synthesis weighting windows are used to get effective signal gain=1 after blocks summation.
Since v1.5 it is possible to get noise pattern (shape) by spectrum analyisis of some empty block (without any objects), and then to reduce the noise with the same pattern in whole frame and in whole clip. It may be useful for removal of film (especialy amateur 8 mm) grain and analog TV capture interference.
At sharpening stage (after denoising) the plugin amplifies high spectrum (spatial, 2D) frequencies .
There is also sharpen-only mode without denoising (bt=-1).
Since version 1.1, some special limited sharpening method is used :
The sharpening strength is maximal for frequencies with middle-range amplitudes. Of course, you can control both these margins and general sharpening strength.
Since v.1.7, Gaussian High Pass Filter with variable cutoff frequency is used for sharpening.
Since v.1.9, plugin has special option dehalo for decreasing of strong frequencies, it may be used for spatial adaptive softening of oversharped pictures (halo removal). This mode may be combined with denoising and sharpening.
float "sigma", float "beta", int "plane", int "bw", int "bh", int "bt", int "ow", int "oh",
float "kratio", float "sharpen", float "scutoff", float "svr", float "smin", float "smax",
bool "measure", bool "interlaced", int "wintype",
int "pframe", int "px", int "py", bool "pshow", float "pcutoff", float "pfactor",
float "sigma2", float "sigma3", float "sigma4", float "degrid",
float "dehalo", float "hr", float "ht"
All parameters are named.
first parameter - input clip
- given noise value for all (or highest) frequencies (float>0, default=2.0)
beta - noise margin (float>=1.0, default=1.0 for no noise left):
control noise amount left in the restoration for Wiener filter, so max filter noise attenuation = (beta-1)/beta.
plane - processed color plane: 0 - luma(Y), 1 - chroma U, 2 - chroma V (default = 0)
bw - block width (integer, default = 48 since v.1.2)
bh - block height (integer, default = 48 since v.1.2)
bt - block temporal size, number of frames (-1, 0, 1, 2 or 3, default = 3):
0 - all previous frames (switch Kalman filter mode);
1 - only current frame (spatial 2D Wiener filter);
2 - previous and current frame (3D Wiener filter);
3 - previous, current and next frame (3D Wiener filter)
4 - two previous, current and next frame (3D Wiener filter)
-1 - sharpen only (2D);
ow - overlap width (default=bw/3 since v.1.2)
oh - overlap height (default=bh/3 since v.1.2)
kratio - ratio of threshold to sigma to reset Kalman filter (default = 2.0):
variation threshold = sigma*kratio, good value is about from 1.5 to 3.0;
sharpen - sharpening strength (default=0 - not sharpen)
good values about 0.3 to 1.0 (negative values results in reverse effect)
scutoff - sharpening cutoff frequency, relative to max (default=0.3)
svr - sharpening vertical ratio (to horizontal) (default=1.0 - same as horizontal, 0 - no vertical sharpening)
smin - minimum limit (approximate noise margin) for sharpening stage (default=4.0)
smax - maximum limit (aproximate oversharping margin) for sharpening stage (default=20.0)
measure - select the most optimal (fastest) FFT method by speed measure (with longer init stage)
instead of simple estimation (default=true since v.0.9.2)
interlaced - separate fields processing (default=false)
wintype - weighting windows type (default=0 again since v1.8):
0 - same analysis and synthesis half-cosine window, used in all versions before 1.4;
1 - intermediate between 0 and 2;
2 - flat analysis window, rised cosine (Hanning) synthesis window.
pframe - noise pattern frame number (default=false)
px - noise pattern block horizontal X position (default=0)
py - noise pattern block vertical Y position (default=0)
if px=px=0, then the pattern block is defined automatically with minimal power spectral density.
pshow - show noise pattern block and its properties (default=false)
pcutoff - noise pattern cutoff frequency (relative to max) (default=0.1)
pfactor - noise pattern denoise strength (0 to 1.0, default=0, this method disabled)
sigma2 - given noise value at second scale level frequencies (float>0, default=sigma)
sigma3 - given noise value at third scale level frequencies (float>0, default=sigma)
sigma4 - given noise value at lowest frequencies (float>0, default=sigma)
degrid - weigthing window compensation degree for grid decreasing (float>0, default=1.0)
dehalo - halo removal strength (float>0, default=0)
hr - halo approximate radius (float>0, default=2.0)
ht - halo approximate threshold (float>0, default=50.0)
The most important parameter is a given noise value sigma.
You must use reasonable value based on a priori info for current clip.
Typical value for digital sources is about 1.5 to 2.5, and about 3 and above for analog captured video.
The good value of overlapping size is about quarter to half of block size. The half is the best, but slower.
Good values of block size are about 32 to 64.
Filter can produce some grid artifacts for large sigma
and small bw, bh, ow, oh or small relative overlap sizes, especially with sharpening enabled.
Filter can produce ghosting for large sigma (and kratio) for 3D modes.
Weighting window wintype=0 can produce the worst grid artifactes, window type wintype=2 do not produce grid artifactes, but can produce some ringing, wintype=1 is intermediate case.
I recommend to use the weigthing window compensation with degrid=1 (since version 1.8), it improves the denoise quality and decreases the grid artifactes, especially for 2D.
Sharpening will result in worse clip compression.>
In order to use noise pattern method in place of ordinary (blind) method you must:
Firstly switch show mode pshow=true, and set some non-zero value pfactor=1.0.
Then select frame number and some block position, thus the block shown must not contain any objects beside typical noise pattern.
The switch off show mode pshow=false,
and set noise reduction strength, recomended value pfactor= 0.5 to 1.0.
The best windows type for this method is wintype=2.
The sigma and beta parameters are not used in this denoising method, but you can use this method in show mode pshow=true to estimate sigma value for ordinary denoising method (it is not strictly the same, but similar value).
There is also combined method, when you can directly set different noise values sigma for highest and sigma2, sigma3, sigma4 for lower frequencies. The pattern coefficients will be created internally from these sigmas values by interpolation. Set pfactor=0 for this method (internally it will be =1).
Avisource("input.avi") loadplugin("c:\plugins\fft3dfilter.dll") FFT3DFilter(sigma=3)
Of course, you must replace "c:\plugins" by your plugins folder path (and set your video pathname too :). You can skip LoadPlugin command if you put fft3dfilter.dll file to autoloading plugins folder (C:\Program Files\Avisynth 2.5\plugins).
Avisource("input.avi") loadplugin("fft3dfilter.dll") FFT3DFilter(bt=-1, sharpen=0.7)
Avisource("input.avi") loadplugin("fft3dfilter.dll") FFT3DFilter(sigma=3, sharpen=0.3, interlaced=true)
Note: The processing speed will be decreased more,
so you can consider other (simpler and faster) filters using for chroma denosing
Avisource("input.avi") loadplugin("fft3dfilter.dll") YToUV(fft3dfilter(sigma=3, plane=1).UToY,\ fft3dfilter(sigma=3, plane=2).VToY,\ fft3dfilter(sigma=2, plane=0))
It is especially important if you use motion compensation
Avisource("input.avi") loadplugin("fft3dfilter.dll") loadplugin("depan.dll") motion=DepanEstimate(trust=2.5, fftw=true) DepanInterleave(data=motion) YToUV(fft3dfilter(sigma=3, plane=1).UToY,\ fft3dfilter(sigma=3, plane=2).VToY,\ fft3dfilter(sigma=2, plane=0)) SelectEvery(3,1)
Avisource("input.avi") loadplugin("fft3dfilter.dll") fft3dfilter(sigma=2, plane=4)
Avisource("input.avi") loadplugin("fft3dfilter.dll") fft3dfilter(bt=-1, plane=0, dehalo=1.0, hr=2.0, ht=50, svr=0)
The algorithm of Wiener filter is based on the 3D IIR/3D Frequency Domain Filter from:
MOTION PICTURE RESTORATION. by Anil Christopher Kokaram. Ph.D. Thesis. May 1993.
http://www.mee.tcd.ie/~ack/papers/a4ackphd.ps.gz in postscript format (use GSview with Ghostscript to read).
Search more info about Wiener and Kalman filters in special literature or Googles.
Filter discussion is at DOOM9 Avisynth forum, thread
"New very slow FFT denoiser:"
There is also
FFT3DGPU plugin (by tsp) with similar
algo, but using modern videocard processor for FFT.
tsp ported many fft3dfilter features.
$Date: 2006/12/17 10:28:23 $