Tone Mapping中luma滤波对噪声放大的定性分析
在tone mapping过程中,通常经过统计之后得到一条mapping曲线,记这条曲线为f(x)f(x)f(x),mapping过程中,对于给定的点,假定其亮度为xxx,映射后为f(x)f(x)f(x),信号的放大增益为k=f(x)xk=\frac{f(x)}{x}k=xf(x),上述放大增益的过程在luma平面上进行,若tone mapping在raw图像上,且像素值为xrawx_{raw}xraw,则经过tone mapping后的像素值为k⋅xraw=xraw⋅f(x)xk\cdot x_{raw}=x_{raw}\cdot\frac{f(x)}{x}k⋅xraw=xraw⋅xf(x);为了方便讨论,我们仅讨论在luma平面进行tone mapping,那么tone mapping后的像素计算公式为k⋅x=x⋅f(x)x=f(x)k\cdot x=x\cdot\frac{f(x)}{x}=f(x)k⋅x=x⋅xf(x)=f(x);
经过上面的背景介绍和铺垫,下面讨论在tone mapping过程中,对于luma平面进行降噪滤波之后再计算放大增益,对于噪声放大、边缘增强及对比度增强现象的内在原理。
对于图像中的x1x_1x1,x2x_2x2,映射曲线为f(x)f(x)f(x),那么x1x_1x1点映射为f(x1)f(x_1)f(x1),x2x_2x2点映射为f(x2)f(x_2)f(x2),先分析极端情况,x1,x2x_1,x_2x1,x2经过滤波的像素值均变为x0x_0x0,映射之后为f(x0)f(x_0)f(x0),上述数量关系如图2所示,于是
k1=f(x1)x1,k2=f(x2)x2,k0=f(x0)x0k_1=\frac{f(x_1)}{x_1},k_2=\frac{f(x_2)}{x_2},k_0=\frac{f(x_0)}{x_0} k1=x1f(x1),k2=x2f(x2),k0=x0f(x0)
于是,在没有开NR时,其tone mapping后x1x_1x1与x2x_2x2的差异为:
Δ0=k2x2−k1x1\Delta_0=k_2x_2-k_1x_1 Δ0=k2x2−k1x1
开NR后,其tone mapping后x1x_1x1与x2x_2x2的差异为:
Δ1=k0x2−k0x1\Delta_1=k_0x_2-k_0x_1 Δ1=k0x2−k0x1
比较Δ0\Delta_0Δ0与Δ1\Delta_1Δ1的大小:
Δ1−Δ0=f(x0)x0(x2−x1)−f(x2)−f(x1)x2−x1(x2−x1)=[f(x0)x0−f(x2)−f(x1)x2−x1](x2−x1)\Delta_1-\Delta_0=\frac{f(x_0)}{x_0}(x_2-x_1)-\frac{f(x_2)-f(x_1)}{x_2-x_1}(x_2-x_1)=[\frac{f(x_0)}{x_0}-\frac{f(x_2)-f(x_1)}{x_2-x_1}](x_2-x_1) Δ1−Δ0=x0f(x0)(x2−x1)−x2−x1f(x2)−f(x1)(x2−x1)=[x0f(x0)−x2−x1f(x2)−f(x1)](x2−x1)
由于x2−x1>0x_2-x_1>0x2−x1>0,于是只需要讨论f(x0)x0−f(x2)−f(x1)x2−x1\frac{f(x_0)}{x_0}-\frac{f(x_2)-f(x_1)}{x_2-x_1}x0f(x0)−x2−x1f(x2)−f(x1)的符号即可,令
m(x)=f(x0)x0−f(x2)−f(x1)x2−x1m(x)=\frac{f(x_0)}{x_0}-\frac{f(x_2)-f(x_1)}{x_2-x_1} m(x)=x0f(x0)−x2−x1f(x2)−f(x1)
为了分析Δ1−Δ0\Delta_1-\Delta_0Δ1−Δ0的符号,我们回过头来对mapping曲线f(x)f(x)f(x)做一些数学层面的条件归纳:首先f(x)f(x)f(x)为单调函数,其一阶导数f′(x)>0f^{'}(x)>0f′(x)>0,其次曲线f(x)f(x)f(x)的斜率在不断减小,即其一阶导数为减函数,于是f′′(x)<0f^{''}(x)<0f′′(x)<0,此外还有f(0)=0f(0)=0f(0)=0,于是问题简化为一个简单的数学问题:
对于f(x)f(x)f(x),有f(0)=0,f′(x)>0,f′′<0f(0)=0,f^{'}(x)>0,f^{''}<0f(0)=0,f′(x)>0,f′′<0,当x∈(x1,x2)x\in(x_1,x_2)x∈(x1,x2)时,讨论m(x)m(x)m(x)与0的大小
针对这个问题,我们先直观判断一下m(x)m(x)m(x)的符号,f(x2)−f(x1)x2−x1\frac{f(x_2)-f(x_1)}{x_2-x_1}x2−x1f(x2)−f(x1)为(x1,f(x1))(x_1,f(x_1))(x1,f(x1))点与(x2,f(x2))(x_2,f(x_2))(x2,f(x2))连线的斜率,f(x0)x0\frac{f(x_0)}{x_0}x0f(x0)为(0,0)(0,0)(0,0)点与(x0,f(x0))(x_0,f(x_0))(x0,f(x0))的斜率,从直观上看,后者大于前者,于是有m(x0)>0m(x_0)>0m(x0)>0,如图3所示;
下面证明这个判断,令g(x)=f(x)xg(x)=\frac{f(x)}{x}g(x)=xf(x),分析其单调性,
g′(x)=xf′(x)−f(x)x2g^{'}(x)=\frac{xf^{'}(x)-f(x)}{x^2} g′(x)=x2xf′(x)−f(x)
要判定其单调性,只需判定其符号,而分母显然为正数,从而只需要分析分子,令
h(x)=xf′(x)−f(x)h(x)=xf^{'}(x)-f(x) h(x)=xf′(x)−f(x)
那么
h′(x)=f′(x)+xf′′(x)−f′(x)=xf′′(x)h^{'}(x)=f^{'}(x)+xf^{''}(x)-f^{'}(x)=xf^{''}(x) h′(x)=f′(x)+xf′′(x)−f′(x)=xf′′(x)
由于x>0,f′′(x)<0x>0,f^{''}(x)<0x>0,f′′(x)<0,于是h′(x)<0h^{'}(x)<0h′(x)<0,从而h(x)h(x)h(x)为单调减函数,又有h(0)=0−0=0h(0)=0-0=0h(0)=0−0=0,所以h(x)≤0h(x)\le 0h(x)≤0,从而对于x∈(0,∞),g′(x)≤0x\in (0,\infty),g^{'}(x)\le 0x∈(0,∞),g′(x)≤0,从而可知,g(x)=f(x)xg(x)=\frac{f(x)}{x}g(x)=xf(x)为减函数。
回到m(x)=f(x)x−f(x2)−f(x1)x2−x1m(x)=\frac{f(x)}{x}-\frac{f(x_2)-f(x_1)}{x_2-x_1}m(x)=xf(x)−x2−x1f(x2)−f(x1),对于x∈(x1,x2)x\in (x_1,x_2)x∈(x1,x2)其最小值为
mmin=m(x2)=f(x2)x2−f(x2)−f(x1)x2−x1=x2f(x1)−x1f(x2)x2(x2−x2)m_{min}=m(x_2)=\frac{f(x_2)}{x_2}-\frac{f(x_2)-f(x_1)}{x_2-x_1}=\frac{x_2f(x_1)-x_1f(x_2)}{x_2(x_2-x_2)} mmin=m(x2)=x2f(x2)−x2−x1f(x2)−f(x1)=x2(x2−x2)x2f(x1)−x1f(x2)
其分母为正数,因而只需要考虑其分子符号;对于分子x2f(x1)−x1f(x2)x_2f(x_1)-x_1f(x_2)x2f(x1)−x1f(x2),将其除以 x1x2x_1x_2x1x2,不改变其符号,于是有
x2f(x1)−x1f(x2)x1x2=f(x1)x1−f(x2)x2\frac{x_2f(x_1)-x_1f(x_2)}{x_1x_2}=\frac{f(x_1)}{x_1}-\frac{f(x_2)}{x_2} x1x2x2f(x1)−x1f(x2)=x1f(x1)−x2f(x2)
由于前面已经分析过,g(x)=f(x)xg(x)=\frac{f(x)}{x}g(x)=xf(x)为减函数,以及x1<x2x_1<x_2x1<x2,有f(x1)x1−f(x2)x2>0\frac{f(x_1)}{x_1}-\frac{f(x_2)}{x_2}>0x1f(x1)−x2f(x2)>0,从而mmin>0m_{min}>0mmin>0,又由于 x0<x2x_0<x_2x0<x2,于是m(x0)≥0m(x_0)\ge 0m(x0)≥0,从而Δ1−Δ0>0,Δ1Δ0>1\Delta_1-\Delta_0>0,\frac{\Delta_1}{\Delta_0}>1Δ1−Δ0>0,Δ0Δ1>1,这说明luma经过滤波之后对于相同差异的输入点,其输出差异Δ1\Delta_1Δ1会比luma没经过滤波输出差异Δ0\Delta_0Δ0大;反应在图像的tone mapping中,对luma降噪相对于直接tone mapping,会放大噪声,同时也增强边缘,增加对比度,也就是图1中的现象。
下面将问题放到更一般的情况,降噪将x1x_1x1变为x3x_3x3,将x2x_2x2变为x4x_4x4,大小关系为x1<x3<x4<x2x_1<x_3<x_4<x_2x1<x3<x4<x2,在曲线上的关系如图4所示
从而k1=f(x1)x1,k2=f(x2)x2,k3=f(x3)x3,k4=f(x4)x4k_1=\frac{f(x_1)}{x_1},k_2=\frac{f(x_2)}{x_2},k_3=\frac{f(x_3)}{x_3},k_4=\frac{f(x_4)}{x_4}k1=x1f(x1),k2=x2f(x2),k3=x3f(x3),k4=x4f(x4),于是有
Δ0=f(x2)x2x2−f(x1)x1x1\Delta_0=\frac{f(x_2)}{x_2}x_2-\frac{f(x_1)}{x_1}x_1 Δ0=x2f(x2)x2−x1f(x1)x1
Δ1=f(x4)x4x2−f(x3)x3x1\Delta_1=\frac{f(x_4)}{x_4}x_2-\frac{f(x_3)}{x_3}x_1 Δ1=x4f(x4)x2−x3f(x3)x1
Δ1−Δ0=f(x4)x4x2−f(x3)x3x1−[f(x2)x2x2−f(x1)x1x1]=x2[f(x4)x4−f(x2)x2]+x1[f(x1)x1−f(x3)x3]\Delta_1-\Delta_0=\frac{f(x_4)}{x_4}x_2-\frac{f(x_3)}{x_3}x_1-[\frac{f(x_2)}{x_2}x_2-\frac{f(x_1)}{x_1}x_1]=x_2[\frac{f(x_4)}{x_4}-\frac{f(x_2)}{x_2}]+x_1[\frac{f(x_1)}{x_1}-\frac{f(x_3)}{x_3}] Δ1−Δ0=x4f(x4)x2−x3f(x3)x1−[x2f(x2)x2−x1f(x1)x1]=x2[x4f(x4)−x2f(x2)]+x1[x1f(x1)−x3f(x3)]
由于0<x1<x3<x4<x20<x_1<x_3<x_4<x_20<x1<x3<x4<x2以及g(x)=f(x)xg(x)=\frac{f(x)}{x}g(x)=xf(x)为减函数,[f(x4)x4−f(x2)x2]>0,[f(x1)x1−f(x3)x3]>0[\frac{f(x_4)}{x_4}-\frac{f(x_2)}{x_2}]>0,[\frac{f(x_1)}{x_1}-\frac{f(x_3)}{x_3}]>0[x4f(x4)−x2f(x2)]>0,[x1f(x1)−x3f(x3)]>0,从而Δ1−Δ0>0,Δ1Δ0>1\Delta_1-\Delta_0>0,\frac{\Delta_1}{\Delta_0}>1Δ1−Δ0>0,Δ0Δ1>1;于是,对于一般情况而言,luma上降噪也会放大噪声,增强边缘,增加对比度,而且降噪强度越大,会导致luma平面越平滑,Δ1−Δ0\Delta_1-\Delta_0Δ1−Δ0也变得越大,从而这种噪声放大与图像增强现象也越强。
综上,对于映射曲线满足f(x)f(x)f(x)为单调增函数,f(x)f(x)f(x)的斜率在不断减小,f(0)=0f(0)=0f(0)=0时,对luma平面降噪会放大图像中的噪声,增强边缘,增加对比度,而通常的tone mapping曲线f(x)f(x)f(x)或者整条曲线满足上述条件,或者整条曲线的一部分满足上述条件,满足上述条件的部分都会对图像进行不同程度的增强和噪声放大。