内容来自OpenCV-Python Tutorials 自己翻译整理 角点检测原理参考此博客 目标: 理解Harris角点检测的概念
使用函数cv2.cornerHarris(),cv2.cornerSubPix() 原理: 借上面参考博客的一张图
Harris角点检测的方法大概原理就是建立一个窗口区域,然后以当前窗口为中心向各个方向进行偏移。 如上图所示,第一个窗口向各个方向偏移的时候,像素的值没有变化,因为窗口偏移的时候没有遇到任何边缘信息。
第二个图,窗口当中有一个直线,如果当前窗口进行上下的移动,也没回有像素值发生变化(在其他方向上灰度值也会变化)。
第三个图,窗口覆盖了一个“拐角”,如果窗口进行偏移,任何方向上都会有像素变化。 所以,第三张图片判断为检测到角点。 设图像<span class="MathJax" id="MathJax-Element-6010-Frame" tabindex="0" data-mathml="I(x,y)" role="presentation" style="box-sizing: border-box; outline: 0px; display: inline; line-height: normal; text-align: left; word-spacing: normal; word-wrap: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border: 0px; word-break: break-all; position: relative;">I(x,y)I(x,y)表示图像在(x,y)点的像素信息,窗口在点<span class="MathJax" id="MathJax-Element-6011-Frame" tabindex="0" data-mathml="(x,y)" role="presentation" style="box-sizing: border-box; outline: 0px; display: inline; line-height: normal; text-align: left; word-spacing: normal; word-wrap: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border: 0px; word-break: break-all; position: relative;">(x,y)(x,y)处移动<span class="MathJax" id="MathJax-Element-6012-Frame" tabindex="0" data-mathml="(Δx,Δy)" role="presentation" style="box-sizing: border-box; outline: 0px; display: inline; line-height: normal; text-align: left; word-spacing: normal; word-wrap: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border: 0px; word-break: break-all; position: relative;">(Δx,Δy)(Δx,Δy)后计算到的所有差异总和记为
<span class="MathJax" id="MathJax-Element-6013-Frame" tabindex="0" data-mathml="c(x,y;Δx,Δy)=∑u,v∈Window(x,y)w(u,v)(I(u,v)−I(u+Δx,v+Δy))2" role="presentation" style="box-sizing: border-box; outline: 0px; display: inline; line-height: normal; text-align: left; word-spacing: normal; word-wrap: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border: 0px; word-break: break-all; position: relative;">c(x,y;Δx,Δy)=∑u,v∈Window(x,y)w(u,v)(I(u,v)−I(u+Δx,v+Δy))2c(x,y;Δx,Δy)=∑u,v∈Window(x,y)w(u,v)(I(u,v)−I(u+Δx,v+Δy))2 <span class="MathJax" id="MathJax-Element-6014-Frame" tabindex="0" data-mathml="Window(x,y)" role="presentation" style="box-sizing: border-box; outline: 0px; display: inline; line-height: normal; text-align: left; word-spacing: normal; word-wrap: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border: 0px; word-break: break-all; position: relative;">Window(x,y)Window(x,y)是以x,y为中心的窗口,w(u,v)是一个加权值,可以是常数值也可以是高斯加权函数(在后会被忽略掉) 使用泰勒展开式对图像<span class="MathJax" id="MathJax-Element-6015-Frame" tabindex="0" data-mathml="I(x,y)" role="presentation" style="box-sizing: border-box; outline: 0px; display: inline; line-height: normal; text-align: left; word-spacing: normal; word-wrap: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border: 0px; word-break: break-all; position: relative;">I(x,y)I(x,y)在移动<span class="MathJax" id="MathJax-Element-6016-Frame" tabindex="0" data-mathml="(Δx,Δy)" role="presentation" style="box-sizing: border-box; outline: 0px; display: inline; line-height: normal; text-align: left; word-spacing: normal; word-wrap: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border: 0px; word-break: break-all; position: relative;">(Δx,Δy)(Δx,Δy)处一阶展开。
I(u+Δx,v+Δy=I(u,v)+Ix(u,v)Δx+Iy(u,v)Δy+O(Δx2,Δy2)I(u+Δx,v+Δy=I(u,v)+Ix(u,v)Δx+Iy(u,v)Δy+O(Δx2,Δy2)
上面公式右侧约等于
I(u,v)+Ix(u,v)Δx+Iy(u,v)ΔyI(u,v)+Ix(u,v)Δx+Iy(u,v)Δy
其中<span class="MathJax" id="MathJax-Element-6019-Frame" tabindex="0" data-mathml="Ix" role="presentation" style="box-sizing: border-box; outline: 0px; display: inline; line-height: normal; text-align: left; word-spacing: normal; word-wrap: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border: 0px; word-break: break-all; position: relative;">IxIx和<span class="MathJax" id="MathJax-Element-6020-Frame" tabindex="0" data-mathml="Iy" role="presentation" style="box-sizing: border-box; outline: 0px; display: inline; line-height: normal; text-align: left; word-spacing: normal; word-wrap: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border: 0px; word-break: break-all; position: relative;">IyIy为分别对x和y的偏导数 将上面得到的结果代入第一个公式中 <span class="MathJax" id="MathJax-Element-6021-Frame" tabindex="0" data-mathml="C(x,y;Δx,Δy)≈∑Window(Ix(u,v)Δx+Iy(u,v)Δy)2" role="presentation" style="box-sizing: border-box; outline: 0px; display: inline; line-height: normal; text-align: left; word-spacing: normal; word-wrap: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border: 0px; word-break: break-all; position: relative;">C(x,y;Δx,Δy)≈∑Window(Ix(u,v)Δx+Iy(u,v)Δy)2C(x,y;Δx,Δy)≈∑Window(Ix(u,v)Δx+Iy(u,v)Δy)2 使用矩阵的表示方法就是 (图片来自上面的网站,为了方便,直接截图) 可以将图中的第一个公式看成是一个椭圆的方程,矩阵的特征值就是它的长轴和短轴长度。
记为<span class="MathJax" id="MathJax-Element-6022-Frame" tabindex="0" data-mathml="λ1" role="presentation" style="box-sizing: border-box; outline: 0px; display: inline; line-height: normal; text-align: left; word-spacing: normal; word-wrap: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border: 0px; word-break: break-all; position: relative;">λ1λ1和<span class="MathJax" id="MathJax-Element-6023-Frame" tabindex="0" data-mathml="λ2" role="presentation" style="box-sizing: border-box; outline: 0px; display: inline; line-height: normal; text-align: left; word-spacing: normal; word-wrap: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border: 0px; word-break: break-all; position: relative;">λ2λ2 可以根据这些特征值判断一个区域是否是角点,这里使用打分函数
<span class="MathJax" id="MathJax-Element-6024-Frame" tabindex="0" data-mathml="R=det(M)−k(trace(M))2" role="presentation" style="box-sizing: border-box; outline: 0px; display: inline; line-height: normal; text-align: left; word-spacing: normal; word-wrap: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border: 0px; word-break: break-all; position: relative;">R=det(M)−k(trace(M))2R=det(M)−k(trace(M))2来判断是否是角点
k值一般取0.04到0.06
- 当λ1λ1和λ2λ2都小时,R也小,此区域平坦。
- 其中一个远大于另外一个时,R小于0,此区域为边缘
- 都很大的时候,R也很大,判断为角点。
如下图
方向导数<span class="MathJax" id="MathJax-Element-6027-Frame" tabindex="0" data-mathml="Ix" role="presentation" style="box-sizing: border-box; outline: 0px; display: inline; line-height: normal; text-align: left; word-spacing: normal; word-wrap: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border: 0px; word-break: break-all; position: relative;">IxIx和<span class="MathJax" id="MathJax-Element-6028-Frame" tabindex="0" data-mathml="Iy" role="presentation" style="box-sizing: border-box; outline: 0px; display: inline; line-height: normal; text-align: left; word-spacing: normal; word-wrap: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border: 0px; word-break: break-all; position: relative;">IyIy可以使用cv2.Sobel()函数得到 Harris角点检测的结果是灰度图,图中的值为角点检测的打分值。需要选取合适的阈值对结果进行二值化来检测角点。 opencv中的Harris检测方法
使用函数 cv2.cornerHarris() 参数如下: - img 输入图像,数据类型为float32
- blockSize 角点检测当中的邻域值。
- ksize 使用Sobel函数求偏导的窗口大小
- k 角点检测参数,取值为0.04到0.06
代码如下 import cv2 import numpy as np
filename = '2.jpg'
img = cv2.imread(filename) gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY) gray = np.float32(gray) #图像转换为float32 dst = cv2.cornerHarris(gray,2,3,0.04) #result is dilated for marking the corners, not important dst = cv2.dilate(dst,None)#图像膨胀 # Threshold for an optimal value, it may vary depending on the image. #print(dst) #img[dst>0.00000001*dst.max()]=[0,0,255] #可以试试这个参数,角点被标记的多余了一些 img[dst>0.01*dst.max()]=[0,0,255]#角点位置用红色标记 #这里的打分值以大于0.01×dst中最大值为边界
cv2.imshow('dst',img) if cv2.waitKey(0) & 0xff == 27: cv2.destroyAllWindows() 能看到有些红点 如果打分值降低为0.00000001*dst.max()
结果如下
亚像素级精确度的角点 使用cv2.cornerSubPix()函数 首先找到Harris角点,然后将找到某个角点的所有角点坐标值的重心传入该函数(因为一个角点上面可能一堆找到的角点值,需要取重心)
红色标记Harris角点,绿色像素标记修正后的角点。进行迭代运算,迭代一定次数后停止即可。 import cv2 import numpy as np
filename = 'chessboard2.jpg' img = cv2.imread(filename) gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
#找到Harris角点 gray = np.float32(gray) dst = cv2.cornerHarris(gray,2,3,0.04) dst = cv2.dilate(dst,None) ret, dst = cv2.threshold(dst,0.01*dst.max(),255,0) dst = np.uint8(dst)
#找到重心 ret, labels, stats, centroids = cv2.connectedComponentsWithStats(dst)
#定义迭代次数 criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 100, 0.001) corners = cv2.cornerSubPix(gray,np.float32(centroids),(5,5),(-1,-1),criteria) #返回角点 #绘制 res = np.hstack((centroids,corners)) res = np.int0(res) img[res[:,1],res[:,0]]=[0,0,255] img[res[:,3],res[:,2]] = [0,255,0]
cv2.imwrite('subpixel5.png',img) 其中用到的函数
connectedComponentsWithStats(InputArray image, OutputArray labels, OutputArray stats, OutputArray centroids, int connectivity=8, int ltype=CV_32S) 计算连通分量,图像是一个被布尔型标记的4邻域或者8邻域的图像
cv2.cornerSubPix(image, corners, winSize, zeroZone, criteria) 参数为输入图像;角点初始化坐标;搜索窗口边长的一半;zeroZone为搜索区域中间的盲区(dead region)边长的一半,有时用于避免自相关矩阵的奇异性。如果值设为(-1,-1)则表示没有这个区域;搜索终止条件
返回的是一组角点 示例结果图片如下
|