× {{alert.msg}} Never ask again
Get notified about new tutorials RECEIVE NEW TUTORIALS

What are keypoints in image processing?

Ray Phan
Mar 24, 2015
<p>Those are some very good questions. Let's tackle each point one by one:</p> <blockquote> <p>My question is what actually are these keypoints?</p> </blockquote> <p>Keypoints are the same thing as interest points. They are spatial locations, or points in the image that define what is <strong>interesting</strong> or what <strong>stand out</strong> in the image. The reason why keypoints are special is because no matter how the image changes... whether the image rotates, shrinks or is subject to distortion, you should be able to find the <strong>same</strong> keypoints in the image. Here's an example from a post I wrote a while ago:</p> <p><img src="http://i.stack.imgur.com/L4RUT.png" alt=""></p> <p><sup>Source: <a href="http://stackoverflow.com/questions/20259025/module-object-has-no-attribute-drawmatches-opencv-python/26227854#26227854">module' object has no attribute 'drawMatches' opencv python</a></sup></p> <p>The image on the right is a rotated version of the left image. I've also only displayed the top 10 matches between the two images. If you take a look at the top 10 matches, these are points that we probably would want to focus on that would allow us to remember what the image was about. We would want to focus on the face of the cameraman as well as the camera, the tripod and some of the buildings in the background. You see that these same points were found <strong>between the two</strong> images and these were successfully matched.</p> <p>Therefore, what you should take away from this is that these are points in the image that are interesting and that they should be found no matter how the image is distorted.</p> <blockquote> <p>I understood that there are some kind of "points of interest" of an image, I know they are scale invariant and I know they are circular.</p> </blockquote> <p>You are correct. Scale invariant means that no matter how you scale the image, you should still be able to find those points. </p> <hr> <p>Now, as what William said in his comment to you above, keypoints are really just spatial locations in the image. What makes keypoints <strong>different</strong> between frameworks is the way you <strong>describe</strong> these keypoints. These are what are known as descriptors. Each keypoint that you detect has an associated descriptor that accompanies it. Some frameworks only do a keypoint detection, while other frameworks are simply a description framework and they don't detect the points. There are also some that do <strong>both</strong> - they detect <strong>and</strong> describe the keypoints. SIFT and SURF are examples of frameworks that both detect and describe the keypoints.</p> <hr> <p>Now what we're venturing into is the descriptor part - scale and orientation. The keypoints we've nailed that concept down, but we need the <strong>descriptor</strong> part if it is our purpose to try and match between keypoints in different images. Now, what you mean by "circular"... that correlates with the scale that the point was detected at. Take for example this image that is taken from the <a href="http://www.vlfeat.org/overview/sift.html" rel="nofollow">VLFeat Toolbox tutorial</a>:</p> <p><img src="http://www.vlfeat.org/demo/xsift_basic_2.jpg.pagespeed.ic.-3OPdEpiBJ.jpg" alt=""></p> <p>You see that any points that are yellow are interest points, but some of these points have a different circle radius. These deal with <strong>scale</strong>. How interest points work in a general sense is that we decompose the image into multiple scales. We check for interest points at each scale, and we combine all of the points together that build the output. The larger the "circle", the <strong>larger</strong> the scale was that the point was detected at. Also, there is a line that radiates from the centre of the circle to the edge. This is the <strong>orientation</strong> of the keypoint, which we will cover next.</p> <blockquote> <p>Also I found out that they have orientation but I couldn't understand what actually it is. It is an angle but between the radius and what?</p> </blockquote> <p>Basically if you want to detect keypoints regardless of scale and orientation, when they talk about <strong>orientation</strong> of keypoints, what they really mean is that they search a pixel neighbourhood that surrounds the keypoint and figure out how this pixel neighbourhood is oriented or what direction this patch is oriented in. This is important for <strong>matching</strong> so that you can match keypoints together. Take a look at the first figure I have with the two cameramen - one rotated while the other isn't. If you take a look at some of those points, how do we figure out how one point matches with another? We can easily identify that the top of the cameraman as an interest point matches with the rotated version because we take a look at points that <strong>surround</strong> the keypoint and see what orientation all of these points are in... and from there, that's how the orientation is computed.</p> <p>Usually when we want to detect keypoints, we just take a look at the locations. However, if you want to <strong>match</strong> keypoints between images, then you definitely need the scale and the orientation to facilitate this.</p> <hr> <p>Hope this helps!</p> <p>This tip was originally posted on <a href="http://stackoverflow.com/questions/29133085/What%20are%20keypoints%20in%20image%20processing?/29137285">Stack Overflow</a>.</p>
comments powered by Disqus