orb descriptor

ORB, Oriented FAST and Rotated BRIEF, published by Ethan Rublee and colleagues at Willow Garage in 2011, was created to give the open-source community a fast, free alternative to the patented SIFT and SURF. Its design philosophy is pragmatic: take two cheap existing components, the FAST corner detector and the BRIEF binary descriptor, and fix their two biggest weaknesses, namely that FAST has no orientation or scale and that BRIEF is not rotation invariant. The payoff is a descriptor that matches in a few microseconds and runs comfortably in real time on a phone or a robot, which is why ORB became the default feature in OpenCV and the engine inside the popular ORB-SLAM systems.

For detection, ORB runs FAST (a corner detector that simply checks whether a ring of pixels around a candidate is consistently brighter or darker than the centre, which is extremely fast) across an image pyramid to gain coarse scale invariance, and keeps the strongest corners by Harris response. It then gives each keypoint an orientation using the intensity centroid: compute the centre of mass of the patch's intensities, and the vector from the keypoint centre to that centroid defines a reference angle. This is a cheap but effective way to make the descriptor rotation aware.

For description, ORB uses a steered, learned version of BRIEF. Plain BRIEF describes a patch by a string of bits, each bit recording whether one pre-chosen pixel is brighter than another pre-chosen pixel within the smoothed patch. ORB rotates the set of test point pairs by the keypoint's orientation before sampling (rotated BRIEF, rBRIEF), so the bit string is consistent under rotation. Rotating the tests, however, makes the bits more correlated and less discriminative, so ORB additionally learns, by greedy search on training data, a set of 256 test pairs that are highly variant and mutually uncorrelated. The output is a 256-bit binary descriptor compared with the Hamming distance, the count of differing bits, which a CPU computes in a couple of instructions.

ORB's strengths are speed, zero memory overhead per bit, free licensing, and good rotation invariance; its weaknesses are weaker invariance to large scale and viewpoint change than SIFT, and lower precision on hard wide-baseline matching. It is the right default whenever throughput matters, such as visual odometry, SLAM, augmented reality on mobile, and panorama stitching on constrained hardware. For the highest accuracy under extreme transformations, learned detectors and descriptors now outperform it, but ORB's combination of speed, simplicity, and permissive licensing keeps it ubiquitous.