Natural Feature Tracking

Top: For natural feature tracking, feature descriptors (red dots) on a reference image are matched with features on a video. Left: The tracking system was designed for circuit board assembly, which are 3D objects with a spatial structure.

Natural feature tracking (NFT) is an object tracking method that utilizes natural interest points of an object to track,

such as texture-patches, edges, color blobs, etc. Physical objects can be represented by storing those interest points as feature descriptors (or short: features) in a feature map. The objects can be identified and tracked by comparing the feature maps with features that are extracted from a video. The feature maps are labeled, thus, the object to which it belongs to is known. If a feature map's features meet the features from a video, the object is considered as known.


The goal of our research is to leverage NFT tracking technology for assembly assistance and quality control which introduces two challenges: first, it is required to distinguish several feature objects with a high fidelity rate and robustness. Second, objects to track need to be considered as partially covered in general, since operators on the factory floor typically use their hands to handle them.


Our research focuses on a statistical parameter that allows us to assess the matching quality and to evaluate whether the matched object is the correct one. In a typical applications data structure, all feature descriptors are stored in a kd-tree, which sorts all features due to dimensions with the highest variance. Matching features means to pick only features from a kd-tree, which are close to the search features. However, when too many features are too close, the search feature cannot be assigned to one pre-stored feature; we cannot be sure to make a mistake when picking one, thus, the search feature will be rejected. This is the typical case in assembly when too many similar-looking objects are stored in a kd-tree.


We introduced a statistical parameter that considers all data inside the kd-tree to calculate a probability factor that allows us to estimate the probability whether a search feature maps with a feature from a feature map. Since the reliability is the inverse of the probability, we can use this parameter to determine whether the parameter matches. This increases the robustness, which does not rely on unambiguous features.


Our addressed application is circuit board assembly; our demo and test object was a computer. The components on a circuit board appear as squares with different dimensions. Our tests indicate that we can increase the number of objects that we are able to distinguish.



Bermudez, Francely Franco; Santana Diaz, Christian; Ward, Sheneeka; Radkowski, Rafael; Garrett, Timothy; Oliver, James, "Comparison of Natural Feature Descriptors for Rigid-Object Tracking for Real-Time Augmented Reality," Proceedings of the ASME 2014 International Design Engineering Technical Conferences & Computers and Information in Engineering Conference, IDETC/CIE 2014, August17-20, 2014, Buffalo, NY


Radkowski, R.; Oliver J.: Natural Feature Tracking Augmented Reality for on-site Assembly Assistance Systems. In: Human Computer Interaction International Conference, July, 22-26, 2013, Las Vegas, Nevada, 2013

About Us


Recent Works

The Augmented Reality Lab explores the augmented reality (AR) technology and its capabilities for engineering applications.

+1 (515) 294-7044

Iowa State University

1620 Howe Hall

Ames, IA 50011-2161



© 2014 All Rights Reserved