Comment on page
Object Detector and Tracker
With ML Kit's on-device Object Detection and Tracking API, you can detect and track objects in an image or live camera feed.
Optionally, you can classify detected objects, either by using the coarse classifier built into the API.
Because object detection and tracking happens on the device, it works well as the front end of the visual search pipeline.
- Fast object detection and tracking Detect objects and get their locations in the image. Track objects across successive image frames.
- Optimized on-device model The object detection and tracking model is optimized for mobile devices and intended for use in real-time applications, even on lower-end devices.
- Prominent object detection Automatically determine the most prominent object in an image.
- Coarse classification Classify objects into broad categories, which you can use to filter out objects you're not interested in. The following categories are supported: home goods, fashion goods, food, plants, and places.
- Classification with a custom model (Coming Soon) Use your own custom image classification model to identify or filter specific object categories. Make your custom model perform better by leaving out background of the image.
Import VoxelBusters.EasyMLKit and VoxelBusters.CoreLibrary namespaces
using VoxelBusters.EasyMLKit;
using VoxelBusters.CoreLibrary;
Create an instance of the ObjectDetectorAndTracker instance by passing one of the input sources.
ObjectDetectorAndTracker detector;
private ObjectDetectorAndTracker CreateObjectDetectorAndTracker()
{
IInputSource inputSource = CreateImageInputSource(YOUR_TEXTURE_HERE);
detector = new ObjectDetectorAndTracker(inputSource);
return detector;
}
private IInputSource CreateImageInputSource(Texture2D texture)
{
return new ImageInputSource(texture);
}
private IInputSource CreateLiveCameraInputSource()
{
IInputSource inputSource = new LiveCameraInputSource()
{
EnableFlash = false,
IsFrontFacing = false
};
return inputSource;
}
For preparing, you need to pass on ObjectDetectorAndTrackerOptions and a callback to know when prepare is complete.
private ObjectDetectorAndTrackerOptions CreateObjectDetectorAndTrackerOptions(IInputSource inputSource)
{
ObjectDetectorAndTrackerOptions.Builder builder = new ObjectDetectorAndTrackerOptions.Builder(inputSource);
builder.EnableClassification(true);
builder.EnableMultipleObjectDetection(true);
builder.SetClassificationConfidenceThreshold(0.5f);
builder.SetCustomModelPath(null);
return builder.Build();
}
private void Prepare()
{
ObjectDetectorAndTrackerOptions options = CreateObjectDetectorAndTrackerOptions();
Debug.Log("Starting prepare...");
detector.Prepare(options, OnPrepareComplete);
}
private void OnPrepareComplete(ObjectDetectorAndTracker detector, Error error)
{
Debug.Log("Prepare complete..." + error);
if (error == null)
{
Debug.Log("Prepare completed successfully!");
}
else
{
Debug.Log("Failed preparing Object Detector and Scanner : " + error.Description);
}
}
Once prepare is complete, you can start processing which gives the result in a callback.
private void OnPrepareComplete(ObjectDetectorAndTracker detector, Error error)
{
Debug.Log("Prepare complete..." + error);
if (error == null)
{
Debug.Log("Prepare completed successfully!");
detector.Process(OnProcessUpdate);
}
else
{
Debug.Log("Failed preparing Object detector and tracker : " + error.Description);
}
}
private void OnProcessUpdate(ObjectDetectorAndTracker detector, ObjectDetectorAndTrackerResult result)
{
if (!result.HasError())
{
Debug.Log(string.Format("Received {0} detected objects", result.DetectedObjects.Count));
foreach (DetectedObject each in result.DetectedObjects)
{
Debug.Log(string.Format("Tracking Id : {0}, Labels : {1}, Bounding Box : {2}", each.TrackingId, string.Join(",", each.Labels), each.BoundingBox));
}
if(result.DetectedObjects.Count > 0)
{
detector.Close(null);
}
}
else
{
Debug.Log("Object detector failed processing : " + result.Error.Description, false);
}
});
Close the detector once you are done processing.
detector.Close(null);//Or pass a callback to know once its complete
Last modified 1yr ago