Object Detection Model Inference at the Edge with TensorFlow
In this section, we'll focus on getting TensorFlow setup with Java code, the model loaded, and inferences produced from the model to send to the Kafka cluster.
The core java classes for this object detection system running on each cart are listed below:
To get this project going we'll use Apache Maven so we'll need a pom.xml to bring in the needed dependencies. Here, we see the pom.xml
We can see the Confluent dependencies that will support the Kafka Producer API operations along with the TensorFlow dependencies needed to load a pre-trained model (component versions are in the variables section of the pom.xml file; specifically we're using TensorFlow 1.8 in this example).
Let's take a closer look at how we'll load the TensorFlow model and make inferences in Java with the TensorFlow R-CNN object detection model. In the code section below, we can see the
scanAllFilesInDirectory(...) method highlighted which performs the bulk of the work in the class.
scanImageForObjects(...) method takes 3 parameters:
- modelFile: the TensorFlow object detection model to load for inference
- labelMapFile: list of labels the associated modelFile can output (i.e. "the vocabulary of labels the saved model understands")
- inputImageFile: the image file path that we want to use as input to the modelFile to get object detections as output from
This class is a convenient wrapper around the base TensorFlow classes needed to load a
and produce inference output on an arbitrary TensorFlow model.
We'll point out a few key areas in the TensorFlow code above:
is handy because it contains all of the needed files to run a TensorFlow object detection model.
class is of note because it wraps functionality for doing things like loading label files
and converting the image file input
into the proper vectorized
After we get the output of the scores, classes, and bounding boxes from the TensorFlow inference output, the code converts these into a VisualObject
class wrapper to make them easier to work with.
Now that we've located the VisualObjects
in our basket and given them classifications, let's move on to how we'll send the classified objects to the Kafka cluster for processing.