%20(2).svg){kind=small}
How does the EP-200Q work with an object detection model?
This article introduces how the EP-200Q works with an object detection model.
Yolov8n model is used for this demonstration. The input image resolution is 1280 x 720 at the 30 frame-per-second.
The EP-200Q can run inference on CPU, GPU or NPU. We run FP32 model on CPU and quantized INT8 model on NPU.
Inference on NPU with INT8 model vs Inference on CPU with FP32 model
Here is a brief summary of the AI inference when two bottles, one cup and a tablet is placed on a table. Note that the tablet is not labeled in the model used, so the AI inference does not recognize the tablet as an object.
| NPU | CPU | |
| Model | Quantized INT8 | FP32 |
| Average FPS (Frame Per Second) | 29.95 | less than 1 |
| Average power consumption | 7.4W | 12W |
The frame-per-second is used to indicate how fast the inference is done. 30fps means that the AI inference to one frame is processed within 33ms (1 second divided by 30). In this result, the process flow is as below:
1. Capture camera frame
2. Run AI inference
3. Annotate boundary boxes and object names (output from the AI model) to the frame
4. Display the annotated image on a display
As these processes are sequential, the output FPS becomes lower as the AI inference takes more time. In this example, the camera feeds the input data in 30fps. Therefore, the the result is capped at the 30fps.
So, 29.95 average FPS result is good meaning every single frame is processed by the NPU using quantized INT8 object detection model. On the other hand, running AI inference on CPU with the FP32 is still possible but it takes more than a second to process one frame. In addition to the AI inference time, the power consumption when running AI inference on CPU is much higher.
Summary
Running quantized INT8 object model on NPU provides great benefit in inference time and power consumption. During the demonstration, object detection results are very stable with the quantized INT8 mode, which means that the objects on the table are stably detected properly.
When the AI inference is run on NPU, CPU and GPU are free to process other things. The EP-200Q is the great solution for battery-powered on-device edge vision AI product, which need to process other sensing, data processing, networking or controlling tasks simultaneously.