Network Deployment
It is one thing to train a model and another to deploy it and optimize. After you are done training a model, you want it to be as fast as possible during inference.
We go from .pt → ONNX → TensorRT
TensorRT seems to be platform specific
Resources
- How to Convert a Model from PyTorch to TensorRT and Speed Up Inference
- Understanding Nvidia TensorRT for deep learning model optimization
- This repo has a bunch of models with already transformed into ONNX: https://github.com/PINTO0309/PINTO_model_zoo
- Check out this GitHub ONNX Inference Template.
Task 1 for WATonomous
First task: Look into ONNX for yolov5 and see how it performs on inference.
- YoloV5 allows you to do inference using a pretrained model. ONNX is just a kind of format that they use, which is faster
Rosbags are found in /mnt/wato-drive/rosbags/year5/, this basically feeds through the Rosbridge
To compare speed, run ROS
rostopic list
rostopic hz <name>
Installation
python export.py --weights yolov5m.pt --include engine --device 0
Testing
python detect.py --weights yolov5s.onnx --source dog.png
Big file: VRAM → use nvidia-smi
To look into: Deployment here