MVP Integration

[Ishaan-Datta]

Stage 1: Bare Minimum Python Implementation (Ultralytics API) -> Testing on Oct 26th

• !!! under examples/ROS2 Yolo Nodes there should be a template for exactly what you need to do
• Load the model and run inference using the API (in the appropriate nodes)
• Preprocessing should be automatic assuming you specify the input size
• Postprocessing should automatically scale the bounding boxes and can save them as an object you can work with
• Make a script for recording Zed frames as pictures in a folder for data collection

Extermination ROI Output:

• Ask for camera back from navigation and copy example for working with multiple cameras
• ROI should be centered around the sprayer point region (crop to arbitrary dimensions without resizing)
• Post-processing (crop the image to the resized bounding boxes and pass them through the color filter for more accurate boxes)
• (check if the more accurate boxes are within the scaled ROI (sprayer area) and publish 1 or 0 every 0.10 seconds)

Stage 2: C++ Foundations -> Testing on March 8th

• Split the Node functionality into modules

Stage 3: C++ Optimizations -> Testing on May 3rd
ROS2 Specific Features: (with Aqil)

• Integrated Tracing
• Custom Message Types
• Composable Containers/Intra-Process Communication
• Multi-threaded Executor

CUDA Specific Features: (with Kezia)

• Implement best performing CUDA operations/TensorRT Engine Plugins

Model Specific Features: (with Amy)

• Implement best performing converted TensorRT Engine
• Implement Ultralytics Examples for C++ TensorRT Inference (/examples/YOLOV8-*)
• Implement Nvidia Examples for C++ TensorRT Inference Google Drive