Real-Time Obstacle Detection and Visualization System (RTOVS)

Added: 2026-05-25

[00:00:05]Hi everyone, welcome to our graduation project presentation video. I'm Hussein.

[00:00:10]This project aims to improve real-time obstacle detection and visualization system for autonomous vehicles and I was responsible for the MQTT communication protocol and metadata metadata stream.

[00:00:22]Uh also co-development of the GUI. I'm Sanchuk and uh I'm also responsible for uh co-development of GUI and uh I'm also responsible to uh improving the our uh AI data.

[00:00:41]>> I am Sajan. For this project, I handle the data set customization and also the YOLO V5 model training for object detection. I'm Makub and I'm responsible for guitar sensor integration and trade score formalization. I'm Ahmed. In this project, my responsibility is Jal environment setup and HPI optimization.

[00:01:03]>> What are you watching on the screen right now is our custom trained Artos AI actively processing real dash cam footage. As we move through dance urban traffic, notice the bounding boxes. The model is simultaneously tracking our seven targeted classes because we apply the strict balancing operation to our data set. You can see that the AI does not just focus on the massive number of cars. It perfectly isolates vulnerable targets like pedestrians, two wheelers, and read the traffic lights with high confidence.

[00:01:32]Although we are operating at high frames per second, the bounding boxes remain incredibly stable without flickering even when objects are partially hidden or overlapping. One of our biggest achievements is local customization.

[00:01:45]Standard global models often fail in regional environments. Notice how our model specifically recognizes Turkish stop sign. By fusing these highly accurate visual targets with our LAR depth data, we give our autonomous system the ultimate spatial awareness.

[00:02:05]Uh here on the JSON Nano, you can see our LE camera feed. Uh our custom sevenclass model detects objects in real time and uh drafts uh bounding boxes with their names. Uh to get high FPS on this device uh we used Tensor RT FP16 optimization and Pya for better GPU memory control. Uh finally uh by celebrating the camera with our leadersh uh the system calculates the exact distance uh and uh speed of the object instantly.

[00:02:43]We formulated two different values trait A and trade B. Trait A represents the danger we pose the object and trade B represents danger the other side posed to us.

[00:02:57]Uh so expect a score increase while pedestrian is approached to the camera.

[00:03:03]You can see from the graph and we expect an increment on the trade B score while heavy vehicle is approached.

[00:03:18]So you can see from the graph a trade base score is increased.

[00:03:23]Uh if the object is car TA and TB will be affected by the action and you can see from the graph trade A and trade B is increased. Uh our Jetson nano edge device successfully detects objects and has a role of both publisher and broker in the MQTT protocol. Uh it shows the active threats on the screen.

[00:03:47]So it detects three objects and now when I took one of them out we have two.

[00:03:53]Uh it sends uh 20 packets per second. Uh and this metadata includes different parameters such as distance, angle, class, speed etc. Uh our subscriber device receives these metadata correctly and visualization of these data datas can be observed.

[00:04:11]Uh as we can see on this uh plane this planes is represent the um our live feed camera uh perspective and uh we can see the detected objects exact location on this perspective uh as we can see this area and uh at the the right side table uh we see the obstacles uh their classes and their uh location information and their speed situations and also we can see the their uh threat level uh numbers.