Supervisors info:
Αθανασία Αλωνιστιώτη, Επίκουρη Καθηγήτρια, Τμήμα Πληροφορικής και Τηλεπικοινωνιών, Εθνικό και Καποδιστριακό Πανεπιστήμιο Αθηνών
Summary:
We exploit the new features provided by WebRTC in terms of interoperability and state-of-the-art real-time communications, in order to develop a system for piloting unmanned vehicles using video analysis. Specifically, we define a topology where a ROS-based vehicle transmits its video using WebRTC to an intermediate server, who in turn relays it to a client. The server takes advantage of the OpenCV library and applies video analysis, with respect to a selected task (i.e. face detection) defined by the client. The corresponding commands are transmitted to the vehicle, resulting in an automatically driven unmanned vehicle. The client monitors the vehicle’s movement and can dynamically change the selected use case; that is, either change slightly its operation (i.e. from human tracking to children tracking) or enable an entirely new core philosophy (i.e. to fire detection) by sending the appropriate requests to the server. Upon reception of these requests, the server utilizes the corresponding OpenCV functionalities to serve the new task, and sends the new piloting commands to the vehicle, forcing the system to adopt a new autopiloting mode. This communication between the vehicle, the server and the client is established using SIP/SDP and orchestrated via a WebSocket server that serves as a Signaling Server, the media are transferred through SRTP/UDP, and the commands are carried via the WebRTC Data Channel over SCTP. We explain and describe how to combine all of these heterogeneous components (WebRTC – OpenCV – ROS), in order to compose a web-based infrastructure for autopiloting ROS-based vehicles upon a specific use case. Finally, the results prove our concept, meaning a horizontal infrastructure that (a) consists of a modular architecture, (b) provides the necessary components for machine-to-machine communication, (c) uses state-of-the-art technologies, (d) allows a developer to implement her own logic vertically, and (e) provides IoT with a solution that can be easily exploited in numerous ways.
Keywords:
WebRTC, OpenCV, web infrastructure, autopiloting, unmanned vehicles, ROS, IoT, technical instrumentation
