Publications

2020

1. BMVC

   HASTE: multi-Hypothesis Asynchronous Speeded-up Tracking of Events

   Alzugaray, Ignacio, and Chli, Margarita

   In British Machine Vision Virtual Conference (BMVC), Sep 2020.

   Video Abs HTML PDF Code

   Feature tracking using event cameras has experienced significant progress lately, with methods achieving comparable performance to feature trackers using traditional frame-based cameras, even outperforming them on certain challenging scenarios. Most of the event-based trackers, however, still operate on intermediate, frame-like representations generated from accumulated events, on which traditional frame-based techniques can be adopted. Attempting to harness the sparsity and asynchronicity of the event stream, other approaches have emerged to process each event individually, but they lack both in accuracy and efficiency in comparison to the event-based, frame-like alternatives. Aiming to address this shortcoming of asynchronous approaches, in this paper, we propose an asynchronous patch-feature tracker that relies solely on events and processes each event individually as soon as it gets generated. We report significant improvements in tracking quality over the state of the art in publicly available datasets, while performing an order of magnitude more efficiently than similar asynchronous tracking approaches.
2. MSSP

   Full-field structural monitoring using event cameras and physics-informed sparse identification

   Lai, Zhilu,  Alzugaray, Ignacio, Chli, Margarita, and Chatzi, Eleni

   Mechanical Systems and Signal Processing Sep 2020.

   Abs HTML

   This paper exploits a new direction of full-field structural monitoring and vibration analysis, using an emerging type of neuro-inspired vision sensors – namely event cameras. Compared to traditional frame-based cameras, event cameras offer salient benefits of resilience to motion blur, high dynamic range, and microsecond latency. Event cameras are herein exploited for structural monitoring, in order to extract dense measurements of structural response in terms of both spatial and temporal resolution. The output of an event camera is a stream of so called “events”, which is different to traditional snapshots. Due to this fundamentally different working principle, basic computer vision algorithms, such as optical flow or feature tracking, should be re-designed for processing event-based measurements. In this work, we present a novel framework termed physics-informed sparse identification, for full-field structural vibration tracking and analysis. The framework leverages sparse identification guided by assimilation of the underlying structural dynamics in the assembly of a library matrix, which is used to characterize the system’s dynamics. The stream of event data generated from event cameras is sparsely represented by means of well-chosen basis functions, allowing for a physical interpretation of the system’s response. The proposed framework is extended to boundary condition learning/classification by fusion of characteristic basis functions, representing different classes of support conditions, into the library matrix. The results obtained by means of an illustrative numerical example, as well as experimental tests on vibrating beams recorded by an event camera demonstrate an accurate tracking of structural vibration and the developed strains, in the form of full-field measurements rather than point-wise tracking. What is more, the proposed sparse learning process enables identification of the boundary conditions of monitored structural elements, which comes with key benefits for structural monitoring.

3. IROS

   IDOL: A Framework for IMU-DVS Odometry using Lines

   Le Gentil, Cedric, Tschopp, Florian,  Alzugaray, Ignacio, Vidal-Calleja, Teresa, Siegwart, Roland, and Nieto, Juan

   In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Oct 2020.

   Abs HTML PDF

   In this paper, we introduce IDOL, an optimization-based framework for IMU-DVS Odometry using Lines. Event cameras, also called Dynamic Vision Sensors (DVSs), generate highly asynchronous streams of events triggered upon illumination changes for each individual pixel. This novel paradigm presents advantages in low illumination conditions and high-speed motions. Nonetheless, this unconventional sensing modality brings new challenges to perform scene reconstruction or motion estimation. The proposed method offers to leverage a continuous-time representation of the inertial readings to associate each event with timely accurate inertial data. The method’s front-end extracts event clusters that belong to line segments in the environment whereas the back-end estimates the system’s trajectory alongside the lines’ 3D position by minimizing point-to-line distances between individual events and the lines’ projection in the image space. A novel attraction/repulsion mechanism is presented to accurately estimate the lines’ extremities, avoiding their explicit detection in the event data. The proposed method is benchmarked against a state-of-the-art frame-based visual-inertial odometry framework using public datasets. The results show that IDOL performs at the same order of magnitude on most datasets and even shows better orientation estimates. These findings can have a great impact on new algorithms for DVS.

2019

1. 3DV

   Asynchronous Multi-Hypothesis Tracking of Features with Event Cameras

   Alzugaray, Ignacio, and Chli, Margarita

   In International Conference on 3D Vision (3DV), Sep 2019.

   Video Abs HTML PDF Code

   With the emergence of event cameras, increasing research effort has been focusing on processing the asynchronous stream of events. With each event encoding a discrete intensity change at a particular pixel, uniquely time-stamped with high accuracy, this sensing information is so fundamentally different to the data provided by traditional frame-based cameras that most of the well-established vision algorithms are not applicable. Inspired by the need of effective event-based tracking, this paper addresses the tracking of generic patch features relying solely on events, while exploiting their asynchronicity and high-temporal resolution. The proposed approach outperforms the state-of-the-art in event-based feature tracking on well-established event camera datasets, retrieving longer and more accurate feature tracks at higher a frequency. Considering tracking as an optimization problem of matching the current view to a feature template, the proposed method implements a simple and efficient technique that only requires the evaluation of a discrete set of tracking hypotheses.

2018

1. 3DV

   ACE: An Efficient Asynchronous Corner Tracker for Event Cameras

   Alzugaray, Ignacio, and Chli, Margarita

   In International Conference on 3D Vision (3DV), Sep 2018.

   Video Abs HTML PDF

   The emergence of bio-inspired event cameras has opened up new exciting possibilities in high-frequency tracking, overcoming some of the limitations of traditional frame-based vision (e.g. motion blur during high-speed motions or saturation in scenes with high dynamic range). As a result, research has been focusing on the processing of their unusual output: an asynchronous stream of events. With the majority of existing techniques discretizing the event-stream into frame-like representations, we are yet to harness the true power of these cameras. In this paper, we propose the ACE tracker: a purely asynchronous framework to track corner-event features. Evaluation on benchmarking datasets reveals significant improvements in accuracy and computational efficiency in comparison to state-of-the-art event-based trackers. ACE achieves robust performance even in challenging scenarios, where traditional frame-based vision algorithms fail.
2. RA-L / IROS

   Asynchronous Corner Detection and Tracking for Event Cameras in Real Time

   Alzugaray, Ignacio, and Chli, Margarita

   IEEE Robotics and Automation Letters Oct 2018.

   Video Abs HTML PDF Code

   The recent emergence of bioinspired event cameras has opened up exciting new possibilities in high-frequency tracking, bringing robustness to common problems in traditional vision, such as lighting changes and motion blur. In order to leverage these attractive attributes of the event cameras, research has been focusing on understanding how to process their unusual output: an asynchronous stream of events. With the majority of existing techniques discretizing the event-stream essentially forming frames of events grouped according to their timestamp, we are still to exploit the power of these cameras. In this spirit, this letter proposes a new, purely event-based corner detector, and a novel corner tracker, demonstrating that it is possible to detect corners and track them directly on the event stream in real time. Evaluation on benchmarking datasets reveals a significant boost in the number of detected corners and the repeatability of such detections over the state of the art even in challenging scenarios with the proposed approach while enabling more than a 4× speed-up when compared to the most efficient algorithm in the literature. The proposed pipeline detects and tracks corners at a rate of more than 7.5 million events per second, promising great impact in high-speed applications.
3. FSR

   Autonomous Aerial Inspection Using Visual-Inertial Robust Localization and Mapping

   Teixeira, Lucas,  Alzugaray, Ignacio, and Chli, Margarita

   In Field and Service Robotics, Oct 2018.

   Abs HTML PDF

   With recent technological breakthroughs bringing fully autonomous inspection using small Unmanned Aerial Vehicles (UAVs) closer to reality, the community of Robotics has actively been developing the real-time perception capabilities able to run onboard such constraint platforms. Despite good progress, realistic deployment of autonomous UAVs in GPS-denied environments is still rudimentary. In this work, we propose a novel system to generate a collision-free path towards a user-specified inspection direction for a small UAV using monocular-inertial sensing only and performing all computation onboard. Estimating both the previously unknown scene and the UAV’s trajectory on the fly, this system is evaluated on real experiments outdoors in the presence of wind and poorly structured environments. Our analysis reveals the shortcomings of using sparse feature maps for planning, highlighting the importance of robust dense scene estimation proposed here.

2017

1. ICRA

   Short-term UAV path-planning with monocular-inertial SLAM in the loop

   Alzugaray, Ignacio, Teixeira, Lucas, and Chli, Margarita

   In IEEE International Conference on Robotics and Automation (ICRA), May 2017.

   Video Abs HTML PDF

   Small Unmanned Aerial Vehicles (UAVs) are some of the most promising robotic platforms in a variety of applications due to their high mobility. Their restricted computational and payload capabilities, however, translate into significant challenges in automating their navigation. With Simultaneous Localization And Mapping (SLAM) systems recently demonstrated to be employable onboard UAVs, the focus fall on path-planning on the quest of achieving autonomous navigation. With the vast body of path-planning literature often assuming perfect maps or maps known a priori, the biggest challenge lies in dealing with the robustness and accuracy limitations of onboard SLAM in real missions. In this spirit, this paper proposes a path-planning algorithm designed to work in the loop of the SLAM estimation of a monocular-inertial system. This point-to-point planner is demonstrated to navigate in an unknown environment using the incrementally generated SLAM map, while dictating the navigation strategy for preferable acquisition of sensor data for better estimations within SLAM. A thorough evaluation testbed of both simulated and real data is presented, demonstrating the robustness of the proposed pipeline against the state-of-the-art and its dramatically lower computational complexity, revealing its suitability to UAV navigation.

2016

1. IROS

   Learning the hidden human knowledge of UAV pilots when navigating in a cluttered environment for improving path planning

   Alzugaray, Ignacio, and Sanfeliu, Alberto

   In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Oct 2016.

   Abs HTML PDF

   We propose in this work a new model of how the hidden human knowledge (HHK) of UAV pilots can be incorporated in the UAVs path planning generation. We intuitively know that human’s pilots barely manage or even attempt to drive the UAV through a path that is optimal attending to some criteria as an optimal planner would suggest. Although human pilots might get close but not reach the optimal path proposed by some planner that optimizes over time or distance, the final effect of this differentiation could be not only surprisingly better, but also desirable. In the best scenario for optimality, the path that human pilots generate would deviate from the optimal path as much as the hidden knowledge that its perceives is injected into the path. The aim of our work is to use real human pilot paths to learn the hidden knowledge using repulsion fields and to incorporate this knowledge afterwards in the environment obstacles as cause of the deviation from optimality. We present a strategy of learning this knowledge based on attractor and repulsors, the learning method and a modified RRT* that can use this knowledge for path planning. Finally we do real-life tests and we compare the resulting paths with and without this knowledge.