2024
Kumar, Aakash; Chen, Chen; Mian, Ajmal; Lobo, Neils; Shah, Mubarak
Sparse Points to Dense Clouds: Enhancing 3D Detection with Limited LiDAR Data Conference
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2024.
Abstract | Tags: IROS | Links:
@conference{Kumar2024,
title = {Sparse Points to Dense Clouds: Enhancing 3D Detection with Limited LiDAR Data},
author = {Aakash Kumar and Chen Chen and Ajmal Mian and Neils Lobo and Mubarak Shah},
url = {https://www.crcv.ucf.edu/wp-content/uploads/2018/11/2404.06715v1.pdf
https://aakashjuseja-aj.github.io/Sparse_to_Dense/},
year = {2024},
date = {2024-10-14},
publisher = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
abstract = {3D detection is a critical task that enables machines to identify and locate objects in three-dimensional space. It has a broad range of applications in several fields, including autonomous driving, robotics and augmented reality. Monocular 3D detection is attractive as it requires only a single camera, however, it lacks the accuracy and robustness required for real world applications. High resolution LiDAR on the other hand, can be expensive and lead to interference problems in heavy traffic given their active transmissions. We propose a balanced approach that combines the advantages of monocular and point cloud-based 3D detection. Our method requires only a small number of 3D points, that can be obtained from a low-cost, low-resolution sensor. Specifically, we use only 512 points, which is just 1% of a full LiDAR frame in the KITTI dataset. Our method reconstructs a complete 3D point cloud from this limited 3D information combined with a single image. The reconstructed 3D point cloud and corresponding image can be used by any multi-modal off-the-shelf detector for 3D object detection. By using the proposed network architecture with an off-the-shelf multi-modal 3D detector, the accuracy of 3D detection improves by 20% compared to the state-of-the-art monocular detection methods and 6% to 9% compare to the baseline multi-modal methods on KITTI and JackRabbot datasets.},
keywords = {IROS},
pubstate = {published},
tppubtype = {conference}
}
3D detection is a critical task that enables machines to identify and locate objects in three-dimensional space. It has a broad range of applications in several fields, including autonomous driving, robotics and augmented reality. Monocular 3D detection is attractive as it requires only a single camera, however, it lacks the accuracy and robustness required for real world applications. High resolution LiDAR on the other hand, can be expensive and lead to interference problems in heavy traffic given their active transmissions. We propose a balanced approach that combines the advantages of monocular and point cloud-based 3D detection. Our method requires only a small number of 3D points, that can be obtained from a low-cost, low-resolution sensor. Specifically, we use only 512 points, which is just 1% of a full LiDAR frame in the KITTI dataset. Our method reconstructs a complete 3D point cloud from this limited 3D information combined with a single image. The reconstructed 3D point cloud and corresponding image can be used by any multi-modal off-the-shelf detector for 3D object detection. By using the proposed network architecture with an off-the-shelf multi-modal 3D detector, the accuracy of 3D detection improves by 20% compared to the state-of-the-art monocular detection methods and 6% to 9% compare to the baseline multi-modal methods on KITTI and JackRabbot datasets.
Arul, S. H.; Bedi, A. S.; Manocha, D.
When, What, and with Whom to Communicate: Enhancing RL-based Multi-Robot Navigation through Selective Communication Conference
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2024.
Tags: IROS
@conference{Arul2024,
title = {When, What, and with Whom to Communicate: Enhancing RL-based Multi-Robot Navigation through Selective Communication},
author = {S. H. Arul and A. S. Bedi and D. Manocha},
year = {2024},
date = {2024-10-14},
publisher = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
keywords = {IROS},
pubstate = {published},
tppubtype = {conference}
}
Shek, C. L.; Wu, X.; Suttle, W. A.; Busart, C.; Zaroukian, E.; Manocha, D.; Tokekar, P.; Bedi, A. S.
LANCAR: Leveraging Language for Context-Aware Robot Locomotion in Unstructured Environments Conference
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2024.
Tags: IROS | Links:
@conference{Shek2024,
title = {LANCAR: Leveraging Language for Context-Aware Robot Locomotion in Unstructured Environments},
author = {C. L. Shek and X. Wu and W. A. Suttle and C. Busart and E. Zaroukian and D. Manocha and P. Tokekar and A. S. Bedi},
url = {https://www.crcv.ucf.edu/wp-content/uploads/2018/11/2310.00481v2.pdf},
year = {2024},
date = {2024-10-14},
publisher = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
keywords = {IROS},
pubstate = {published},
tppubtype = {conference}
}
Sun, Xingpeng; Zhang, Yiran; Tang, Xindi; Bedi, Amrit Singh; Bera, Aniket
TrustNavGPT: Trust-Driven Audio-Guided Robot Navigation under Uncertainty with Large Language Models Conference
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) [Oral], 2024.
Abstract | Tags: IROS | Links:
@conference{Sun2024b,
title = {TrustNavGPT: Trust-Driven Audio-Guided Robot Navigation under Uncertainty with Large Language Models},
author = {Xingpeng Sun and Yiran Zhang and Xindi Tang and Amrit Singh Bedi and Aniket Bera},
url = {https://xingpengsun0.github.io/trustnav/},
year = {2024},
date = {2024-10-14},
publisher = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) [Oral]},
abstract = {Large language models (LLMs) exhibit a wide range of promising capabilities -- from step-by-step planning to commonsense reasoning --that provide utility for robot navigation. However, as humans communicate with robots in the real world, ambiguity and uncertainty may be embedded inside spoken instructions. While LLMs are proficient at processing text in human conversations, they often encounter difficulties with the nuances of verbal instructions and, thus, remain prone to hallucinate trust in human command. In this work, we present TrustNavGPT, an LLM-based audio-guided navigation agent that uses affective cues in spoken communication—elements such as tone and inflection that convey meaning beyond words—allowing it to assess the trustworthiness of human commands and make effective, safe decisions.},
keywords = {IROS},
pubstate = {published},
tppubtype = {conference}
}
Large language models (LLMs) exhibit a wide range of promising capabilities -- from step-by-step planning to commonsense reasoning --that provide utility for robot navigation. However, as humans communicate with robots in the real world, ambiguity and uncertainty may be embedded inside spoken instructions. While LLMs are proficient at processing text in human conversations, they often encounter difficulties with the nuances of verbal instructions and, thus, remain prone to hallucinate trust in human command. In this work, we present TrustNavGPT, an LLM-based audio-guided navigation agent that uses affective cues in spoken communication—elements such as tone and inflection that convey meaning beyond words—allowing it to assess the trustworthiness of human commands and make effective, safe decisions.
Kulbaka, Iliya; Dutta, Ayan; Kreidl, O. Patrick; Bölöni, Ladislau; Roy, Swapnoneel
GDM-Net: gas distribution mapping with a mobile robot using deep reinforcement learning and gaussian process regression Conference
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2024.
Tags: IROS
@conference{nokey,
title = {GDM-Net: gas distribution mapping with a mobile robot using deep reinforcement learning and gaussian process regression},
author = {Iliya Kulbaka and Ayan Dutta and O. Patrick Kreidl and Ladislau Bölöni and Swapnoneel Roy},
year = {2024},
date = {2024-10-14},
publisher = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
keywords = {IROS},
pubstate = {published},
tppubtype = {conference}
}
2023
de Blegiers, Tristan; Dave, Ishan Rajendrakumar; Yousaf, Adeel; Shah, Mubarak
EventTransAct: A Video Transformer-based Framework for Event-camera Based Action Recognition Conference
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2023.
Tags: IROS | Links:
@conference{deBlegiers2023,
title = {EventTransAct: A Video Transformer-based Framework for Event-camera Based Action Recognition},
author = {Tristan de Blegiers and Ishan Rajendrakumar Dave and Adeel Yousaf and Mubarak Shah},
url = {https://arxiv.org/pdf/2308.13711
https://github.com/tristandb8/EventTransAct
https://tristandb8.github.io/EventTransAct_webpage/
https://www.youtube.com/watch?v=YCff-rTrgco},
year = {2023},
date = {2023-08-25},
publisher = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
keywords = {IROS},
pubstate = {published},
tppubtype = {conference}
}
2022
Kumar, Aakash; Kini, Jyoti; Mian, Ajmal; Shah, Mubarak
Self Supervised Learning for Multiple Object Tracking in 3D Point Clouds Conference
2022 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2022.
Abstract | Tags: IROS | Links:
@conference{Kumar2022,
title = {Self Supervised Learning for Multiple Object Tracking in 3D Point Clouds},
author = {Aakash Kumar and Jyoti Kini and Ajmal Mian and Mubarak Shah},
url = {https://www.crcv.ucf.edu/wp-content/uploads/2018/11/camera_ready_paper.pdf},
year = {2022},
date = {2022-10-23},
urldate = {2022-10-23},
booktitle = {2022 IEEE/RSJ International Conference on Intelligent Robots and Systems},
abstract = {Multiple object tracking in 3D point clouds has applications in mobile robots and autonomous driving. This is a challenging problem due to the sparse nature of the point clouds and the added difficulty of annotation in 3D for supervised learning. To overcome these challenges, we propose a neural network architecture that learns effective object features and their affinities in a self supervised fashion for multiple object tracking in 3D point clouds captured with LiDAR sensors. For self supervision, we use two approaches. First, we generate two augmented LiDAR frames from a single real frame by applying translation, rotation and cutout to the objects. Second, we synthesize a LiDAR frame using CAD models or primitive geometric shapes and then apply the above three augmentations to them. Hence, the ground truth object locations and associations are known in both frames for self supervision. This removes the need to annotate object associations in real data, and additionally the need for training data collection and annotation for object detection in synthetic data. To the best of our knowledge, this is the first self supervised multiple object tracking method for 3D data. Our model achieves state of the art results.},
keywords = {IROS},
pubstate = {published},
tppubtype = {conference}
}
Multiple object tracking in 3D point clouds has applications in mobile robots and autonomous driving. This is a challenging problem due to the sparse nature of the point clouds and the added difficulty of annotation in 3D for supervised learning. To overcome these challenges, we propose a neural network architecture that learns effective object features and their affinities in a self supervised fashion for multiple object tracking in 3D point clouds captured with LiDAR sensors. For self supervision, we use two approaches. First, we generate two augmented LiDAR frames from a single real frame by applying translation, rotation and cutout to the objects. Second, we synthesize a LiDAR frame using CAD models or primitive geometric shapes and then apply the above three augmentations to them. Hence, the ground truth object locations and associations are known in both frames for self supervision. This removes the need to annotate object associations in real data, and additionally the need for training data collection and annotation for object detection in synthetic data. To the best of our knowledge, this is the first self supervised multiple object tracking method for 3D data. Our model achieves state of the art results.