Search Results

It’s important to predict human actions in the industry assembly process. Foreseeing future actions before they happened is an essential part for flexible human-robot collaboration and crucial to safety issues. Vision-based human action prediction from videos provides intuitive and adequate knowledge for many complex applications. This problem can be interpreted as deducing the next action of people from a short video clip. The history information needs to be considered to learn these relations among time steps for predicting the future steps. However, it is difficult to extract the history information and use it to infer the future situation with traditional methods. In this scenario, a model is needed to handle the spatial and temporal details stored in the past human motions and construct the future action based on limited accessible human demonstrations.

In this paper, we present a framework for the robot to learn how to place objects to a workpiece by learning from humans in smart manufacturing. In the proposed framework, the rational scene dictionary (RSD) corresponding to the keyframes of task (KFT) are used to identify the general object-action-location relationships. The Generalized Voronoi Diagrams (GVD) based contour is used to determine the relative position and orientation between the object and the corresponding workpiece at the final state. In the learning phase, we keep tracking the image segments in the human demonstration. For the moment when a spatial relation of some segments are changed in a discontinuous way, the state changes are recorded by the RSD. KFT is abstracted after traversing and searching in RSD, while the relative position and orientation of the object and the corresponding mount are presented by GVD-based contours for the keyframes.

This paper presents an approach for performing software in the loop testing of autonomous vehicle software developed in the Autoware framework. Autoware is an open source software for autonomous driving that includes modules such as localization, detection, prediction, planning and control [8]. Multitudes of autonomous driving frameworks exist today, each having its own pros and cons. Often, MATLAB-Simulink is used for rapid prototyping, system modeling and testing, specifically for the lower-level vehicle dynamics and powertrain control features. For the autonomous software, the Robotic Operating System (ROS) is more commonly used for integrating distributed software components so that they can easily share information through a publish and subscribe paradigm. Thorough testing and evaluation of such complex, distributed software, implemented on a physical vehicle poses significant challenges in terms of safety, time, and cost, especially when considering rare edge cases.

Collaborative robots are more and more used in smart manufacturing because of their capability to work beside and collaborate with human workers. With the deployment of these robots, manufacturing tasks are more inclined to be accomplished by multiple humans and multiple robots (MH-MR) through teaming effort. In such MH-MR collaboration scenarios, the task distribution among the multiple humans and multiple robots is very critical to efficiency. It is also more challenging due to the heterogeneity of different agents. Existing approaches in task distribution among multiple agents mostly consider humans with assumed or known capabilities. However human capabilities are always changing due to various factors, which may lead to suboptimal efficiency. Although some researches have studied several human factors in manufacturing and applied them to adjust the robot task and behaviors.