Introduction

The Vargi Bots theme implementation(Task5) : This theme was inspired by the idea of Industry 4.0. This task was focused on automating the operations that are involved in the Warehouse Management System(W.M.S.). The objective of this task is to implement a Warehouse Management System to sort packages based on incoming customer orders from different cities. The arena is an automated warehouse setting where essential packages are required to be sent out to different parts of the country. The warehouse consists of two industrial robotic arms, UR5_1 and UR5_2 as the theme mainly focuses on automation(like pick,place and sort operations).As the orders are received, UR5_1 decides accordingly and serves the order of utmost priority first. It is responsible for dispatching the order. An order gets dispatched when a package is picked from the shelf and placed on the conveyor belt. Once the dispatched order reaches UR5_2,it is picked up and dropped into the corresponding colored bin.An order gets shipped when a package is dropped into its corresponding bin. Not only this,but the occurrence of these events(packages detection by 2D camera,incoming orders,dispatching,shipping) are updated in the appropriate google sheet in real time and apt email alerts are also sent to the user(dispatching and shipping details).

Implementation

The ideas that led to the implementation of this theme are as follows:

  • The first step in this task is to detect the packages as well as their color, as sensed by the 2D camera. We did this by using computer vision techniques like cropping,horizontal and vertical concatenation, drawing bounding box, and by extracting the QR code data to get the colors of the packages placed on the shelf. The color of the packages are stored in a list and getting published on topic "/topic_t5_QR_code_data" so that any other node subscribing to it can use it for further manipulation.These packages’ information is getting updated in the Inventory sheet as well.
     

  • The next step is to receive incoming orders via ros iot bridge. For this purpose, in the Action Server script, we have subscribed to the mqtt topic on which the incoming order messages are getting published,i.e.,"/eyrc/vb/XYZabcVB/orders".The action server needs to publish these details on the topic “/ros_iot_bridge/mqtt/sub” for any other ros node to access it as well.These orders are then getting updated in the IncomingOrders sheet.
     

  • The next step is to decide the order to be prioritized. As an order is given priority over others, the corresponding saved trajectories are traced by the UR5_1 robotic arm.Once an order is dispatched, it's details are updated in the OrdersDispatched sheet. Also, these details are published on the topic “/topic_t5_dispatch_details”.
     

  • The next part involves the packages getting picked and sorted accordingly. The conveyor belt is stopped once a package is at an appropriate position to get picked up.The target bin is found out on the basis of the dispatch details of the package.The translations are calculated and performed by the robotic arm, the package is picked up and is then dropped in the appropriate bin. Hence the order is shipped and these shipping details are updated in the OrdersShipped sheet.
     

  • Email alerts are sent to the user when their order is dispatched or shipped. The Dashboard sheet is updated simultaneously as the other sheets are updated. The contents of this sheet are visually represented in the Warehouse Dashboard webpage(table, markers in map, bar graph) and are updated in real time.
     

  • All the sheet updation is done via Ros-IoT bridge only.


 

API Documentation

  • pkg_ros_iot_bridge:
     

This package is responsible for performing all tasks to link IoT and Ros. The msgRosIot.action file is the action file in the action folder used for goal initiation and tracking. The config_pyiot.yaml file in the config folder contains all the necessary configurations like sub topics,pub topics, spreadsheet ids,qos etc. The python script node_action_server_ros_iot_bridge.py is responsible to accept goals from the Action Clients. On receiving an appropriate goal, the Action Server pushes as many data points on the google sheet at once. This node subscribes to the mqtt topic /eyrc/vb/XYZabcVB/orders in order to get the incoming order messages. These messages are then published on another topic /ros_iot_bridge/mqtt/sub for the ros nodes to access it. The message file msgMqttSub.msg is used for this purpose.
 

  • pkg_task5:
     

The files present in the "rviz folder" are the scene files for ur5_1 and ur5_2. The "saved_trajectories2" folder contains all the saved trajectory yaml files which are played in the ur5_1 script for smooth pick and place operations. The script node_t5_camera_1.py is responsible for finding the color of all the packages placed on the shelf. This is done by dividing the image captured into subparts, extracting information from the QR code data for each package. This information is published on the topic /topic_t5_QR_code_data. The message file colorListMessage.msg is used for this purpose.
 

The script node_t5_ur5_1.py is the node that controls the ur5_1 robotic arm. The goals for updating Inventory sheet, IncomingOrders sheet and the OrdersDispatched sheet are to the Action Server in this script only. This node subscribes to topic /topic_t5_QR_code_data to get the color of all packages. This information is further manipulated to get parameters for updating the Inventory sheet. This node also subscribes to ros topic /ros_iot_bridge/mqtt/sub to get the incoming order details for updating the IncomingOrders sheet. When the order placing starts, decision regarding which order to prioritize is made(HP-Medicine,MP-Food,LP-Clothes).Once it is decided, the corresponding saved trajectories are played for the ur5_1 arm to pick the package from the shelf and place it on the conveyor belt.
 

The dispatch details are also published on the ros topic /topic_t5_dispatch_details. The message file dispatchDetails.msg is used for this purpose. The goal to update the OrdersDispatched sheet is sent to the Action Server at that very instant. The script node_t5_ur5_2.py is the node that controls the ur5_2 robotic arm. This node subscribes to the ros topic /topic_t5_dispatch_details to get the details of the order that has been dispatched recently. A conditional check is implemented in this script to stop the conveyor belt when the package has reached an apt position(mid position,under the logical camera2). Cartesian translations are calculated and performed in order to make the gripper of ur5_2 robotic arm reach the package. The target bin is found using the dispatch details of that package. The package is picked up and dropped in the appropriate bin on the basis of that. The shipping details are manipulated for that package and a goal is sent to the Action Server to update the ShippedOrders sheet at that very instant.

Dashboard

User Dashboard consists of a :
MAP - Geolocation showing information about Inventory order,
TABLE - Table showing the live status of the orders in the dashboard which includes the following information &,
GRAPH - Graph showing the status of time taken for each order

  • When an order arrives, a new row is appended to the table and it's background color is red.

  • The shelf-side robot picks up the order (& sorts orders if necessary according to priority) and places it on the conveyer belt. This is "ORDER DISPATCHED" event. When the order is dispatched the row turns yellow in color.

  • When the package reaches bin-side robot, it is sorted and placed according to it's color in respective bins.

  • Google Spreadsheets as json-endpoint act as backend for the data required by dashboard.

See Dashboard