Alexis Maldonado & Georg Bartels Specifications & Requirements from Universität Bremen Alexis Maldonado & Georg Bartels Universität Bremen

Table of Contents Scenario Contributions Hardware Platforms & Testbeds Software Components Specifications Software Components Requirements

Scenario Contributions

Bremen sees itself driving both scenario processes Autonomous scanning unit as robotic platform Continuous updates of semantic store maps Know which SKU per facing Know how full per facing Know background information of SKU (meshes, weight, etc.)

1b) Shelf monitoring: perceiving products on shelves Scenario 1 1a) Layout identification: perceiving bar-codes on price labels per facing 1b) Shelf monitoring: perceiving products on shelves No counting of individual products with fixed sensor Distinguish full, partially full and empty facings Detect misplaced products at the front of facings

Bremen sees itself only in a minor role for this scenario. Not involved in the multi-robot control /coordination. Not involved in depalletizing. Bremen does provide the semantic store maps Provide query interface for accessing the required data for this scenario. Model the required SKU background knowledge for this scenario.

Scenario 3 Bremen sees itself driving this scenario together with other scientific partners Keep specs and evaluation criteria as open as possible Autonomous transport unit and mobile handling unit as robotic platforms Focus on issues in mobile manipulation not complex multi- robot coordination

Hardware Platforms & Testbeds

Platforms at Bremen Mockup Mockup environment almost ready and available to all partners as of right now “Boxy” robot as stand- in for mobile handling unit and autonomous scanning unit Wrist-mounted Intel sensor Additional omni- directional robot as autonomous transport unit.

Platforms at Bremen Mockup Strategy until 1st review Push Scenario 1 and Scenario 3 in Bremen as far as possible Have prototypes running with partner software fully integrated on Bremen platforms Obtain a realistic impression of needs of (and holes in?) both Scenarios To do this, Bremen has to go beyond its role as a scientific partner and also play the role of an integrator Willing to do this but only on our platforms. Committed to open source and open data research.

Platforms at DM pilot shop To ensure successful transfer of software developed in 1st phase Robotic units developed in REFILLS need to come with equivalent (ideally identical) ROS interfaces Delivery dates in work plan WP2 mean hardware and software ready and tested Location of DM pilot shop? Technology transfer to industry partners not a goal

Software Component Specifications

Contain the ‘Realogram’ information Semantic Store Maps Contain the ‘Realogram’ information 3D representation of the shelves with product facings over time 3D representation of products on shelves over time Background knowledge of SKUs Runtime interface to query, add, edit, and delete of information Export information into files for exchange and persistent storage Import information from files Graphical 3D user interface

Visual Perception Framework Detection, recognition, and localization of products on shelves, free shelf space, and bar-codes Robust, flexible, and fast performance Render expected images using current belief state Specify specific perception tasks through DSL Optimized low-level routines No counting of individual products; free shelf space only qualitatively

Plan Interpretation System Provide plan-based and knowledge- enabled plan interpretation systems Acts as high-level executive with failure handling Logs performed actions and failures for subsequent analysis and learning Programming on an abstract level using entity descriptions Tight integration with perception system, reactive controllers, and manipulation knowledge base

Manipulation Knowledge Base Calculates context- specific motion commands for reactive control layer Collision free paths Parametrize low-level controllers with object- specific information Parametrize low-level event detection routines Configure low-level event reactions Aim for stereotypic motions

Software Component Requirements

SLAM maps and semantic store maps need to be aligned Coordinate with partners that do SLAM on REFILLS robots Clearly defined file format for data input and output Unclear requirements How many shelves? How many facings? How many products per facings? How many scans stored in maps? Multi-client support? Support for certain batch queries? Runtime requirements for specific queries

Visual Perception Framework Various ROS interfaces needed Consumes streams of RGB-D and RGB data via topics from sensors Offers service interface to trigger perception tasks (with DSL to specify perception tasks and return perception results) Uses service interface to semantic store maps to obtain environment maps and object information Uses service interface to manipulation knowledge bases to obtain robot information (proprioception, sensors placements, telemetry over time)

Plan Interpretation System Various ROS interfaces needed Queries perception framework for perception tasks Queries semantic store maps for environment knowledge Queries manipulation knowledge bases for motion commands Commands reactive control layer No direct direct interface to robots Reactive control layer for commands and non-visual perception Perception framework for visual perception Manipulation knowledge bases for proprioception

Manipulation Knowledge Base Need URDF models of REFILLS robots Autonomous scanning unit, autonomous transport unit, mobile handling unit REFILLS robots need to provide various ROS interfaces Telemetry through ROS topics SLAM via ROS topics Base, gripper, pan-tilt, and arm movements via ROS actions or topics (to be detailed in D2.1, more informed after first project phase) Uses service interface to semantic store maps to obtain environment models Uses service interface to motion planning suite (probably MoveIt!)

Suggested target operating systems and ROS versions Basic requirements Suggested target operating systems and ROS versions First phase: Ubuntu 16.04 and ROS Kinetic Rest of REFILLS: Ubuntu 18.04 and ROS M Unclear requirements Required computation performance Required network characteristics

Thank you for your attention! Q&A Thank you for your attention!