Public Summary Month 6/2012

In the last bimester we have worked on with the activities of Task 3: System Development, Implementation, Integration and Testing, namely on the implementation and testing of the first prototype of the VUI for the chosen domains. According to the results of the design phase, we are focusing on the generalized components that are the basis for the implementation of the VUI and development of the wizard is moved to a later stage. More specifically, we have a complete modular grammar for the humanoid robot in the home domain, according to the methodology described in the appendix of the Technical Report D21. To this end, we have updated the command description therein, since we have now released a more polished and complete set reference for the commands of the robot. In addition, we have refined the software architecture and implemented a simple prototype version of the components for selecting the interpretation and managing the dialog with the user. Moreover, we have started the work on Task 4: Experimental Evaluation; in particular, we have begun working on Task T4.1: Definition of experimental set-up and performance metrics. Finally, a poster describing the current state of the project has been accepted for presentation at AAAI-12 Media Fair.


Public Summary Month 4/2012

In the last bimester, we have consolidated the S4R activities related with Task 1, and we have completed the S4R activities planned in Task 2, concerning the design of the process for the generation Vocal User Interface for specific robotic applications. The details are available in the Technical report released at the end of Task 2 (April 30th). In March, S4R decided to discontinue the development with the ASR Loquendo (due to unavailability of the product), and adopted the Microsoft Speech Platform, which required an additional integration effort, after which the new platform became functionally compatible with the previous one. In addition, we have implemented the interface for testing our interface on the Cobot platform developed at Carnegie Mellon University, as an additional robotic platform to support a solution that can be easily customized to different robotic platforms. In March, the prototype implementation of S4R on the NAO platform has been demonstrated at RomeCup 2012 (a RoboCup Junior event, held on March 23, 2012 in the Rome City Hall) and tested by the vice mayor of Rome.


Public Summary Month 2/2012

We have built two example collection of commands to model the two application domains. The sentences have been collected from the students of the class “Seminars AI and Robotics”, section on Natural Language Processing. The goal was experimenting the process of acquiring domain specific information, and the collections represent a first characterization of domain specific knowledge, in terms of robot capabilities, environments and tasks, not the final resources for language generation. We have combined the collection of the lexical units defined in the RoboFrameNet environment (available in Ros), with the more general representation provided by Wordnet and defined a first characterization of the possible wordings of the commands.

We have defined a methodology for testing the performance of our system. The baseline will be a system where a set of sentences from a corpus is associated with a set of commands. The performance of the customized system will be compared through systematic experiments with the baseline, on a set of vocal data recorded from different users. A dedicated tool has been built in order to support the systematic evaluation of the system performance.

The first technical report provides a comprehensive report of the first activities carried out in the project.


Public Summary Month 12/2011

Experiment started in November still pending the signature of the contract.

Activities have focussed on the analysis and initial steps in the design. Speciifcally: Robotic Domain Definition and Representation and Robotic Voice Development Kit Design

 

The main steps accomplished in task 1 are the implementation of  a prototype to interact using  Speaky both with the NAO humanoid robot by Aldebaran and with a small wheeled Erratic robot by Videre Design. The goal is to acquire some practical experience on the possible wordings used to command the robot. In the implementation of the above mentioned prototypes we have provided a first characterization of the robot capabilities, both for a humanoid robot and for a wheeled robot.

 The main steps accomplished in task 2 consist of a first architectural design in the implementation of the above described prototype, which is specifically concerned with the communication between Speaky and the mobile platform.We have also identified the internal structure of the speaky devolpment environment and identified two key components: the dynamic grammar manager and the the dialogue manager. The first one is instrumental to provide suitable data for a linguistic post-processing of the output of the speech recognition and the second one needs to integrate the state of the dialogue with the state of the robot.