A talking and drawing robot for children with communicative disabilities

This project is a cooperation between DART (Centre for Alternative and Augmentative Communication, and Assistive Technology; in Swedish: "Kommunikations- och dataresurscenter för personer med funktionshinder"), and the University of Gothenburg (department of Philosophy, Linguistics and Theory of Science). The project was financed by the Promobilia Foundation and Magn. Bergvall's Foundation.

TRIK started in September 2008 and ended in December 2009. The main contact person is Peter Ljunglöf. It was followed by the Lekbot project.

Abstract (in English)

In this project we develop and evaluate setup involving a communication board (for manual sign communication) and a drawing robot, which can communicate with each other via spoken language. The purpose is to help children with severe communication disabilities to learn language, language use and cooperation, in a playful and inspiring way. The communication board speaks and the robot is able to understand and talk back. This encourages the child to use the language and learn to cooperate to reach a common goal, which in this case is to get the robot to draw figures on a paper.

Abstract (in Swedish)

I detta projekt utvecklar vi och utvärderar en teckentavla och en ritande robot som kan kommunicera med varandra med hjälp av talat språk. Syftet är att hjälpa barn och ungdomar med svåra språkliga funktionshinder att tillgodogöra sig språket, språkliga regler och språkligt samarbete, på ett lekfullt och inspirerande sätt. Genom att teckentavlan pratar och roboten förstår och kan svara själv, så uppmuntras barnet att använda språket och lär sig att samarbeta för att nå ett gemensamt mål, vilket i detta fall är att få roboten att rita figurer.

Movies

• Two (normal developed) children starting to play with TRIK: [youtube or download quicktime, 5.6MB]
• The same session, 10 minutes later: [youtube or download quicktime, 10.3MB]

Publications

• The original project description, in Swedish: [TRIK-beskrivning.pdf] 
• Translated to English, presented as a short paper at Nodalida 2009: [TRIK-description.pdf]
• Poster in English: [TRIK-poster.pdf]

Presentations

The robot has been presented at the following workshops and conferences:

• NordiCHI'08 Robot Workshop, 17 oktober 2008, Lund: [www]
• Nodalida'09, 17th Nordic Conference on Computational Linguistics, 14–16 maj 2009, Odense, Danmark: [www]
• 8:e Västsvenska Kommunikationskarnevalen, 1–2 juni 2009, Göteborg: [www]
• DiaHolmia'09, 13th Workshop on the Semantics and Pragmatics of Dialogue, 24–26 juni 2009, Stockholm: [www]
• ID-dagarna 2009, 7–9 oktober 2009, Stockholm: [www]

Press

TRIK has been mentioned in the press too:

• Vårt Göteborg, 1 juni 2009: [www]
• Metro Teknik, 19 november 2009: [www]

Downloads

Building instructions for the TRIK robot, together with some unsorted Python utilities for connecting to and drawing with the robot from a Mac computer:

• [generic-turtle-controller.zip] — note that there is no program that can be run directly in this archive, you have to make a wrapper program yourself. For this you need to know at least some Python programming. The building instructions can be viewed in Lego Digital Designer.

Attachments: 

generic-turtle-controller.zip

TRIK-beskrivning.pdf

TRIK-description.pdf

TRIK-poster.pdf