Goal

To develop a voice interface to an Android smartphone application.

Background

Voice interfaces give users the possibility to interact with a device without using their eyes or hands. This can be particularily useful in situations where interaction with a device would otherwize interfere with other tasks. A typical example is driving, in which the simultaneous use of a screen-based device such a smartphone can constitute a direct danger. This project focuses in the potential benefits of voice enabled smartphone apps in driving and similar scenarios.

Problem description

The problem consists of developing a spoken dialog interface to an Android application, e.g. GPS-based navigator or Facebook. The user should be able to switch freely between touch-screen and voice interaction. To facilitate the development of this multimodal interface, Talkamatic Dialog Manager (TDM) will be used.

The problem mainly consists of the following tasks:

• Write a TDM device resource, which communicates with the app functionality (Python and Android SDK)
• Write a TDM domain and ontology specification, describing the structure of the app (Python)
• Write a TDM grammar, describing the verbal phrases used by the user and the app (Grammatical Framework)

Recommended:

• Python
• Grammar formalisms

Experience of Android app development is valuable but not required.

Supervisors

Peter Ljunglöf, Chalmers Data- och informationsteknik or Staffan Larsson, FLoV, together with Talkamatic AB. Talkamatic is a university research spin-off company based in Göteborg.