Spoken dialog systems for language learning and assessment

alt text 

I am an active developer of HALEF, a telephony-based open-source spoken dialog system that can be used with different plug-and-play back-end modules. HALEF stands for Help Assistant – Language-Enabled and Free. The system is compliant with World Wide Web Consortium and related industry standards while maintaining an open codebase to encourage progressive development and a common standard testbed for spoken dialog system development and benchmarking. The system can be deployed toward a versatile range of potential applications, including intelligent tutoring, language learning and assessment. HALEF is written primarily in Java and leverages a number of open-source tools. The figure on the left schematically depicts the main components of the HALEF system. Note that unlike a typical SDS, which consists of sequentially-connected modules for speech recognition, language understanding, dialog management, language generation and speech synthesis, in HALEF some of these are grouped together forming independent blocks which are hosted on different virtual machines in a distributed architecture.

For more details, please check out the codebase at the SourceForge repository, or read the following papers:

  • Vikram Ramanarayanan, David Suendermann-Oeft, Patrick Lange, Robert Mundkowsky, Alexei V. Ivanov, Zhou Yu, Yao Qian and Keelan Evanini (2017), Assembling the jigsaw: How multiple open standards are synergistically combined in the HALEF multimodal dialog system, in: Multimodal Interaction with W3C Standards: Towards Natural User Interfaces to Everything, D. A. Dahl, Ed., ed New York: Springer, 2017. [link]

  • Vikram Ramanarayanan, David Suendermann-Oeft, Patrick Lange, Alexei V. Ivanov, Keelan Evanini, Zhou Yu, Eugene Tsuprun, and Yao Qian (2016), Bootstrapping Development of a Cloud-Based Spoken Dialog System in the Educational Domain From Scratch Using Crowdsourced Data, in: ETS Research Report Series, Wiley. doi: 10.1002/ets2.12105. [link]

  • David Suendermann-Oeft, Vikram Ramanarayanan, Moritz Teckenbrock, Felix Neutatz and Dennis Schmidt (2015), HALEF: an open-source standard-compliant telephony-based spoken dialog system – a review and an outlook, in proceedings of: Natural Language Dialog Systems and Intelligent Assistants, Springer. [link]

  • Vikram Ramanarayanan, David Suendermann-Oeft, Alexei V. Ivanov, and Keelan Evanini (2015). A distributed cloud-based dialog system for conversational application development, in proceedings of: 16th Annual SIGdial Meeting on Discourse and Dialogue (SIGDIAL 2015), Prague, Czech Republic [pdf].

  • Alexei V. Ivanov, Vikram Ramanarayanan, David Suendermann-Oeft, Melissa Lopez, Keelan Evanini, and Jidong Tao (2015). Automated speech recognition technology for dialogue interaction with non-native interlocutors, in proceedings of: 16th Annual SIGdial Meeting on Discourse and Dialogue (SIGDIAL 2015), Prague, Czech Republic [pdf].