Posted on Nov 23, 2017 | Rating

LipSync Generator

LipSync Generator generates lip-synchronized speech animation for the game characters given text as input.

Short non-technical description: LipSync Generator is implemented as a Unity 3D plug-in. The component works together with a Text-to-Speech (TTS) system. Since Unity 3D is multi-platform, the component can be deployed on different platforms including Web and mobile applications.

Technical description: The component takes as input text, sends it to the text-to-speech (TTS) system, parses the phonemes, maps them to visemes (visual counterparts of phonemes) and finally blends them for smooth speech animation. It currently uses an off-the-shelf text-to-speech (CereVoice) library. It is also possible to link it with other TTS systems. The component provides a state-of-the-art co-articulation algorithm to generate natural lip-sync animation for game characters.

Language: English


Game development environment: Unity

Target platform: Windows

Programming language: C#

Version: 0.1

Version notes: Initial version

Development status: Completed

Commit URL:

Type: Apache 2.0 (Apache License 2.0)


Interpersonal skills Communication
Computers in other domains Computer games
Component Avatar Systems

Related Articles

BML Realizer
Utrecht University Virtual Human Controller, Rage project, Utrecht University

Dialogue Scenario Editor
Utrecht University, Rage project, Johan Jeuring

Dialogue Scenario Reasoner
Utrecht University, Rage project, Johan Jeuring

Performance Statistics
Open University of the Netherlands, Rage project, Rage project

Client Side Game Storage Asset
Rage project, Rage project, Open University of the Netherlands

Player-centric rule-and-pattern-based adaptation asset
Dessislava Vassileva, Rage project, Sofia University

Real-Time Arousal Detection Using Galvanic Skin Response
Dessislava Vassileva, Boyan Bontchev, Rage project, Sofia University

RAGE Tutorial Demo
Rage project, Rage project, Dominic Heutelbeck

Competence Component v2
Matthias Maurer, Alexander Nussbaumer, Christina Steiner, Rage project, Graz University of Technology

Virtual Labs Authoring tool
ENVISAGE project, Rage project, Dimitrios Ververidis