Microsoft has recently unveiled its new VASA-1 AI model, designed to generate lifelike talking faces for virtual characters with appealing visual affective skills. The VASA-1 framework can create short videos with realistic facial and head dynamics, synchronizing lip movements with audio and capturing a wide spectrum of facial nuances and natural head motions.
According to Microsoft, the VASA-1 model can generate up to one-minute videos using a single static image and a speech audio clip. The model also gives users granular control to adjust different aspects of the video, such as main eye gaze direction, head distance, and emotion offsets. The controls over disentangled appearance, 3D head pose, and facial dynamics enable users to modify the output in a better way.
The VASA-1 AI model supports the online generation of 512 x 512 videos at up to 40fps with negligible starting latency. The model exhibits the capability to handle photo and audio inputs that are out of the training distribution, such as artistic photos, singing audios, and non-English speech. These types of data were not present in the training set.
Microsoft has emphasized its intent to utilize the VASA-1 technology for creating realistic virtual characters rather than releasing it as a standalone product or API. The company has announced that VASA-1 will not be available to the public, and there are no plans to release an online demo, API, or additional implementation details related to VASA. This decision stems from Microsoft’s commitment to ethical AI practices and its opposition to any use of this technology to mislead or deceive.
Addressing concerns about potential misuse, Microsoft clarified its stance that their research focuses on positive applications of generating visual affective skills for virtual AI avatars. While their method could potentially be misused for impersonation, they are dedicated to advancing forgery detection techniques to mitigate such risks.
In conclusion, Microsoft’s VASA-1 AI model is a breakthrough in generative AI that can create lifelike talking faces for virtual characters with appealing visual affective skills. While the technology is not available to the public, it holds great potential for creating realistic virtual characters for various applications.
Resources: https://www.microsoft.com/en-us/research/project/vasa-1/