​Laughing and Crying According to People's Emotions... Making "AI Human."

The company demonstrated R2MIX, a digital human service platform developed by Metabuild, at a software research center in Seocho District, Seoul on Wednesday. R2MIX enables users to create virtual human beings in three dimensions by synchronizing human facial expressions, mouth shapes, and gestures in real time. In the platform, users can create their own digital human with simple settings and use them in video production, live streaming, and metaverse spaces. When they smiled broadly and waved their hands in front of the camera, the digital human in the screen imitated the same movements without delay. It was like an avatar.

Another feature of R2MIX is that it has technology to detect human emotions through conversation.

Metabuild developed its own Large Language Model (LLM) last year. LLaMON is an LLM specialized in counseling, summary, and emotion analysis. The company explained that it has implemented AI human technology that reads people's emotions by fusing R2MIX. When the AI emotion analysis solution was activated among the R2MIX functions, another digital human emerged. In the chat room, it wrote, "It was cold last week, but it feels good to have a little warmer day." Then, the digital human laughed and answered, "I feel good, too. Drink warm drinks and take care of your health as the temperature difference can be big." When he wrote in the chat room, "I had an argument with my co-worker yesterday while I was working," the digital human said with a sad face, "Arguing can happen when you're under stress. Take a deep breath, find your composure, and have a conversation with your co-worker." The MBTI measurement function was also interesting. They simply answered four questions that the digital human gave through voice, and they matched the actual MBTI.

"R2MIX applied with LLaMON technology enhances immersion because it detects people's emotions in conversations, presents answers that fit the situation, and even makes the right facial expressions," said Park Jae-hyung, head of Metabuild SW Research Division. "We pre-learned 3 billion corpus related to various emotions such as joy, fear, and sadness."

Metabuild is preparing for a variety of new services by learning specialized data based on the LLaMON language model. "We will expand the scope of AI content services to broadcasting production, sentence generation and summary, psychological counseling, composition and writing through the interface," said Park. To this end, Metabuild is cooperating with Chilloen, the developer of AI composition program "KEENEAT."

Established in 1998, Metabuild is Korea's No. 1 operator leading the integrated software platform market. It has commercialized data-linked integration and analysis products such as MESIM ESB and MESIM DataHUB, as well as smart city and transportation digital twins to serve more than 4,000 domestic and foreign clients in various fields such as finance, public, manufacturing, and distribution. Since 2020, it has been expanding its AI service business including digital human, accelerating its efforts to tap into overseas markets.​

Published: 05 Feb, 2024