Generative Metahuman avatars built in Unreal Engine and powered by Python and Blueprints, exploring how AI could be used to extend NPC interactions in gaming or beyond.

• Interactive conversational AI extending existing prototypes through Twitch / Live streaming integration or through real-time in person interactions through the microphone / speech-to-text
  • OpenAI, ElevenLabs
• Python based animation system to interface between Unreal Engine and Nvidia Audio2Face to create a dynamic and realtime animation system to power custom Metahumans.
  • Combining motion capture with AI generated animation to create believable and lifelike AI Avatars
  • Generating TTS audio via elevenlabs API
  • Or reading pre-generated audio from a script json for offline broadcasts
  • Sending audio via RPC from A2F to Unreal
  • Driving animations with OSC messaging from Python to Unreal
• Combining traditional AI behaviour trees with generative AI to create engaging interactions.
  •