Nvidia's New AI Model Can Turn Text and Images into 3D Assets

Edify 3D is a technology that uses physically based rendering to create detailed 3D geometry as well as high-resolution 4K textures. According to Nvidia, these assets may be used in production situations with little processing.

The researchers trained Edify 3D using non-public datasets of high-resolution images, pre-rendered multi-angle views, and 3D shape data. Multiple pre-processing procedures assured the quality and uniformity of training assets.

Nvidia's Edify 3D converts text and images to 3D assets

So, how does the system work? First, a diffusion model creates numerous similar images of the item from various angles. A customized reconstruction model then combines these views to create a comprehensive 3D model.

Edify 3D generates quad meshes, which are 3D models with clear structures and UV maps that 3D designers may change. The output seems more sophisticated than other 3D-creating tools currently accessible.

Read more: New YouTube’s AI Tool Can Help Create Video Backgrounds