3DALL-E: Integrating Text-to-Image AI in 3D Design Workflows
AI-generated Key Points
- Text-to-image AI systems are well established and increasingly used in 3D design workflows
- Researchers developed 3DALL-E, a plugin that integrates DALL-E, GPT-3, and CLIP within CAD software to allow users to construct text and image prompts based on what they are modelling
- A study with 13 designers found that designers saw potential to incorporate 3DALL-E into their workflows for reference images, renders, materials and design considerations
- The team identified different patterns of prompting that can identify endpoints at which text-to-image can help
- Many prompts fall within a range of two to six concepts
- Prompt bibliographies were proposed as a way to clarify ownership and intellectual property concerns while enriching the design histories that software tools provide
- Sharing prompt bibliographies with their outcomes (i.e., 3D models) can also help respect all the parties behind these AI systems
- Advancements in prompting may push text prompting as a type of interaction into creative tools even if creative workflows have traditionally not revolved around text
- Results demonstrate enthusiasm for text-to-image tools within 3D workflows and propose best practices for helping users bring their own work into AI assisted design workflows.
- The study highlights the implications of these workflows and how they can merge with existing generative design workflows.
Authors: Vivian Liu, Jo Vermeulen, George Fitzmaurice, Justin Matejka
Abstract: Text-to-image AI systems are capable of generating novel images for inspiration, but their applications for 3D design workflows and how designers can build 3D models using AI-provided inspiration is less understood. To investigate this, we integrated DALL-E, GPT-3, and CLIP within a CAD software in 3DALL-E, a plugin that allows users to construct text and image prompts based on what they are modelling. In a study with 13 designers, we found that designers saw great potential to incorporate 3DALL-E into their workflows and to use text-to-image AI for reference images, renders, materials, and design considerations. Additionally, we elaborate on prompting patterns and provide measures of prompt complexity observed across participants. We conclude on a discussion of how 3DALL-E can merge with existing generative design workflows and propose prompt bibliographies as a form of human-AI design history.
Ask questions about this paper to our AI assistant
You can also chat with multiple papers at once here.
Assess the quality of the AI-generated content by voting
Why do we need votes?
Votes are used to determine whether we need to re-run our summarizing tools. If the count reaches -10, our tools can be restarted.
Similar papers summarized with our AI tools
Navigate through even more similar papers through atree representation
Look for similar papers (in beta version)
By clicking on the button above, our algorithm will scan all papers in our database to find the closest based on the contents of the full papers and not just on metadata. Please note that it only works for papers that we have generated summaries for and you can rerun it from time to time to get a more accurate result while our database grows.
Disclaimer: The AI-based summarization tool and virtual assistant provided on this website may not always provide accurate and complete summaries or responses. We encourage you to carefully review and evaluate the generated content to ensure its quality and relevance to your needs.