Latest news with #LegoGPT
Hypebeast
4 days ago
- Science
- Hypebeast
You Can Now Turn Text Prompts Into Lego Designs With LegoGPT AI
Summary Researchers at Carnegie Mellon University (CMU) have unveiled a groundbreaking newAImodel called LegoGPT, designed to bridge the gap between natural language and physical creation. This innovative system can transform text prompts into fully buildable LEGO models, specifically focusing on vehicles. Imagine simply typing 'a red sports car with big wheels' or 'a blue spaceship with pointy wings,' and having the AI instantly generate instructions for assembling it with LEGO bricks. That's the promise of LegoGPT. The system operates by understanding natural language and processing and interpret the descriptive elements of a text prompt. It then translates the words into LEGO geometry, turning concepts into a 3D LEGO model, taking into account the unique constraints and connectivity of LEGO bricks. It provides step-by-step instructions that a human builder can follow to construct the physical model. According todezeen, the team, which is led by Ava Pun, a PhD student at Carnegie Mellon University (CMU) and one of the study's coauthors, hope to expand the AI model beyond LEGOs to give it real-world applications to help 'architects design structurally sound buildings or designers create custom furniture.' Pun shared with the publication, 'We wanted to bring user imagination into the physical world. Imagine if you could type in a prompt or show us a picture of a chair, and boom – we could actually make that product and ship it to you in a week or two.' This development is a significant leap in AI's ability to translate abstract ideas into tangible forms. It showcases advancements in generative AI, computational design, and robotics, demonstrating how AI can aid in creative design and engineering processes. While currently focused on vehicles, the potential applications for LEGO-GPT are vast, from educational tools to rapid prototyping and even assisting LEGO designers themselves. It's a playful yet powerful example of AI moving beyond virtual spaces into the realm of physical construction.
Fast Company
19-05-2025
- Science
- Fast Company
‘LegoGPT' designs Lego models with nothing but a prompt
Researchers at Carnegie Mellon University (CMU) have just answered a question that's probably occurred to Lego fans for decades: What if I could instantly turn any idea into a Lego set? In a paper titled 'Generating Physically Stable and Buildable LEGO Designs from Text,' published last week, six coauthors lay out an invention they're calling 'LegoGPT.' This generative AI model can take a text-based prompt, like 'an acoustic guitar with an hourglass shape,' and determine all of the necessary Lego pieces needed to build that structure and how to assemble them. The LegoGPT demo and code is publicly available through the study, meaning that Lego hobbyists are free to try it out at home. Although outputs are currently limited to around 20 categories (including basic items like chairs, guitars, boats, trains, and cars), the researchers are working to expand the model's capabilities into more complicated categories. Ultimately, they think a LegoGPT-type tool might serve as the basis for a variety of real-world tasks in architecture and product design. How LegoGPT predicts its next block LegoGPT is a fine-tuned version of Meta's LLaMA-3.2-Instruct-1B language learning model, which you can think of as an open source ChatGPT. To teach the model how to make Lego structures, researchers trained it using a database of 47,000 Lego structures and 28,000 unique 3D shapes, each with their own descriptive captions. Based on that vast swath of designs, LegoGPT is able to predict how to build a hypothetical object using only a text prompt. To do that, LegoGPT uses something called an autoregressive model, which is common among the most popular generative AI platforms. 'ChatGPT and Llama are autoregressive models because, given the string of words that they've already outputted, they want to predict the next word,' explains Ava Pun, one of the study's coauthors and a PhD student at CMU. 'So if you ask, 'What is the weather,' and it predicts 'The weather today is,' then it will try to predict the next word: sunny, rainy, and so on. With Lego GPT, instead of predicting the next word, it wants to predict the next brick.' Once LegoGPT has created a 3D model it thinks will work, the LLM needs a way to make sure that the structure will actually be stable. According to Pun, that proved tricky, considering that existing simulators aren't trained to understand the physics of a Lego brick. So, the CMU team built their own physics algorithm for LegoGPT to check its work. 'We developed a customized physics reasoning algorithm that accounts for all the physical forces that the bricks experience: for example, the downward force due to gravity, friction forces, and contact forces from the other bricks that they're touching,' Pun says. 'The algorithm constructs a force model for the structure and then evaluates the forces over the entire structure. If these physical forces sum to zero, that means the structure will not move around.' LegoGPT automatically uses this algorithm to ensure that it's found a viable solution. If any of the block it's chosen is causing the model to turn out wobbly, the model will continue iterating until it lands on a new version that passes the test. A future real-world application So far, researchers have used LegoGPT to create a range of structures, including vintage cars, steamships, and an electric guitar. Currently, the model only works on a 20x20x20 voxel grid, though Pun says the team is already planning on adding more brick types to the model's database and expanding the grid resolution. For Lego fans who want to play around at home, the study's demo, available through a public portal, can turn simple prompts into a buildable 3D Lego model and a list of necessary parts. Because LegoGPT isn't made to be Lego-builder-facing, it doesn't produce step-by-step instructions, meaning the main challenge will be figuring out how to arrange the component parts in the right order. Pun says her team used Lego brick assembly to test AI's 3D-building capabilities because of the blocks' accessibility. Eventually, though, they believe this concept could be applied to real-world scenarios, like helping architects draft buildings or designing custom furniture from a predefined set of parts. 'Today's generative AIs can't offer that—you can generate a cool image or video of a chair, but the model doesn't know how these things can be made in the real world,' Pun says. 'We wanted to address this challenge by integrating physical laws and assembly constraints into generative models and creating objects that function in reality.'
Mint
14-05-2025
- Science
- Mint
Meet LegoGPT that thinks in bricks: AI that builds real-world lego models from text prompts
A new artificial intelligence model namedLegoGPT has been introduced by researchers at Carnegie Mellon University, offering a fascinating look into the potential of generative AI in physical design. The system is capable of producing three-dimensional Lego structures based on written descriptions—ensuring they are not only visually accurate but also structurally sound. Built upon a fine-tuned version of theLLaMA-3.2-Instruct model with one billion parameters, LegoGPT is designed to interpret natural language prompts and produce corresponding Lego builds that obey the laws of physics. To validate structural stability, the model incorporates a mathematical optimisation tool calledGurobi, which evaluates each design's resilience and balance. The AI is trained on a purpose-built dataset calledStableText2Lego, which includes over 47,000 Lego constructions and more than 28,000 distinct 3D models. Each entry in the dataset is accompanied by detailed captions, design code, and 3D model files to enrich training accuracy. LegoGPT has been released as an open-source project under the permissive MIT licence and is now available for public use via GitHub. Users can prompt the system with imaginative requests such as a 'streamline elongated vessel' or a 'backless bench with armrest,' and receive ready-to-build designs that are both aesthetically pleasing and physically viable. To ensure the model's outputs could withstand real-world construction, researchers tested the AI-generated designs using a dual robotic assembly system. The robots were instructed to physically construct the designs, evaluating their stability post-build. Human participants also attempted assembly, helping the team assess performance with less precise manipulation. Impressively, the study reports that 99.8 per cent of the generated structures passed the stability benchmark. This breakthrough not only highlights the intersection of generative AI and engineering but also paves the way for future research in AI-assisted construction and design. By making both the model and dataset publicly accessible, the team hopes to foster innovation in physics-aware generative modelling and inspire new applications across education, design, and robotics.
The Star
14-05-2025
- Science
- The Star
This AI can help you build anything with Lego
A team of artificial intelligence researchers at Carnegie Mellon University, US, recently unveiled LegoGPT, a system capable of generating construction models using the famous little bricks, from simple text descriptions. AI helps with the design, while robotic arms build them. LegoGPT is the result of a collaboration between experts in machine learning and robotics. Their aim was to create a tool for transforming text-based instructions into buildable and stable Lego models, for hobbyists and professionals alike. The system is based on a language model trained on a vast dataset of over 47,000 Lego structures, combined with highly detailed descriptions. When a query is entered, LegoGPT predicts the addition of each brick in sequence, ensuring that the structure remains physically stable at each stage. The aim is to offer a multipurpose tool, useful in both education and design, enabling a project to be designed from a simple text query. Although LegoGPT is still in its infancy, the researchers have made their code and database available, enabling any interested person or entity to explore and use their tool. Details and resources are available on the project's official website. LegoGPT is part of a wider trend of AI-powered tools for designing Lego constructions. Other projects already well advanced include BrickCenter, a free online platform that uses AI to generate custom Lego models from text descriptions, and Brickit, a mobile application that scans available Lego bricks to suggest feasible constructions. – AFP Relaxnews
Yahoo
09-05-2025
- Science
- Yahoo
LegoGPT Eliminates AI Weirdness, Creates Brick Designs You Can Actually Build
PCMag editors select and review products independently. If you buy through affiliate links, we may earn commissions, which help support our testing. A group of computer scientists at Carnegie Mellon University is applying generative AI to Legos with "LegoGPT," which can produce a Lego block design based on the user's text input. "Our experiments show that LegoGPT produces stable, diverse, and aesthetically pleasing Lego designs that align closely with the input text prompts,' they wrote in a paper published this week. However, this isn't your average AI image generator, which spit out any image you request. The difference is that LegoGPT creates "stable" 3D Lego designs that respect the laws of physics. The paper notes that most AI-powered 3D object generation doesn't translate into the real world because the "objects may be difficult to assemble or fabricate using standard components" or "the resulting structure may be physically unstable even if assembly is possible." Translation: AI generates some truly strange stuff that you couldn't easily turn into a Lego design. Researchers approached the problem by training the AI model on 'physically stable Lego designs paired with captions." This led the team to create a virtual collection, dubbed the StableText2Lego dataset, with 47,000+ Lego structures. Each structure was also run through a program to calculate and assign a "stability score." In addition, LegoGPT can construct physically stable designs by checking for errors during the generation process. "If the resulting design is roll back the design to the state before the first unstable brick was generated," the researchers wrote. "We repeat this process iteratively until we reach a stable continue generation from the partial structure." According to the paper, LegoGPT generated a physically stable design over 98% of the time, outperforming other AI approaches. The program was also smart enough to create valid Lego designs free of errors that adhered to the text prompt 100% of the time. In addition, the researchers say the designs from LegoGPT 'can be assembled manually by humans and automatically by robotic arms' in the real world. The researchers uploaded LegoGPT to GitHub, so anyone can download the program and try it out. However, it's currently limited to '20 × 20 × 20 grid' 3D Lego creations. LegoGPT also appears to be focused on simpler designs, rather than the pricey sets the company sells styled after hit properties like Star Wars and Harry Potter. Still, the paper adds: 'Our method currently supports a fixed set of commonly used Lego bricks. In future work, we plan to expand the brick library to include a broader range of dimensions and brick types, such as slopes and tiles, allowing for more diverse and intricate Lego designs.'
