logo
MIT Teaches Soft Robots Body Awareness Through AI And Vision

MIT Teaches Soft Robots Body Awareness Through AI And Vision

Forbes07-07-2025
MIT CSAIL researchers have developed a new system that teaches robots to understand their own ... More bodies, using only vision. Instead of relying on sensors, the system allows robots to learn how their bodies move and respond to commands just by watching themselves.
Researchers from the Massachussets Institute of Technology's (MIT) CSAIL lab have developed a new system that teaches robots to understand their bodies, using only vision. Using consumer-grade cameras, the robot watched itself move and then built an internal model of its geometry and controllability.
According the researchers this could dramatically expand what's possible in soft and bio-inspired robotics, enabling affordable, sensor-free machines that adapt to their environments in real time.
The team at MIT said that this system and research is a major step toward more adaptable, accessible robots that can operate in the wild with no GPS, simulations or sensors. The research was published in June in Nature.
Daniela Rus, MIT CSAIL Director said with Neural Jacobian Fields, CSAIL's soft robotic hands were able to learn to grasp objects entirely through visual observation with no sensors, no prior model and no manual programming.
'By watching its own movements through a camera and performing random actions, the robot built an internal model of how its body responds to motor commands. Neural Jacobian Fields mapped these visual inputs to a dense visuomotor Jacobian field, enabling the robot to control its motion in real time based solely on what it sees,' added Rus.
Rus adds that the reframing of control has major implications.
"Traditional methods require detailed models or embedded sensors but Neural Jacobian Fields lifts those constraints, enabling control of unconventional, deformable, or sensor-less robots in real time, using only a single monocular camera.'Vincent Sitzmann, Assistant Professor at MIT's Department of Electrical Engineering and Computer Science and CSAIL Principal Investigator said the researchers relied on techniques from computer vision and machine learning. The neural network observes a single image and learns to reconstruct a 3D model of the robot which relies on a technique called differentiable rendering which allows machine learning algorithms to learn to reconstruct 3D scenes from only 2D images.
'We use motion tracking algorithms - point tracking and optical flow - to track the motion of the robot during training,' said Sitzmann. "By relating the motion of the robot to the commands that we instructed it with, we reconstruct our proposed Neural Jacobian Field, which endows the 3D model of the robot with an understanding of how each 3D point would move under a particular robot action.'
Sitzmann says this represents a shift towards robots possessing a form of bodily self-awareness and away from pre-programmed 3D models and precision-engineered hardware. 'This moves us towards more generalist sensors, such as vision, combined with artificial intelligence that allows the robot to learn a model of itself instead of a human expert,' said Sitzmann. "This also signals a new class of adaptable, machine-learning driven robots that can perceive and understand themselves.'
The researchers said that three different types of robots acquired awareness of their bodies and the actions they could take as a result of that understandi
A 3D-printed DIY toy robot arm with loose joints and no sensors learned to draw letters in the air with centimeter-level precision. It discovered which visual region corresponds to each actuation channel, mapping 'which joint moves when I command actuator X' just from seeing motion.
A soft pneumatic hand learned which air channel controls each finger, not by being told, but just by watching itself wiggle. They inferred depth and geometry from color video alone, reconstructing 3D shape before and after actions.
A soft, wrist-like robot platform, physically disturbed with added weight, learned to balance and follow complex trajectories. They quantified motion sensitivity, for example, measuring how a command that slightly changes an actuator produces millimeter‑level translations in the gripper.
Changing soft robotics
The CSAIL researchers aid that soft robots are hard to model because they deform in complex ways. One reasercher said in an email interview that the method they used in the research doesn't require any manual modeling. The robot watches itself move and figures out how its body behaves similar to a human learning to move their arm by watching themselves in a mirror.
Sitzmann says conventional robots are rigid, discrete joints connected by rigid linksbuilt to have low manufacturing tolerance. "Compare that to your own body, which is soft: first, of course, your skin and muscles are not perfectly solid but give in when you grasp something.'
'However, your joints also aren't perfectly rigid like those of a robot, they can similarly bend and give in, and while you can sense the approximate position of your joints, your highest-precision sensors are vision and touch, which is how you solve most manipulation tasks,' said Sitzmann. "Soft robots are inspired by these properties of living creatures to be similarly compliant, and must therefore necessarily also rely on different sensors than their rigid cousins.'
Sitzmann says that this kind of understanding could revolutionize industries like soft robotics, low‑cost manufacturing, home automation and agricultural robotics.
'Any sector that can profit from automation but does not require sub-millimeter accuracy can benefit from vision‑based calibration and control, dramatically lowering cost and complexity,' said Sitzmann. "In the future, with inclusion of tactile sensing (=touch), this paradigm may even extend to applications that require high accuracy.'
A new approach to soft robotics
Researchers say their approach removes the need for experts to build an accurate model of the robot, a process that can take months. It also eliminates reliance on expensive sensor systems or manual calibration. The simplified process entails recording the robot moving randomly and the model learns everything it needs to know from that video.
'Instead of painstakingly measuring every joint parameter or embedding sensors in every motor, our system heavily relies on a camera to control the robot," said Sitzmann. 'In the future, for applications where sub-millimeter accuracy is not critical, we will see that conventional robots with all their embedded sensors will increasingly be replaced by mass-producible, affordable robots that rely on sensors more similar to our own: vision and touch."
Orange background

Try Our AI Features

Explore what Daily8 AI can do for you:

Comments

No comments yet...

Related Articles

This Bezos-backed EV startup is betting you'll pay extra for a stereo in your petite pickup
This Bezos-backed EV startup is betting you'll pay extra for a stereo in your petite pickup

Yahoo

time21 minutes ago

  • Yahoo

This Bezos-backed EV startup is betting you'll pay extra for a stereo in your petite pickup

By Kalea Hall and Nora Eckert (Reuters) -When Will Haseltine saw images online of a small, boxy electric pickup from startup Slate Auto this past spring, he got on the waitlist right away. The sparse interior and crank windows reminded him of the no-frills pickups he grew up around in Memphis, Tennessee – but he was most enamored with the sub-$20,000 price tag. That price, though, factored in a $7,500 federal tax break, which is set to expire Sept. 30, a casualty of the budget package U.S. President Donald Trump signed into law earlier this month. Now Haseltine isn't sure the truck will fit his budget when it comes out, expected late next year. 'The Slate was the first time that I looked at a car, wanted it, and could also really make it happen," said Haseltine, a 39-year-old musical instrument technician. Without the tax credit, he said: "That's just plain too much." Michigan-based Slate has raised $700 million from investors, including founder Jeff Bezos, and has racked up more than 100,000 reservations for its cars. But the company is launching into a tough U.S. market. A few years ago, the electric-vehicle space was awash in hopeful entrepreneurs looking to cash in on the global transition to electric cars. But U.S. EV sales growth has cooled as consumer interest has faded. The loss of federal tax breaks will further hurt demand, auto executives and analysts predict. Like other EV startups, Slate likely faces a long road to profitability. The EV business has proven to be a money loser for most industry players, partly because batteries remain relatively expensive. Even in China, where smaller, inexpensive EVs have proliferated and companies enjoy a cost advantage over Western automakers, most are unprofitable. Slate founders believe the company can overcome those obstacles by offering something that is in short supply in today's U.S. car market: affordability. The average new-vehicle selling price is above $45,000. 'We are building the affordable vehicle that has long been promised but never delivered,' Slate CEO Chris Barman said at a Detroit conference in July. The company has a chance to fill a void left by Tesla, which has backtracked on plans to introduce a mid-$20,000s electric vehicle. The startup has taken a bare-bones approach to its two-seat pickup, which is slightly smaller than a Honda Civic hatchback. How bare-bones? A stereo and power windows will cost extra. Slate hasn't disclosed the cost of such add-ons. 'IT'S A COOL IDEA' Slate's creation started with an idea from Miles Arnone, the CEO of Re:Build Manufacturing, a Massachusetts-based startup that includes several former Amazon employees. Arnone believed workers needed better access to affordable vehicles. Arnone shared his idea with Jeff Wilke, the company's chairman and a former Amazon executive, and eventually, a small team was formed. The group hired Barman, who spent most of her career as an engineering executive at Fiat Chrysler, now part of Stellantis. Barman told Reuters recently that Slate will be able to absorb the loss of the $7,500 tax credit because the truck's price still will undercut competitors. The company plans to build the pickup at an old catalog factory in Warsaw, Indiana. Executives are taking steps to hold down costs, starting with a simplified design that uses about 500 parts in the truck's assembly, compared with a few thousand for a traditional truck. The plan to build all of its trucks in a basic package – what the company calls a 'SKU of one' – allows customers to choose to add a stereo, center console, special lighting, and other features later. The pickup will be built with composite body panels in gray, with an option for a vinyl wrap. That will sidestep the need for a paint shop, which is one of the most expensive investments in a typical car factory. Slate's minimalist approach is a leap of faith that Americans will forgo creature comforts they have been increasingly willing to splurge on. Last year, U.S. buyers spent 33% above the base price on average, springing for higher-end trim packages and extra features, according to . That was up from 28% in 2014. But there is mounting evidence that new cars are becoming out of reach for many Americans. That could worsen under the effects of the Trump administration's tariffs, which threaten to increase prices on popular budget cars imported from Mexico, Korea and elsewhere. From that standpoint, Slate's price-conscious pickup might be hitting at the right time, said Paul Waatti, director of industry analysis at AutoPacific. 'There's a growing appetite, especially among younger drivers, for vehicles that are more honest, more modular and less over-engineered,' he said. 'Slate taps right into that.' Traditional automakers and startups have found mixed success rolling out larger electric pickup trucks in recent years. Now, startups like Slate and California-based Telo are focusing on smaller electric pickups. In a town hall meeting in early May, Ford CEO Jim Farley and Executive Chair Bill Ford told employees they admired the company's customer-centered ethos and focus on affordability. Tim Kuniskis, Stellantis' head of American brands, called Slate 'super interesting' at a June event, while also questioning how affordable it would be for some shoppers once they added all the options they wanted. 'The idea behind it, we've talked about that idea a million times,' he said. "It's a cool idea.' Connectez-vous pour accéder à votre portefeuille

Interactive Brokers considers launching new stablecoin for customers
Interactive Brokers considers launching new stablecoin for customers

Yahoo

time21 minutes ago

  • Yahoo

Interactive Brokers considers launching new stablecoin for customers

By Anirban Sen NEW YORK (Reuters) -Interactive Brokers Group is considering launching a stablecoin for customers, joining a number of large financial firms that are betting big on the digital token boom as the U.S. eases regulations around the crypto industry. The deliberations come at a time when the underlying infrastructure of global financial markets is undergoing a once-in-a-generation transformation due to the proliferation of blockchain-based assets like stablecoins. In an interview with Reuters, Interactive Brokers' billionaire founder Thomas Peterffy said the company is working on potentially issuing stablecoins, but has yet to make a final decision on how that will be offered to customers. Interactive Brokers, which is one of the world's leading discount brokers with a market value of about $110 billion, currently has a partnership with crypto platform Paxos and is also an investor in crypto exchange Zero Hash. Through these tie-ups, Interactive Brokers offers trading in various cryptocurrencies to customers. The popular trading platform is now working on enabling instant, 24/7 stablecoin funding for brokerage accounts, as well as supporting asset transfers for commonly traded cryptocurrencies, said Peterffy, who also sounded a note of caution on the risks of rapid widespread adoption of crypto. "It's basically hard to grasp its fundamental value. If we see people adopting it and ascribing a value to it, I'm okay with that, but I'm still not convinced," said Peterffy. Among the options being considered, the Greenwich, Connecticut-based firm could allow customers to use stablecoins issued by other financial institutions to fund their accounts, depending upon the credibility of the issuer. Stablecoins are blockchain-based tokens acting as a proxy for an asset that allow people to move money across borders without interacting with the banking system. Critics have warned that this makes them useful for criminals who want to avoid banks' anti-money laundering checks. Online brokerage firm Robinhood recently launched a stablecoin pegged to the U.S. dollar called the Global Dollar Network through a consortium that included other crypto platforms like Kraken and Galaxy Digital. The Global Dollar Network is based around a stablecoin called USDG, which is issued by Paxos. Interactive Brokers, which last year launched a predictions market called ForecastEx that allows investors to buy "yes" or "no" contracts in response to questions, is one of the largest U.S. trading platforms with about 3.87 million customer accounts at the end of June, a jump of 32% from last year. Like other brokerages, Interactive Brokers has benefited from elevated levels of trading due to market volatility this year that was triggered by U.S. tariff policy. Its shares have surged about 47% since the start of the year, outperforming the S&P 500 Investment Banking & Brokerage index, which has risen roughly 20% during the same period. "We view the two innovations (prediction markets and crypto investments) as an effective hedge to any disruption in the firm's core equity, futures, and derivatives businesses," Morningstar analysts said in a note dated July 18. Sign in to access your portfolio

Better Dividend Stock: Alphabet vs. AT&T
Better Dividend Stock: Alphabet vs. AT&T

Yahoo

time21 minutes ago

  • Yahoo

Better Dividend Stock: Alphabet vs. AT&T

Key Points Income-seeking investors have a difficult choice between dividend stocks that grow their payouts quickly and those that offer high yields. Alphabet shares offer a minuscule yield at recent prices, but earnings are growing by leaps and bounds. AT&T offers a high yield, but it has been several years since shareholders last saw its quarterly payout grow. 10 stocks we like better than Alphabet › Investors looking to grow their passive income stream with dividend stocks have two basic options. Dividend payers that raise their payouts rapidly tend to offer low yields up front, while higher-yielding stocks tend to increase their payouts slowly, if at all. Right now, Alphabet (NASDAQ: GOOG)(NASDAQ: GOOGL) and AT&T (NYSE: T) represent opposite ends of the dividend investors' dilemma. Profit that could be used to boost dividend payments are surging for the parent company behind Google and YouTube, but it offers a very low yield. AT&T offers a high yield, but earnings have been rising at a snail's pace. Let's take a closer look at both to see which could be a better fit for your portfolio. 1. Alphabet Last year, Alphabet began a dividend program, and it's already raised the payout once. This April, the company bumped its payout up by 5% to $0.21 per share. While significant, the raise was much smaller than it could probably afford. With a yield of about 0.4% at recent prices, Alphabet is not on many dividend investors' radar. If you have a long time horizon, though, adding this stock to your portfolio could lead to heaps of income generation once you're ready to retire. When viewed on a long-term timeline, earnings per share and dividend payments tend to rise in line with each other. Over the past five years, Alphabet raised earnings per share by a whopping 29.4% annually. The conglomerate's advertising and cloud computing businesses generated around $66.7 billion in free cash flow over the past 12 months. Dividends over the same time frame worked out to less than 15% of the free cash flow available to make the payments. Alphabet was a bit slow to release generative artificial intelligence tools, but strong advantages mean it's likely to own the most popular AI-driven search tools for years to come. Despite Microsoft promoting its Edge browser on everyone's PC, Alphabet's Chrome browser had a global market share of 68% in June, according to Statcounter. Google Search and the Chrome browser are defaults on Alphabet's Android operating system, which runs on 74% of the world's smartphones. With Android and Chrome giving Alphabet access to billions of users and all the data their browsing activity generates, its lucrative advertising business has a good chance to stay on top for the long run. 2. AT&T While profit at Alphabet has been surging, the telecom giant, AT&T, has been lumbering along. Earnings per share over the trailing 12 months are only 15.8% higher than they were five years ago. The spinoff of its media assets a couple of years ago is partly responsible for the lack of growth, and so is the ongoing loss of wireline phone connections. These issues are temporary, but the company's position in America's mobile internet oligopoly is likely to remain intact for many years to come. This gives it a chance to continue growing through sales of its wireless internet and mobility services. AT&T reduced its dividend in 2022, and it hasn't budged since. At recent prices, the stock offers a 4% yield, which is roughly 10 times more than you'd receive from Alphabet. AT&T doesn't have Alphabet's immense cash flow, but it's more than enough to meet its dividend obligation. Free cash flow that reached $19.6 billion over the past 12 months was more than twice what the company needed to meet its dividend obligation. Management expects free cash flow to subside slightly to a little over $16 billion this year, which is still plenty more than necessary to maintain the payout. Business wireline revenue is expected to decline again this year, but AT&T has growth engines pushing in the opposite direction. This year, mobility revenue is expected to grow by 3% or better, plus its consumer broadband products are surging. Second-quarter consumer fiber broadband revenue soared by 18.9% year over year to $2.1 billion and is expected to continue at a similar pace for the rest of 2025. Don't let the proliferation of mobile virtual network operators such as Cricket and Mint confuse you into thinking there's a lot of competition for mobile internet subscribers. After T-Mobile acquired Sprint in 2020, Americans effectively have just three nationwide mobile internet providers to choose from. By revenue, AT&T is firmly in second place. Given the enormous investment required to reach its present position, we aren't likely to see any more competitors able to provide nationwide mobile internet services unless Congress forces one of the big three to break up. Dividends at AT&T probably won't grow quickly, but the company could begin announcing annual payout bumps soon. Earlier this year, it reduced debt to about 2.5 times trailing 12-month EBITDA and initiated a $10 billion share repurchase program. Which is the better dividend stock to buy now? I don't intend to begin leaning on the passive income my dividend stocks generate for at least 20 years. With this in mind, I'm a lot more excited to buy Alphabet. If we project these two companies' rates of earnings growth over the past five years forward, the yield on cost investors receive from Alphabet shares purchased now could surpass the amount they receive from AT&T by 2035. If I were much closer to retirement, I might consider AT&T the better buy. With time to let the payout grow, though, I'd be better off over the long run with shares of Alphabet. Should you invest $1,000 in Alphabet right now? Before you buy stock in Alphabet, consider this: The Motley Fool Stock Advisor analyst team just identified what they believe are the for investors to buy now… and Alphabet wasn't one of them. The 10 stocks that made the cut could produce monster returns in the coming years. Consider when Netflix made this list on December 17, 2004... if you invested $1,000 at the time of our recommendation, you'd have $636,628!* Or when Nvidia made this list on April 15, 2005... if you invested $1,000 at the time of our recommendation, you'd have $1,063,471!* Now, it's worth noting Stock Advisor's total average return is 1,041% — a market-crushing outperformance compared to 183% for the S&P 500. Don't miss out on the latest top 10 list, available when you join Stock Advisor. See the 10 stocks » *Stock Advisor returns as of July 21, 2025 Cory Renauer has no position in any of the stocks mentioned. The Motley Fool has positions in and recommends Alphabet and Microsoft. The Motley Fool recommends T-Mobile US and recommends the following options: long January 2026 $395 calls on Microsoft and short January 2026 $405 calls on Microsoft. The Motley Fool has a disclosure policy. Better Dividend Stock: Alphabet vs. AT&T was originally published by The Motley Fool Sign in to access your portfolio

DOWNLOAD THE APP

Get Started Now: Download the App

Ready to dive into a world of global content with local flavor? Download Daily8 app today from your preferred app store and start exploring.
app-storeplay-store