Researchers use AI to identify similar materials in images - MIT News
🥇 Bonuses, Promotions, and the Best Online Casino Reviews you can trust: https://bit.ly/BigFunCasinoGame
Researchers use AI to identify similar materials in images - MIT News
A robot manipulating objects while, say, working in a kitchen, will benefit from understanding which items are composed of the same materials. With this knowledge, the robot would know to exert a similar amount of force whether it picks up a small pat of butter from a shadowy corner of the counter or an entire stick from inside the brightly lit fridge. Identifying objects in a scene that are composed of the same material, known as material selection, is an especially challenging problem for machines because a material’s appearance can vary drastically based on the shape of the object or lighting conditions. Scientists at MIT and Adobe Research have taken a step toward solving this challenge. They developed a technique that can identify all pixels in an image representing a given material, which is shown in a pixel selected by the user. The method is accurate even when objects have varying shapes and sizes, and the machine-learning model they developed isn’t tricked by shadows or lighting conditions that can make the same material appear different. Although they trained their model using only “synthetic” data, which are created by a computer that modifies 3D scenes to produce many varying images, the system works effectively on real indoor and outdoor scenes it has never seen before. The approach can also be used for videos; once the user identifies a pixel in the first frame, the model can identify objects made from the same material throughout the rest of the video. The researchers' technique can also be used to select similar materials in a video. The user identifies a pixel in the first frame (red dot in the far-left image on the yellow fabric) and the system automatically identifies objects made from the same material throughout the rest of the video.
Image: Courtesy of the researchers In addition to applications in scene understanding for robotics, this method could be used for image editing or incorporated into computational systems that deduce the parameters of materials in images. It could also be utilized for material-based web recommendation systems. (Perhaps a shopper is searching for clothing made from a particular type of fabric, for example.) “Knowing what material you are interacting with is often quite important. Although two objects may look similar, they can have different material properties. Our method can facilitate the selection of all the other pixels in an image that are made from the same material,” says Prafull Sharma, an electrical engineering and computer science graduate student and lead author of a paper on this technique. Sharma’s co-authors include Julien Philip and Michael Gharbi, research scientists at Adobe Research; and senior authors William T. Freeman, the Thomas and Gerd Perkins Professor of Electrical Engineering and Computer Science and a member of the Computer Science and Artificial Intelligence Laboratory (CSAIL); Frédo Durand, a professor of electrical engineering and computer science and a member of CSAIL; and Valentin Deschaintre, a research scientist at Adobe Research. The research will be presented at the SIGGRAPH 2023 conference. A new approach Existing methods for material selection struggle to accurately identify all pixels representing the same material. For instance, some methods focus on entire objects, but one object can be composed of multiple materials, like a chair with wooden arms and a leather seat. Other methods may utilize a predetermined set of materials, but these often have broad labels like “wood,” despite the fact that there are thousands of varieties of wood. Instead, Sharma and his collaborators developed a machine-learning approach that dynamically evaluates all pixels in an image to determine the material similarities between a pixel the user selects and all other regions of the image. If an image contains a table and two chairs, and the chair legs and tabletop are made of the same type of wood, their model could accurately identify those similar regions. Before the researchers could develop an AI method to learn how to select similar materials, they had to overcome a few hurdles. First, no existing dataset contained materials that were labeled finely enough to train their machine-learning model. The researchers rendered their own synthetic dataset of indoor scenes, whi...
-
1:19:18
The New American
2 hours agoThe New American Daily | Lawfare Against Trump Resumes
4.41K -
1:22:10
TheAlecLaceShow
5 hours agoGuest: Kevin Sorbo Trump Blasts DOJ | Hurricane Victims Get The Maui Treatment | The Alec Lace Show
14.5K7 -
1:00:43
The Dan Bongino Show
5 hours agoThe Deep State Strikes Back After The Disastrous Debate (Ep. 2342) - 10/03/2024
647K3K -
2:02:58
The Dilley Show
3 hours agoGovernment Covers Up North Carolina Disaster! w/Author Brenden Dilley 10/03/2024
36.7K19 -
1:42:10
The Shannon Joy Show
8 hours ago🔥🔥LIVE UPDATE With The Fiercest, Strongest Vaccine Injury Survivor In The World - The Amazing Alexis Lorenze & Her Angel Nurse Advocate Angela Wuhlbrecht!🔥🔥
31.9K9 -
LIVE
Rance's Gaming Corner
6 hours ago"And let us consider how we may spur one another on toward love and good deeds,"
157 watching -
43:55
Dave Portnoy
5 hours agoThe Unnamed Show With Dave Portnoy, Kirk Minihane, Ryan Whitney - Episode 31
33.2K1 -
54:56
Tate Speech by Andrew Tate
5 hours agoEMERGENCY MEETING EPISODE 79 - POSITIVE ENERGY
135K175 -
50:58
The Rubin Report
4 hours agoCharlamagne tha God Gets Quiet When Republican Shows Him Receipts
51.5K50 -
1:07:59
Steven Crowder
6 hours agoKamala Abandons America: Dockworker Strike & Hurricane Helene Devastation
284K894