“Alexa, turn on the kitchen light.”
“Alexa, play soothing music at volume three.”
How can I find my keys, Alexa?” I asked.
Using Alexa or Google Assistants, you can ask about facts, news, and the weather, as well as control smart home appliances (lights, alarms, TVs, etc.). Finding things is a capability that doesn’t quite exist yet; smart home assistants are essentially very rudimentary, auditory-only “brains.”
Would it be possible for home assistants to have bodies as well? What else would they be able to do for us? Is the answer “more than we want?”?
It may not be long before home assistants take on a whole new range of capabilities if Facebook’s AI research is successful. This week, the company announced that it will be working on “embodied AI”: basically, a smart robot that moves around your house to help you remember things, find things, and maybe even do things.
Home assistants that see and hear
The authors of a Facebook post discuss audio-visual navigation for embodied AI point out that many robots today are deaf; they use visual perception to move through space. By using both visual and audio data, it provides smart robots with the ability to detect and follow objects that make noise, as well as understand physical spaces through sounds.
Using SoundSpaces as a dataset, the company trains artificial intelligence. In SoundSpaces, sounds are simulated that might be heard in an indoor environment, such as doors opening and closing, water running, a TV show playing, or a telephone ringing. In addition, the nature of these sounds varies based on where they come from; the center of a room versus a corner, or a large, open room versus a small, enclosed space. To enable its AI to navigate based on audio, SoundSpaces incorporates geometric details of spaces.
According to the paper, an AI “can now act upon ‘go find the ringing phone’ rather than ‘go to the phone 25 feet southwest of your current position.’ It can discover the goal position on its own thanks to multimodal sensing.”
As part of the company’s offering, SemanticMapnet is a mapping tool that helps robots understand and navigate indoor spaces at the pixel level. With SemanticMapnet, smart robots will be able to answer questions about your home or office, like “How many pieces of furniture are there in the living room??? Where is the stove? ”””””” Our goal with SemanticMapnet is to enable robots to do the same-and in the process, help us find and remember things.
In mid-2019, Facebook released its Replica dataset and Habitat simulator platform.
The company envisions its new tools eventually being integrated into augmented reality glasses that can capture all kinds of information about the wearer’s environment and recall it at any time. According to Facebook’s chief technology officer, Mike Schroepfer, “If you can build these systems, you’ll be able to remember the important events in your life.”
Is it smart assistants or dumb people?
It is important to consider the deeper implications of these tools before embracing them. Aren’t we glad that our digital assistants can’t help us remember the important things in our lives?
Using paper maps, written directions, and good old-fashioned brain power (and perhaps occasionally asking another human for directions) were all we needed before GPS came along. Now, our phones guide us through every step of our journeys blindly. Did you ever notice how much harder it seems to learn your way around a new place or remember how to get to a new part of town?
Digital tools can seem all-knowing, which can lead to us trusting them without question, sometimes to our detriment (indirectly, like using our brains less, as well as directly, like driving into the ocean or nearly off a cliff).
Our ability to think independently decreases the more we outsource our thinking to machines. Would it be wise to continue this trend? Is it really necessary or desirable for smart robots to tell us where our keys are or whether we forgot to add salt to our food?
In some ways, allowing AI to take on more of our cognitive tasks and functions—to become our memory, which is the goal of Facebook’s new technology—will make our lives easier, but most technologies also have hidden costs or unintended consequences. Our homes and lives must be carefully weighed against these consequences before integrating a technology.