The Case for Using AI to Log Your Every Living Moment

People know Otter.ai as one of the AI-driven transcription services that have popped up over the past few years, automatically converting spoken words in interviews and meetings into text. The service can even distinguish between individual speakers. But its CEO, Sam Liang, sees this handy functionality as just a beachhead into a more sweeping and provocative project: Capturing everything you hear into a master dataset where you can search and reexperience every conversation you’ve ever had.

Liang began thinking about this a decade ago, after he left a job at Google to cofound a startup that monitored people’s behavior on mobile devices to provide services like automatically tracking mileage expenses. “I’m obsessed with getting data and understanding data,” he confesses. “In my first startup, we used a lot of iPhone sensors: location, GPS, Wi-Fi, motion. The one sensor we didn't use was the microphone.” Fixing that would be transformational, he thought. “I was frustrated that with Gmail I could search for something from 10 years ago, but there was no way to search for something I heard three hours ago,” he says. “So I did a thought experiment. What if I keep my microphone on the whole day?” Liang then raised the stakes still further. “What if I do it even better—what if I kept the mic on all the time, my entire life—from the day I started talking till the day I die?” He calculated how much data that would be and figured out that you could store a lifetime of audio on a 2-terabyte USD drive. “Then I can search for everything I heard in my whole life,” he says. “My parents have already died. I really wish I can just retrieve their speech.”

Liang isn’t the only one chasing the dream—or perhaps nightmare—of total AI-powered recall. As I wrote in February 2021, a startup called Rewind has already launched with the promise of life-capture, and it has since tapped the latest AI advances to build out that vision. Founder Dan Siroker recently announced a wearable pendant to more nimbly snare everything within electronic earshot. And just this month a much ballyhooed new startup called Humane announced a replacement for the smartphone in the form of a “pin” that can also capture voice.

These products join countless devices like Amazon’s Alexa with microphones always at the ready, potentially fertile ground for apps that can passively record. Maybe the rise of generative AI marks the inflection point for this idea. Using that technology, recording corpuses can become datasets by which people can search through and summarize their life events, and literally dialog with the minutiae of their existence. It might be like having your personal Robert Caro–level biographer on hand.

As you might expect, civil libertarians have some issues with this concept. Jay Stanley, a senior policy analyst for the American Civil Liberties Union, says that the rise of always-on audio capture raises tensions between personal privacy and the right to record. But he mostly worries about how all that data might be used against people, even if originally intended to enhance their memories. “The prospect raises questions about whether the data will be protected, whether it will be vulnerable to hacking, and whether it can be vulnerable to access by the government,” he says. Overall, he thinks services that record all your conversations are a bad idea. “People might feel like it's empowering to have a record of everything they've ever heard, like a super memory or something like that. But it could actually be disempowering and turn against you.”

Not surprisingly, Liang and Siroker both insist that privacy is built into their systems. Both say that they discourage recording anyone without consent. And of course they vouch for the security of their systems.

Just like email is now a staple in legal cases, those recordings would inevitably find their way into courtrooms. Both Otter and Rewind say that they would comply with official subpoenas. Liang says that it would be good to have new legislation that raises the bar of what the authorities could ask for in a probe. The ACLU’s Stanley says that this wouldn’t be unprecedented. “Under British common law, it used to be the case that even with a warrant the authorities couldn't access your diary or your letters, because that was viewed as making you testify against yourself.” Unfortunately, that’s not current law, but Liang wants to revive and spread the concept. He also thinks we should nix state laws that mandate mutual consent if one party in a conversation wants to record. “I am not a politician, but the eavesdropping law is crazy,” he says. “Fifty or 60 years ago, the intention was good, but now things have evolved tremendously with the AI revolution and ChatGPT.” In his view, you shouldn’t be able to share someone else’s words without consent, but if you can hear something, you should be able to retain it. “Everyone has the right to remember,” he says.

An even thornier question is whether people who speak to the always-on subscription services will be comfortable with the concept of having all their conversations recorded. People may reject this idea as thoroughly as they did Google Glass, incensed at its ability to passively record video. “This may be determined not in the realm of law, but in the realm of etiquette,” says Stanley. “Do I really want to sit down with my old college friend and have a night at a bar talking about old times when he's recording everything?”

If the practice of total recording in pursuit of total recall does catch on, it will be due to AI’s ability to provide tangible benefits from exploiting a lifetime corpus of conversation—or even just a digital log of what you forgot from that recent night on the town. Right now, the cutting edge of this phenomenon occurs in business meetings. Because so many now include remote participants, recording the session is a one-button choice that is increasingly invoked. (Liang says Otter has transcribed over a billion meetings.) In the workplace, at least, the etiquette is established. And those recordings are becoming more useful. Generative AI now provides a huge array of features. With Otter, meetings can be summarized in real time with reasonable accuracy, and chatbots allow participants and latecomers to ask whether a specific issue was addressed. During a meeting you can ask whether a topic had been previously discussed. And when the meeting is over, the results can be dumped into a corpus that’s an invaluable guide to a company’s operations.

For Liang that’s just the beginning. He tells me Otter is working on a feature called Avatar that would enable him to run a meeting without the bother of actually attending. It’s essentially a chatbot built around years of past data on his contributions in meetings. “I’m often double-booked, so for those meetings I can send my avatar, which can answer probably 90 percent of the questions people ask me,” he says. I ask him if that might be risky—what if the avatar OKs a business plan that tanks the company? “We’d give it only a certain level of authority,” he says after considering the concept. “Hey, maybe you can approve anything less than $10,000.” Remind me to ask Liang’s avatar to cut me a check next time we’re in the same Zoom room.

Certainly some of the features useful in business meetings could be applied in our personal lives. Chatbots might take on eternal questions like “What went wrong in my marriage?” and deliver shockingly accurate answers. But would we want this? In general, a lot of what we say might not look good if it were recorded and somehow leaked into the open. In a world where everything was recorded, people might feel constrained in expressing themselves freely. Siroker thinks that won’t happen, because AI will be so good at faking someone’s voice that everyone will be able to plausibly deny any faux pas or criminal utterance. That sounds to me like the new version of denying responsibility for a toxic social media post by claiming your account was hacked. It didn’t work too well for Anthony Weiner.

Ideally, the public will have a reasoned discussion about balancing the loss of privacy from having everything recorded with the benefits of having instant access to our verbal history and most replayable personal moments. But I suspect we won’t get to vote on it. Like everything else that happens in technology, it’s more likely that companies will offer it preemptively. Being on the record all the time might win favor by allowing us to recover a brilliant idea we had at lunch or in the shower, replay conversations with departed loved ones, and settle disputes on who agreed to do the dishes. Before we know it, we might be hooked. Too bad—it would have been nice to have that discussion about balance. And, of course, to record it.

Speculative concerns about always-on microphones are reminiscent of the fears that arose when it first became clear that it might be possible to track people’s location en masse. I wrote about this for Newsweek in June 2004, in a column entitled “A Future With Nowhere to Hide.” Now we carry around location beacons in our pockets, and Apple operates a global location tracking network.

The idea of shedding wires and cables is exhilarating: we can go anywhere and still maintain intimate contact with our work, our loved ones and our real-time sports scores. But the same persistent connectedness may well lead us toward a future in which our cell phones tag and track us like FedEx packages, sometimes when we're not aware …

My guess is that the widespread adoption of tracking will be done not against our will but initially with our consent. As with other double-edged tools, the benefits will be immediately apparent, while the privacy drawbacks will emerge gradually. The first attraction will be based on fear: In addition to employers' keeping workers in line, Mom and Dad will insist that their teenagers have GPS devices in order to be able to follow them throughout their day, a human equivalent of the LoJack system to find stolen cars. The second stage will come as location-based services, from navigation to "friend finding" (some systems tell you when online buddies are in shouting range), make our lives more efficient and pleasurable.

Sooner or later, though, it will dawn on us that information drawn from our movements has compromised our "locational privacy"—a term that may become familiar only when the quality it refers to is lost. If nothing is done, pursuing our love affair with wireless will result in the loss of a hitherto unheralded freedom—the license to get lost. Here's a new battle cry for the wireless era: Don't Geo-Fence me in.

Erick asks, “What is the most boring (but valuable) thing that AI can do really well?”

Thanks for asking, Erick. Special credit for responding to my desperate last-minute plea on Threads for a reader question. People, I know it’s fun to ask ChatGPT anything, but while it’s still possible, take advantage of a human being willing to provide an answer to your burning questions. So don’t be shy about submitting your queries!

Back to the question at hand: Erick, come on—this is like asking what’s a boring-but-valuable use of electricity. AI is embedded into so many products and procedures these days that the vast majority of its uses, from spell-checking to load control in data centers, are pretty boring. What’s more, every exciting AI use case inevitably becomes boring once the task is mastered and implemented. Remember how thrilling it was to see AI play chess? Now it’s just one of those things that computers do much better than us. AI is always more interesting when scientists and engineers can’t quite yet make them happen. Right now, people are obsessed with the failures of autonomous vehicles. Every time a self-driving car gets in an accident—some of which have beentragic—or refuses to move when blocking a fire truck, is noted and amplified. But whenever self-driving cars get their act totally together, the whole subject will be a yawner—just one more boring (but valuable) thing that AI does really well.

You can submit questions tomail@wired.com. Write ASK LEVY in the subject line.

The sandwich chain Quiznos polled 2,000 American adults about their thoughts on the apocalypse, asking questions like whether they could outlast everyone else and be the last human alive, who they’d like to survive with if they had a partner, and of course what they’d eat. “Thinking about the fallout of an apocalypse, we’re thankful for the deliciously toasted subs that are available to us all for now!” the company’s chief marketer said. If I believed the apocalypse was imminent I’d probably order something fancier.

The epic story of three teenage hackers who broke the internet—and wound up working for the FBI.

If a Black Bill Gates emerges, you’d probably find that person at the Black Blockchain Summit.

How Taylor Swift and Beyoncé saved movie theaters.

Here are the best cookbooks of 2023. None deals with postapocalyptic cuisine.

Plaintext will be off next week for Thanksgiving. Enjoy the holiday and see you in December!