Facebook’s chief AI scientist had been among a group of computer scientists who retained faith in deep neural networks during an “AI winter” of reduced funding and interest in the field. In 2019, his efforts earned him a share of the Turing Award, together with his friends Yoshua Bengio and Geoffrey Hinton. Today, AI is now an essential component of Facebook’s vast array of applications, touching everything from Messenger to content moderation. “You take AI out of Facebook, and basically the services crumble,” LeCun tells TNW. But fears are now emerging that another winter will soon arrive if AI can’t live up to its current hype, particularly around the promise of artificial general intelligence (AGI): the idea that a machine can perform any intellectual task a human can — and many that they can’t. LeCun is not a fan of the term. He’s previously argued that “there is no such thing as AGI” because “human intelligence is nowhere near general.” However, he is keenly pursuing “human-level AI.” His chosen technique for reaching it is self-supervised learning. In supervised learning, people painstakingly label data and then feed it to an algorithm to teach it what to look for when solving a problem. But in self-supervised learning — often confused with unsupervised learning — there’s no need for human annotation. Instead, the system generates signals from the data and uses them to train itself. It’s already become an integral part of Facebook’s work on hate speech detection, language translation, and forecasting the spread of COVID-19. LeCun compares the method to how babies learn by interacting with their surroundings. If machines can better replicate their technique, he believes they could gain the common sense they need to reach human-level intelligence:
FAIR origins
FAIR began life after Mark Zuckerberg and Facebook CTO Michael Schroepfer identified AI as critical to the company’s long-term future. “They were 100% right about this,” says LeCun, now Facebook’s chief AI scientist. They decided to create a new research lab from scratch — and handpicked LeCun to lead it. “What attracted me was there was a lot of opportunity. But also, I had somewhat of a carte blanche to organize the lab in the way I thought would be the most successful.” LeCun immediately made open research the cornerstone of his plans. FAIR now publishes almost all of its work, and open sources the majority of its code, datasets, and tools — such as PyTorch, a toolkit for quickly creating new machine learning models. The motivation behind this approach isn’t purely altruistic. It allows Facebook to influence what researchers work on, foster collaborations with academia and industry, and attract talent to FAIR. “The currency for a scientist is his or her intellectual impact on the community,” says LeCun. “So if you want to hire the best scientists in the world, and you tell them you can come here but you can’t talk about what you do — they’re not going to join.” But isn’t he worried that Facebook’s rivals could steal the lab’s secrets? “That’s fine. Why would it be bad? The value of a lot of the technology we produce is multiplied by a coefficient that is basically Facebook’s ability to deploy them in its services. It’s much more difficult — even if it’s open source — for another company to deploy it in ways that would compete directly with us.” On the contrary, sharing the research helps Facebook improve its own products:
AI advances and challenges
Much of the fundamental research behind the AI in Facebook products now takes place at FAIR. Among its most impactful creations are Memory Networks, which improve how machines talk to people by helping them retain enough data to answer general knowledge questions. An influential 2014 paper showed how the approach could answer questions about the plot of the Lord of the Rings. “DeepMind was working on a very similar idea exactly simultaneously,” LeCun recalls. “We posted a paper on arXiv, and then three days later DeepMind posted their paper, because they didn’t want to be completely scooped. ” At that time, DeepMind was one of the driving forces behind London’s emergence as a global AI hotbed. FAIR had initially planned to join them by launching its first European lab in the city. But after studying the availability of talent, the company pivoted to Paris. “It was a more difficult turf to enter, whereas continental Europe was completely open,” says LeCun. “There was essentially no ambitious fundamental research lab in AI or even in information technology in continental Europe really.” The lab has since become one of FAIR’s largest AI research centers and the birthplace of many of its computer science breakthroughs. Its recent innovations include Facebook AI Similarity Search (FAISS), a tool for quickly finding videos, text, or images that are similar to each other. The system can be used to recommend Instagram posts or to detect extremist propaganda videos that have been tweaked and then reposted to evade removal. FAIR has also helped Facebook use AI to find hate speech on the platform. But these systems have received criticism for not detecting every language. However, self-supervised learning is expanding the linguistic range of the tools. But LeCun believes the technique will only reach its potential once it can reason like a human. So you like our media brand Neural? You should join our Neural event track at TNW2020, where you’ll hear how artificial intelligence is transforming industries and businesses. We don’t know how long it’s going to take up to break through this first wall. It could be next year, it could be five years from now, it could be 10 years, it could be 20 years. There’s good hope it’s going to happen soon, but how do we know? And then we’ll most likely encounter other obstacles that we don’t realize exist.
