In 2015, MIT Technology Review named Ilya Sutskever as one of the visionaries under 35. At that time, he was 29 and had made a significant breakthrough a year earlier by publishing a research paper on the seq2seq functioning of language models. Sequence-to-sequence represented a novel approach to text interaction (indeed, to any elements), enabling the processing of text itself and its context. This seq2seq model would later become foundational in a significant update to Google Translate and underpin all modern online translators, heralding new capabilities in translation. It moved beyond mere mechanical word and phrase translation to incorporate contextual understanding. Sutskever himself noted that researchers were skeptical about machine learning’s potential to achieve text translation, so its success was a “big surprise.”
However, most people became aware of Ilya Sutskever not because of his AI achievements but because of the story surrounding Sam Altman’s dismissal. It’s said that Sutskever was behind his firing and was the main instigator. Ultimately, this episode turned into a fiasco, with Altman returning and nearly all OpenAI employees rallying behind the CEO, putting the company at a crossroads: either face a complete halt in operations or revert to the status quo. It seems that for Altman, things ended well, as he only solidified his leadership within the company. On the other hand, according to Business Insider, this put Sutskever’s future into question, at least within OpenAI.
On the one hand, sources within the company suggest that Sutskever is likely to receive a new position, and his smiling photo with Brockman hints at a desire within the company to return to standard operational activities. On the other hand, other sources indicate that there are questions about a return to “business as usual,” primarily concerning trust within the company. A former OpenAI employee stated, “Once trust is undermined, it’s impossible to restore.”
The conflict at OpenAI reflected not just a power struggle, as many believe. This overly simplistic view fails to capture the fundamental contradictions within OpenAI. The company was founded as a non-profit, but in 2019, it established a subsidiary, OpenAI LP, to attract investments and commercialization. According to Bloomberg, the difference in approach between Altman and Sutskever was rooted in their visions for the company’s future. Chief Scientist Ilya Sutskever saw the company as developing and exploring AI and machine learning possibilities. However, CEO Sam Altman viewed the company more as a typical Silicon Valley tech startup. With the release of ChatGPT, the company rode the wave of generative artificial intelligence, becoming an industry pioneer and a leader in this promising new direction.
According to the Information, Sutskever did not see his actions as a coup, but after facing criticism from employees, he acknowledged that “it could be interpreted as a coup.” As a result, he publicly apologized. And even signed a letter asking for Altman’s return. Now, nearly two weeks later, Sam Altman has returned, but Sutskever’s future remains uncertain, and he has yet to receive a reassignment, reflecting ongoing trust issues within OpenAI.
Content
The beginning
Ilya Sutskever was born in 1984 in Nizhny Novgorod, which was then called Gorky in the USSR. In 1991, following the dissolution of the USSR, he immigrated to Israel with his parents, where he studied computer science at the Open University of Israel. Later, he moved to Canada with his family. 2007, he earned a master’s degree in computer science; in 2013, he got a PhD. In a university interview in 2017, he said, “I was surprised when I found out that computers can learn to some extent because, in my mind, learning was not something computers could do at the time.”
At UofT, Sutskever met Geoffrey Hinton, an expert in artificial intelligence and, in particular, deep learning, one of the fields behind AI and machine learning that utilizes large volumes of data without being confined to a specific algorithm. Such models can independently create new functions and solve more complex tasks.
It could be considered an incredible stroke of luck, as is everything in this world, because Hinton is often called a pioneer in deep learning. Deep learning implies that the system not only performs specific actions within a single algorithm, as standard systems did before but acquires something akin to a human brain, enhancing its capabilities with external data. Of course, this is a highly simplified representation and description, but casually speaking, that’s essentially it. Hinton wanted to develop a system capable of learning like humans, going beyond the confines of an algorithm. At that time, this was a groundbreaking idea, not just in terms of implementation but in its essence. Most scientists were convinced that it was impossible.
At that time, the most advanced artificial intelligence system, if one could apply that term, was Deep Blue: a chess program developed by IBM that defeated Garry Kasparov in 1997. In 2010, Sutskever became the only Canadian recipient of the Google PhD Fellowship Program, aimed at the most promising researchers in artificial intelligence. At that time, Sutskever worked on RNN technology (recurrent neural networks). Sutskever and his team processed a sample of 16 million texts to teach the algorithm to select the next symbol after entering the previous ones. This technology now underlies many chatbots.
AlexNet, DNNResearch and Google Brain
The development of the RNN model was only the first step. In 2012, Ilya Sutskever, along with Alex Krizhevsky, another student of Hinton, turned their attention to the possibilities of machine learning in image analysis. This marked the beginning of AlexNet, a neural network for image identification capable of analyzing thousands of images and identifying objects depicted in them. Previously, the chance of error was around 25%, but AlexNet reduced this figure by almost half. This was an impressive accuracy level at that time, thanks to significant computational power and many analyzed images (in this case, a million). It won the ImageNet and was recognized as one of the most innovative models. Thus, Sutskever and AlexNet laid the foundation for the AI era. This was the first work that popularized convolutional neural networks capable of solving classification tasks, processing elements, and their detection.
Nvidia played a significant role in developing the first GPUs and graphic processors. Initially, they were designed for fast processing and rendering of images, creating new possibilities in game development. However, it turned out that the calculations, which GPUs handled well, were similar to those needed for training neural networks. Essentially, this involved multiplying massive grids of numbers, which was almost the same as deep learning.
There is a myth that Sutskever went to New York and brought back a trunk full of GTX 580s, which were then used in AlexNet. This is likely just a myth that surrounds many innovations. Another version states that Sutskever ordered the GTX 580s online, and he voiced this version. Regardless, this was a unique convergence of ideas with new technological possibilities.
Together, Hinton, Sutskever, and Krizhevsky created the research startup DNNResearch, which focused on research in Natural-language understanding, an AI field related to speech recognition. Sutskever was convinced that the possibilities of deep learning could be used for generating text and images. As Jeff Dean said, Sutskever had an intuitive understanding of where artificial intelligence could go. A few months later, DNNResearch was acquired by Google.
After joining Google, Sutskever worked at Google Brain, focusing on further studies in artificial intelligence and machine learning. During this time, he wrote a paper on the seq2seq algorithm, which later became the foundation of a new approach to Google Translate, changing the way we use online translators. Previously, they resembled more of an online dictionary, translating words or popular phrases. Still, the new Google Translate gained the ability to analyze text context, preserving the meaning across different languages.
In the Google Brain team, Ilya Sutskever was instrumental in developing TensorFlow, a platform for large-scale machine learning. Today, TensorFlow stands as the world’s most renowned machine learning system, enabling the creation of neural networks.
AlphaGo
Ilya Sutskever’s tenure at Google included collaboration with DeepMind researchers on AlphaGo, an artificial intelligence program designed to play Go, a profoundly strategic and ancient Chinese board game. Historically, Go was considered one of AI’s most challenging classical games due to the complexity of evaluating positions and selecting advantageous moves. In a landmark achievement for AI development in 2016, AlphaGo defeated Go champion Lee Sedol, signifying a machine’s superiority over humans in one of the most intricate strategic board games. AlphaGo’s approach transcended mere probability calculations; it demonstrated creativity and an ability to grasp the intuitive aspects of the game. This system transformed perspectives on Go gameplay and revolutionized scientific understanding. For instance, AlphaGo’s methodologies contributed to advancements in protein folding research, which is crucial for drug development and combating diseases.
OpenAI and the rise of AI
At the end of 2015, with funding from Elon Musk and a group of investors, the foundation was laid for OpenAI, a non-profit organization aimed at developing digital intelligence for the benefit of humanity. Ilya Sutskever, joining the project as the Chief Scientist and co-founder, left his position at Google, inspired by the mission of OpenAI. The company aimed to democratize advanced AI technologies, promising openness and accessibility in their developments. In an interview with Jensen Huang from NVIDIA, Sutskever emphasized the importance of unsupervised learning through data compression as a fundamental idea that led to a series of breakthrough research at OpenAI.
Since its inception, OpenAI has demonstrated impressive achievements: the development of the Gym platform for AI training, the creation of Universe for teaching AI to perform various tasks, and the success of the OpenAI Five bot in Dota2 that has defeated the Dota 2 Champions, OG in 2019. Within OpenAI, Sutskever led the development of the GPT language models up to GPT-4 and participated in creating DALL-E, a neural network for generating images. In March, the company introduced GPT-4, showcasing the remarkable capabilities of this model.
In an interview with Craig S. Smith, Sutskever expressed his belief in the immense benefits of AI but also shared concerns about its potential harm to humanity. He advocated for a world where AI is a support, not a replacement, for human experience and freedom of choice.
The release of ChatGPT marked a pivotal moment for the company, as the language model, combined with a simple interface and accessibility to anyone, had an explosive impact on people. For the first time, hundreds of millions of people worldwide could experience AI and appreciate what OpenAI has been working on. ChatGPT might not have been perfect in many aspects, but it introduced people to generative AI and allowed them to try it in real life. OpenAI staff expressed many concerns associated with the release of ChatGPT. “What if people didn’t like it?” one of the developers said. But people did like it, and this opened a new chapter in AI, where it became a mainstream direction in technology development. What many researchers considered nonsensical 10 years ago became accessible to everyone with just a few clicks.
Of course, it remains just a toy for most people, but 100 million users in less than two months speaks to its appeal. Aaron Levie, CEO of Box, a data storage company, said, “ChatGPT is one of those rare moments in technology when you see a glimpse of how everything will be different in the future.” And for many people, it was not just a toy but a vision of the future that was coming. It inspired some, but it raised fears about their future for many.
Undoubtedly, ChatGPT is not perfect and often produces mistakes and inaccurate information. On the internet, questions about finding the position of the letter “r” in the word blueberry or jokes about the chatbot giving contradictory answers depending on the question have become memes. As soon as you encounter this, the wonder behind ChatGPT collapses, and you realize how imperfect the system is. But by that point, you are already so impressed that it’s enough.
And he, Ilya Sutskever, has primarily predetermined the direction in which our world will head.
