"Like humans, artificial intelligence models do not start learning from scratch every second. Instead, a type of neural network called LSTM (Long Short-Term Memory) has been developed to incorporate previous observations with each new observation, improving the accuracy significantly. LSTM was co-developed by Dr. Sepp Hochreiter and German scientist Juergen Schmidhuber in the late 1990s.
"As the successor to LSTM, we have something new. It has not been publicly published but has been kept hidden. It is called XLSTM," revealed Professor Josef "Sepp" Hochreiter, the current head of the Machine Learning Research Institute at Johannes Kepler University Linz, in an exclusive interview with AIM.
Hochreiter and his team are currently training each transformer by combining it with a smaller dataset compared to GPT. He excitedly stated, 'We are much better than GPT and hope to surpass OpenAI in autoregressive language modeling and kick them out of the market.'
OpenAI, a Silicon Valley startup led by Sam Altman, has gained reputation since the launch of its profitable chatbot, ChatGPT. According to Reuters, OpenAI is projected to reach $1 billion in revenue by 2024, gaining market support.
Before LSTM became a crucial component of language models, it had fascinating success in reinforcement learning applications such as Deepmind's Starcraft 2 and OpenAI's Dota2.
Hochreiter said, 'What is even more surprising is its excellent performance in language, even though it was not designed for language. It was designed for time series prediction and sequence analysis.' He also used the model for protein and DNA sequence research before it became popular.
The 55-year-old professor believes that focusing on language is beneficial because language already exists with human-invented vocabulary to describe objects in the real world. He added, 'These concepts, categories, and abstractions always come from humans, and I look forward to seeing artificial intelligence create its own concepts to describe servers and answer its own abstract questions.'
Today, LSTM is not only the core model for intelligent voice assistants like Alexa, Siri, and Cortana but is also used by government agencies worldwide to predict floods and droughts. Hochreiter believes that transformer technology may not be suitable for all domains. He thinks that LSTM interacts with traditional architectures for certain engineering tasks and has better perception of new things.
Some of the largest language models still have mysterious training data behind them. Hochreiter pointed out that initiatives like LAION (Language Artificial Intelligence Open Network) are emerging to create datasets that cannot be used for training. He said, 'It is a very complex matter because different cultures may have different views on what is appropriate or inappropriate. It is a problem.'
He further explained, 'In training, you cannot use certain books. OpenAI is a bit naive in this regard because they use all the data, and lawsuits may arise soon. The more data, the better the model performs, but you have to be careful about selecting data for summaries and content.'
Since the introduction of generative AI tools like Midjourney and ChatGPT, criticism towards tech companies has been increasing. The latest addition to the list of critics is American writer and comedian Sarah Silverman.
Hochreiter said, 'In language models, what you input is what you get as output. The first thing is that there are some regulations on what it can say.' Global regulatory bodies are working hard to address the legal gray areas faced by artificial intelligence.
Background Story:
Young Hochreiter initially found computer science boring until he discovered neural networks. 'Everything in computer science was already known 30 years ago, but here, you can do new things. It is so fascinating and unexplored,' he said.
As a pioneer in deep learning, he discovered the vanishing gradient problem before proposing LSTM. 'When I was writing my thesis, my supervisor moved to the US and became a postdoc there. When he came back, we had a lot to write, and we tried to publish it in NeurIPS in 1995, but it was rejected.'
It became a NeurIPS paper in 1997. Hochreiter said, 'Maybe everyone has similar experiences,' as it is difficult to get a paper accepted due to various issues with the current peer review system. Many papers that have had a significant impact on the field were initially challenging to be accepted, such as the PageRank paper, Kalman filter paper, and LSTM.
Things changed between 2009 and 2011 when Alex Graves, a student of Schmidhuber, collaborated with LSTM, and recurrent neural networks became popular again. Reflecting on the sudden popularity of his work, Hochreiter said, 'Everything went smoothly, from Google, Facebook, Facebook to Amazon, all the IT giants jumped on this bandwagon. Looking back, no one was interested at that time, and I was surprised it became so popular because I knew from the beginning that it would work. And with the growth of computing power and more data, it became even more powerful.'
Although he is currently developing a model to compete with GPT, Hochreiter is unsure whether he wants to keep this technology confidential or pursue commercial development in a company format. He said, 'I published LSTM, but I didn't get a penny.'
'I want to see what I can do without publishing. This way, I can keep something in Europe, a new technology that has a better sensitivity model for language modeling. But who knows, because I have only worked on small datasets. I haven't said much about it, but it is a combination of LSTM and Transformer ideas,' he concluded.
Source: https://analyticsindiamag.com/sepp-hochreiter-to-kick-out-openai-from-llm-supermarket/"
Alphabet Q1 paid subscribers reached 350M, fueled by YouTube & Google One. YouTube ad revenue slightly missed at $9.88B amid Premium growth. Total Q1 revenue hit $109.9B, beating forecasts.
Anthropic releases Claude connectors for Adobe, Blender, and Ableton, enabling AI workflow automation and app control. It also sponsors the Blender Development Foundation.
Apple introduces an App Store model splitting 12-month plans into monthly installments. On iOS 26.4+, it lowers upfront costs without reducing total price, adding billing tracking.
Google launches Aadhaar digital IDs in Wallet for India, enabling secure QR/NFC verification. A phased partner rollout targets key sectors, while passport-based credentials expand globally.
YouTube tests "Ask YouTube," an AI tool blending text and video for complex queries. Initially for US Premium users, it supports interactive follow-ups, expanding Google's AI search.
Google grants US DoD classified AI access, joining OpenAI and xAI in defense deals amid Anthropic's dispute. Contractual safeguards against surveillance and lethal weapons remain unenforceable.
Mistral AI launches Workflows, a Temporal-powered orchestration platform for secure, Python-driven enterprise AI automation featuring human-in-the-loop approvals.