More Gemini Features Coming to Google TV
Google TV integrates Gemini AI for image/video generation, enhances Google Photos, and adds a dedicated YouTube Shorts feed to the homepage.
Read moreGenerate content, images, music, and code, just like humans, but at an astonishing speed and with unparalleled accuracy. Generative AI aims to help businesses improve efficiency and drive innovation. As artificial intelligence becomes more prevalent, there will be stricter scrutiny on the costs (both financial and environmental) of producing these results and how they are generated.
We now have the opportunity to address this issue in advance and assess the primary allocation of resources. Inference is the process by which AI models analyze new data based on the intelligence stored in their artificial neurons, and it is the most energy-intensive and costly practice in model construction. The challenge is to implement more sustainable solutions without compromising quality and output.
Building a Model
For beginners, it is difficult to imagine how artificial intelligence and basic programming algorithms bear such a huge environmental and economic burden. A brief description of machine learning (ML) describes this process as consisting of two stages.
The first stage is training the model to develop intelligence and label information in specific categories. For example, an e-commerce company can input images of its products and customer habits into the model for deeper analysis in subsequent processes.
The second stage is identification, or inference, where the model uses stored information to understand new data. For example, an e-commerce enterprise can categorize products by type, size, price, color, and other segments while providing personalized recommendations to customers.
The inference stage is the less computationally demanding stage between these two stages. However, once deployed on a large scale, such as on platforms like Siri or Alexa, the cumulative computational power can consume significant energy and lead to increased costs and carbon emissions.
One of the most notable differences between inference and training is the funding support involved. Inference is associated with sales costs, which can impact profits, while training is typically associated with research and development expenses that are budgeted separately from actual products or services.
Therefore, inference needs to adopt dedicated hardware to optimize cost and power efficiency, supporting viable scalable business models that align commercial interests with environmental benefits.
Hidden Costs
Generative AI (gen AI) inspiration - ChatGPT is a significant example of the enormous cost of inference, generating millions of dollars in costs daily (excluding training costs).
GPT-4 recently released by OpenAI is estimated to require three times the computing resources of previous versions, with 16 expert models running on a 128-GPU cluster, reportedly having 1.8 trillion parameters and consuming massive amounts of energy.
The problem is further exacerbated by the significant flow of data through input and output. When DLAs process data, CPU hosts undergo significant data flow for handling input and output tasks of large amounts of data, as well as processing output data from DLAs. At the same time, DLAs themselves require energy for processing input and output data.
Again, it is important to emphasize that as a serial processing component, the CPU creates a bottleneck and cannot efficiently execute tasks as it cannot keep the DLA busy.
When a company relies on the CPU to manage inference in deep learning models, regardless of how powerful the DLA is, the CPU will reach an optimal threshold and then become overloaded. Imagine a car that can only travel at the speed allowed by its engine: if a small car's engine is replaced by a sports car's engine, the small car will collapse due to the speed and acceleration exerted by the more powerful engine.
The same applies to CPU-dominated AI inference systems - in most cases, DLAs (especially GPUs) are operating at high speeds, completing thousands of inference tasks per second, but the limitations of the CPU prevent them from fully utilizing their capabilities.
Need for System-Level Solutions
As NVIDIA CEO Jensen Huang said, "AI needs a holistic revolution in computing... from chips to systems."
With the exponential growth of AI applications and dedicated hardware accelerators (such as GPUs or TPUs), we need to shift our focus to the systems surrounding these accelerators and build system-level solutions that can support the required data processing capacity and speed. We need solutions that can handle large-scale AI applications while achieving seamless model migration with reduced costs and energy input.
Alternative solutions outside CPU-centric AI inference servers are crucial for providing efficient, scalable, and economically viable solutions to meet the sustainable demands of AI in enterprises, while also addressing the environmental impact of AI growth.
Promoting AI Accessibility
Currently, there are many solutions in the industry that can reduce costs while maintaining the vitality and development of generative AI. One approach could be focusing on using clean energy to power AI, while another approach could be performing computational processing during specific periods of available renewable energy.
For data centers, AI-driven energy management systems could achieve cost savings and improve environmental credentials of operations. In addition to these strategies, investing in hardware is the most valuable investment in AI. Hardware is the anchor for all computations and bears the burden of energy-intensive calculations.
A hardware platform or AI inference server chip that can support all processing at lower financial and energy costs would be revolutionary. This would be our way of promoting AI accessibility, as small businesses can leverage AI models that do not rely on resources exclusive to large enterprises.
The ChatGPT query machine consumes millions of dollars daily, and an alternative service chip solution based on fewer power consumption and GPU quantities of available resources would save resources and alleviate the burden on the global energy system, achieving an environmentally friendly, economically feasible, and inclusive generative AI application for everyone.
Google TV integrates Gemini AI for image/video generation, enhances Google Photos, and adds a dedicated YouTube Shorts feed to the homepage.
Read moreAlphabet Q1 paid subscribers reached 350M, fueled by YouTube & Google One. YouTube ad revenue slightly missed at $9.88B amid Premium growth. Total Q1 revenue hit $109.9B, beating forecasts.
Read moreAnthropic releases Claude connectors for Adobe, Blender, and Ableton, enabling AI workflow automation and app control. It also sponsors the Blender Development Foundation.
Read moreApple introduces an App Store model splitting 12-month plans into monthly installments. On iOS 26.4+, it lowers upfront costs without reducing total price, adding billing tracking.
Read moreGoogle launches Aadhaar digital IDs in Wallet for India, enabling secure QR/NFC verification. A phased partner rollout targets key sectors, while passport-based credentials expand globally.
Read moreYouTube tests "Ask YouTube," an AI tool blending text and video for complex queries. Initially for US Premium users, it supports interactive follow-ups, expanding Google's AI search.
Read moreGoogle grants US DoD classified AI access, joining OpenAI and xAI in defense deals amid Anthropic's dispute. Contractual safeguards against surveillance and lethal weapons remain unenforceable.
Read moreMistral AI launches Workflows, a Temporal-powered orchestration platform for secure, Python-driven enterprise AI automation featuring human-in-the-loop approvals.
Read moreCustom subliminal audio maker

AI Social Media Manager

AI Search Visibility

Workout and Calorie Tracker
Stock research platform

AI interview assistant

Scientific figure generator

AI courses with real code