Silicon’s midlife crisis
AI has evolved from classic ML to deep learning to generate AI. The most recent chapter dominated AI, in two phases (training and reasoning) depend on data and energy intensiveness in computing, data movement and cooling. Meanwhile, Moore’s Law determines that the number of transistors on a chip doubles every two years, reaching the plateau of body and economy.
Silicon chips and digital technologies have pushed each other over the past 40 years, and every step in processing power has enhanced the innovator’s imagination to envision new products, which requires more capability to run. In the AI era, this happened at light speed.
As models become increasingly accessible, large-scale deployments focus on inference and training models suitable for everyday use cases. This transition requires appropriate hardware to effectively handle inference tasks. Central processing units (CPUs) have managed general computing tasks for decades, but the widespread adoption of ML introduces computing requirements that extend the capabilities of traditional CPUs. This has led to the adoption of graphics processing units (GPUs) and other accelerator chips to train complex neural networks because of their parallel execution capabilities and high memory bandwidth that allow efficient handling of large mathematical operations.
But CPUs are already the most extensive deployment and can be companion to processors such as GPUs and Tensor Processing Units (TPUs). AI developers are also hesitant to suit software that is suitable for professional or customized hardware, and they favor CPU consistency and ubiquity. Chip designers are unlocking performance gains with optimized software tools, adding novel processing capabilities and data types, integrating professional units and accelerators specifically for ML workloads, and advancing silicon chip innovations including custom silicon. AI itself is a useful aid to chip design, creating a positive feedback loop where AI helps optimize the chips that need to run. These enhancements and powerful software support mean that modern CPUs are a good choice for handling a range of inference tasks.
In addition to silicon-based processors, disruptive technologies have emerged to address growing demands for AI computing and data. For example, Unicorn Launch LightMatter introduced a photon computing solution that uses light for data transmission to produce significant improvements in speed and energy efficiency. Quantum computing represents another promising area in AI hardware. Although years or even decades away, the integration of quantum computing and AI can further change fields such as drug discovery and genomics.
Understand models and examples
The development of ML theory and network architecture significantly improves the efficiency and capabilities of AI models. Today, the industry is shifting from a holistic model to a proxy-based system that is characterized by smaller professional models that can be done together on devices such as smartphones or modern vehicles to complete tasks more efficiently at the edge. This allows them to extract improved performance growth from the same or even fewer computations, such as faster model response times.
Researchers have developed techniques that include a small amount of learning for training AI models using smaller data sets and fewer training iterations. AI systems can learn new tasks from a limited number of examples to reduce dependence on large data sets and reduce energy demand. Optimization techniques such as quantization reduce memory requirements by selectively reducing accuracy, which helps reduce model size without sacrificing performance.
New system architectures, such as the generation of retrieval functions (RAG), simplify data access during training and inference processes to reduce computational costs and overhead. DeepSeek R1 (open source LLM) is a fascinating example of how to extract more output using the same hardware. By applying reinforcement learning techniques in novel ways, R1 uses fewer computing resources in some cases, enabling advanced inference capabilities.
