What Large Models Cost You – There Is No Free AI Lunch

Forbes explores the escalating costs behind large language models, revealing how training expenses are projected to exceed $1 billion within years as model sizes outpace hardware capabilities. OpenAI cofounder Sam Altman confirmed training costs already surpass $50-100 million and continue rising exponentially, with compute requirements doubling every few months.

The article examines both training and operational costs, with ChatGPT alone using 30,000 GPUs to handle daily user requests consuming 1 GWh of energy equivalent to 33,000 US households. State-of-the-art models require weeks or months of training across thousands of expensive processors, with longer iterations needed for increasingly large datasets.

Beyond infrastructure costs, LLMs require skilled AI engineers with hefty salaries and armies of human reviewers for development. API usage creates additional expense mountains as models often produce verbose output that wastes computational resources. Research shows over 99% of floating-point operations in large models result in zero calculations, highlighting fundamental inefficiencies.

Case studies demonstrate specialized approaches often outperform LLMs while reducing costs significantly. Diffblue’s reinforcement learning proved 10-25 times faster than OpenAI’s models for code generation, completing tests in 1.5 seconds versus 20-40 seconds while running locally rather than on expensive cloud GPUs. Companies must carefully evaluate whether they need sledgehammers to crack nuts.

Discover the complete cost analysis at Forbes.