Domestic chips and large models' price cuts resonate: Enterprise AI applications shift from "training anxiety" to "inference efficiency"

In August 2026, Xiaomi released the Xuanjie O3 chip, while the API prices of domestic large models continued to decline. "US large model prices have been brought down by China" became a hot topic in the industry. The two signals appear to belong to two different tracks, chips and algorithms, but in fact point to the same thing: the cost structure of AI applications is shifting—the unit computing cost on the inference side has dropped significantly, while the massive investment on the training side is still borne solely by leading manufacturers. For the vast majority of software companies, the real dividing line is not "whether you can train a large model," but "whether you can engineer the inference capabilities of third-party models into your business at a controllable cost."

Training is expensive, inference is cheap: the cost curve is splitting

The large model industry has shown a significant divergence in cost curves over the past two years. On the training side, pretraining a model with hundreds of billions of parameters requires computing investment measured in billions of RMB, and it continues to expand with parameter scale. This determines that training capability will only be concentrated in a few players with deep pockets—whether closed-source vendors or teams taking the open-source route, the training threshold has been completely closed to ordinary enterprises.

The inference side is the opposite. With the maturation of engineering methods such as quantization, sparsification, speculative decoding, and KV Cache compression, the inference cost for the same model output has dropped by more than an order of magnitude over the past twelve months. Taking mainstream domestic APIs as an example, the per-token price of some small and medium-sized models has already fallen to the level of one-thousandth of a yuan. This means that for a business with a million calls per day, model inference costs have slid from "hundreds of thousands per month" to "tens of thousands per month or even lower."

This scissors gap has changed a fundamental decision variable: in the past, when enterprises built AI applications, the biggest concern was "can't afford it"; now the core contradiction has become "can't use it well"—the cost wall is lower, while the high wall of engineering capability and product understanding has become more prominent.

What domestic chips fill in is "deployment certainty"

Another piece of the puzzle in falling inference costs comes from the large-scale deployment of domestic chips. Behind new products like the Xuanjie O3 is the migration of the entire domestic computing ecosystem from "usable" to "easy to use": improved driver maturity, completed operator libraries, and lower adaptation costs with mainstream inference frameworks (such as vLLM, TensorRT-LLM, and domestic inference runtimes).

For companies doing software development, this brings a benefit that was often overlooked before—deployment certainty. When many customers privately deploy AI capabilities, what really blocks them is not model performance, but "can we buy it, can we adapt it, will the supply chain be constrained." The maturation of the domestic chip ecosystem has turned private inference from a "procurement risk item" into a "plannable technology selection item."

One detail worth noting is the collaborative optimization of inference frameworks and domestic computing power. Domestic GPUs/ASICs still lag NVIDIA in the general-purpose operator ecosystem, but through a combination of specialized operators, graph optimization, and quantization toolchains, actual inference throughput on specific architectures (such as Transformer Decoder) can already approach or even exceed imported cards at the same price. In other words, the key to selection is no longer just "what card to buy," but "what middleware to use to fully utilize the card."

Three specific impacts on software development teams

First, the budget model for AI application development needs to be rewritten. In the past, when building AI features, the bulk of the budget was computing power; now the proportion of computing power in costs is falling rapidly, and the budget focus should shift to data engineering, evaluation systems, and observability. For a mature enterprise AI application, what is truly expensive is continuous data feedback, governance of RAG knowledge bases, and evaluation baselines to prevent model "drift"—these are exactly the subjects that do not exist in traditional software engineering.

Second, "model-agnostic" architecture design has become a necessity. Inference costs and the supply landscape change too quickly; today's closed-source flagship may be matched by open source tomorrow, and domestic versus imported, cloud versus private deployment continue to tug back and forth in cost and capability. The smart approach is to encapsulate model calls into an independent inference gateway layer, keeping upper-layer business decoupled from specific models and specific computing power suppliers. In this way, whether switching to a cheaper domestic model or migrating from the cloud to private chips, there is no need to rewrite business logic.

Third, small and medium-sized companies have, for the first time, obtained an entry ticket to "AI-heavy" applications. Once inference costs fall to a certain level, scenarios that previously only big companies could afford to burn money on—such as intelligent customer service with full recall, real-time multimodal quality inspection, and personalized recommendations—can now also be run by SMEs with acceptable bills. This amplifies the space for differentiated competition, and also means that delivery standards for needs such as "software outsourcing development" and "management system development" have been raised: customers will by default expect AI capabilities to be an option, not a bonus.

Trend judgment: from "model worship" back to "system capability"

The industry over the past two years has clearly carried a tint of "model worship"—whoever has larger parameters and higher rankings stands in the C position. But when the cost curves of training and inference completely diverge, judgment will return to more plain technical common sense: a model is just one component in an AI application, and like databases and message queues, it can be replaced and tuned.

It can be foreseen that in the next twelve to eighteen months, the decisive factors in enterprise AI competition will shift to three things: refined management of inference costs, engineering maturity of private deployment, and product design that translates model capabilities into specific business metrics. The fiercer the price war between chips and models, the higher the value of this "dirty and hard work" becomes.

For frontline software teams, what they should do most right now is not chase the latest model launch events, but solidify the three foundations: inference gateway, evaluation system, and cost monitoring. Technology itself is not scarce; using technology to lower business costs and raise delivery certainty is the value that can truly accumulate—this is alsoShenzhen software developmentthe engineering mindset that enterprises need to recalibrate in the AI era.

*Shenzhen Xiangming Technology Co., Ltd. | Create value with technology | xiangmingit.com*