From Token Cost to Task-Level Intelligent Compute Orchestration: Databricks’ AI Coding Experience and Forge’s Engineering Delivery Practice

As coding agents such as Claude Code, Codex, and Cursor increasingly enter real-world software development workflows, enterprises are beginning to face a problem that barely existed before: when a single software engineering task can autonomously read dozens of files, reason continuously, search through code, invoke tools, run tests, and repeatedly revise its implementation, AI coding can no longer be treated as a simple model-API expense.

What enterprises actually need to manage is how much model capability, context, inference compute, verification, and human intervention are required to complete a software engineering task at the required level of quality.

Databricks’ recently published internal coding-agent benchmark provides valuable engineering evidence. Its evaluation was based on historical engineering tasks drawn from a real, multilingual codebase containing millions of lines of code. The results showed that per-Token model pricing is not a reliable predictor of final task cost; models tend to fall into different capability tiers; and even when the same model is used, the cost per task can differ by more than 2× depending on the agent harness. Within Databricks’ internal workload distribution, roughly one quarter of tasks were classified as low-complexity and around 60% as medium-complexity, yet expensive models had often been used as the default.

These findings point to a problem that is more important than simply “reducing Token consumption”:

Enterprises need a software engineering runtime that can dynamically allocate intelligent compute resources according to task difficulty, model capability, reasoning budget, and required quality.

In HaxiTAG’s project delivery practice with Forge, its software engineering delivery agent, this problem has similarly evolved from model selection into a broader engineering-control problem. Requirement inputs, project knowledge, dynamic context, model and project configuration, code execution environments, testing and verification, security checks, and project-version evolution are organized within a persistent project runtime.

From this perspective, the ultimate objective of AI coding cost governance is not to procure a cheaper model. It is to build an engineering system that can continuously answer a more fundamental question:

How much intelligence does this task actually require?

The Core Problem Revealed by Databricks: Token Price Is Not Software Engineering Cost

Traditional LLM cost analysis often begins with the price of input and output Tokens. For simple question-answering workloads, this approach is generally adequate. Once coding agents enter the workflow, however, the cost structure changes substantially.

Databricks provides a straightforward example. Sonnet 5 was approximately 1.7× cheaper than Opus 4.8 on a per-Token basis. Yet on Databricks’ real coding tasks, Sonnet 5 cost approximately $2.09 per task on average, compared with $1.94 for Opus 4.8. Their task-completion rates were approximately 81% and 87%, respectively. The main reason was that Sonnet 5 ran for longer, read more content, and ultimately consumed approximately 1.9× as many Tokens.

The effect of the agent harness is even more noteworthy. When Databricks executed the same tasks using the same model and the same reasoning intensity but different agent harnesses, it observed cases in which the cost per task differed by more than 2× while result quality remained broadly similar.

The main difference came from context management. The lighter-weight harness sent roughly one-third as much context to the model per turn and completed tasks using a more compact working set.

This suggests that the cost of a complete software engineering task can be represented conceptually as:

Task Cost = Model Capability × Reasoning Budget × Context Size × Execution Turns × Tool Calls × Retries and Verification

Enterprises therefore need to abandon an overly simplistic assumption: a cheaper model API does not necessarily mean lower AI coding cost.

The relevant comparison is:

How much total cost is required to complete a verified engineering task?

Complex Tasks Require More Compute, but “More Tokens” Does Not Automatically Mean Waste

Token consumption cannot be governed in isolation for a deeper reason related to how modern reasoning models work: for genuinely difficult problems, additional inference-time compute may improve result quality.

OpenAI’s research on reasoning models has observed continued performance improvements as more test-time compute is allocated. Similar patterns have also been observed in agent benchmarks such as BrowseComp, where additional reasoning and search effort can improve the ability to solve complex tasks.

Accordingly, when an agent is dealing with cross-service concurrency problems, complex performance failures, security vulnerabilities, or architectural migrations, reading more code, spending more compute on reasoning, and executing more tests should not automatically be classified as abnormal cost.

That additional computation may simply be necessary to obtain a correct result.

The real governance targets are three different forms of inefficiency.

The first is capability overprovisioning: tasks such as configuration edits or simple test generation are routed by default to the most capable and expensive models.

The second is compute overprovisioning: the model is already capable enough, but context continues to expand, the same files are repeatedly reread, and the agent performs unnecessary searches or redundant reasoning.

The third is failed compute: substantial model and tool resources are consumed without producing a result that can pass testing, review, and acceptance.

Forge therefore evaluates software engineering agent cost in relation to the task outcome:

Task Complexity → Required Capability → Model Capability → Inference Compute → Verification Result → Total Cost

This aligns with the previously developed Capability–Compute–Quality–Cost framework. Cost is not the primary optimization objective. Quality, success probability, and risk requirements are constraints. Only after those constraints are satisfied should the system search for the lowest-cost execution path.

Forge Practice, Layer One: Turn “I Want to Write Code” into a Manageable Engineering Task

If a system does not understand what task it is executing, it cannot make a rational decision about model selection or compute allocation.

This is why Forge is not designed as a simple coding chatbot.

During the requirements phase, users can provide engineering information through conversation, project documents, and structured forms. The workflow then progresses through requirement confirmation, runtime testing and validation, risk detection, and security review.

A project knowledge base can also mount multiple source-code or document directories and incorporate external technical reference URLs, turning project context into a persistent engineering asset rather than something that must be repeatedly copied into prompts.

The first problem this architecture solves is task boundary definition.

The phrase “fix a bug,” for example, could refer to any of the following:

• changing a null-value check;
• resolving a cross-module state synchronization issue;
• diagnosing a distributed-consistency problem;
• or handling a production incident in a payment system.

These tasks clearly require different levels of model intelligence, context scope, security control, and verification intensity.

A production-oriented software engineering agent therefore needs to determine, at minimum:

• What type of task is this?
• Which code and business knowledge does it involve?
• What is the potential impact?
• Does the task introduce security risk?
• How can the result be verified?

Only after these questions have been answered does model routing become meaningful from an engineering perspective.

This is why we believe enterprises ultimately need task-level intelligent compute orchestration, rather than forcing engineers to repeatedly guess which model to choose from a model list.

Forge Practice, Layer Two: Context Control Matters More Than Simple Token Compression

Databricks’ benchmark already demonstrates that the agent harness can have a major effect on cost, and one of the most important sources of that difference is context management.

Forge addresses this by separating persistent project knowledge from the context needed for the current task, then retrieving relevant information through a dynamic context builder.

Forge currently supports project context from documents, execution logs, and local or remote knowledge retrieval. Its local retrieval layer combines exact-text search, filename search, and vector-based semantic retrieval. When building the working context for a task, Forge can progressively use textual matching, file-location matching, and semantic retrieval rather than sending the entire project knowledge base to the model at once.

The layered retrieval design follows the sequence:

Text Search → Filename Search → Vector Retrieval

The first two layers do not require complex prebuilt indexes. Semantic retrieval is used only when lower-cost retrieval methods are insufficient to obtain useful context.

This reflects an important principle of AI coding economics:

More context is not necessarily better. The system should provide only the information that is genuinely required for the current reasoning process.

Suppose an engineer submits a task containing only 200 Tokens. If the agent then resends 100,000 Tokens of project history during every turn, the dominant cost is no longer the user’s request. It is the context amplification created by the agent harness.

Enterprises should therefore monitor a key metric:

Context Amplification Factor = Cumulative Input Tokens ÷ Original Task Tokens

However, this metric should not be governed by a universal hard threshold.

A cross-repository architecture migration may legitimately create context amplification of several hundred times. A simple configuration-file edit that causes the agent to read an entire repository is far more likely to represent an optimization opportunity.

Forge Practice, Layer Three: Model Configuration Must Become Part of the Project Runtime

Software projects differ in language, framework, domain knowledge, risk profile, and deployment environment. Model strategy therefore cannot exist only as an individual developer preference.

Forge currently adopts a layered configuration model consisting of global defaults, project-level overrides, and system fallback settings. Models, vector engines, remote-context settings, and runtime policies can all be configured at the project level. Project configuration takes precedence over global configuration, while missing project-level settings fall back to shared defaults.

The significance of this architecture goes well beyond conveniently storing API keys.

It provides the control foundation required for project-level model policy.

An enterprise might, for example, define rules such as:

• ordinary internal tooling projects may use cost-efficient coding models;
• core transaction systems default to higher-capability models;
• security-sensitive tasks require larger reasoning budgets and stricter verification;
• deterministic formatting changes and boilerplate generation should preferentially use lower-cost models.

Forge’s current configuration layer can already bind model and runtime parameters to individual projects. The next step—capability-aware routing—should be built on enterprise-specific evaluation data rather than vendor rankings alone.

Databricks provides a useful reference here. Instead of directly using public benchmarks to decide which models should be adopted internally, it built an internal evaluation set from historical pull requests. It reconstructed the underlying engineering intent as tasks and used the original test suites to determine whether agents had actually completed the work.

For final validation, it did not rely solely on another LLM as a judge. Instead, hidden tests were restored and executed directly, reducing the risk of mistaking an answer that merely “looks correct” for an implementation that actually works.

Verification Infrastructure, Not Routing Algorithms, Is What Truly Enables Lower-Cost Models

In practice, model routing is often overestimated, while verification systems are underestimated.

Suppose a lower-cost model costs only 20% as much as a premium model and can complete most routine tasks. In theory, the enterprise should clearly prefer the cheaper model.

But the strategy of “use a cheaper model first and escalate when it fails” works only if the system can reliably determine whether the task has actually been completed.

Software engineering has a major advantage over many other forms of knowledge work: many outputs can be checked by external verifiers.

• Compilers can validate syntax and dependencies.
• Type systems can identify interface errors.
• Unit tests can verify deterministic behavior.
• Integration tests can validate interactions between modules.
• Static analysis can detect code-quality problems.
• Security scanners can identify vulnerable dependencies and unsafe code patterns.

Forge has progressively moved code-quality requirements into its engineering templates, including ESLint, Prettier, TypeScript configuration, and source-map commit controls, so that newly generated projects can inherit these baseline quality constraints.

This demonstrates an important point:

AI coding cost optimization is, to a large extent, a verification-engineering problem.

The stronger an enterprise’s verification capability, the more confidently it can assign routine work to lower-cost models.

The weaker its verification capability, the more it must rely on expensive frontier models and extensive human review.

The execution chain enterprises should build is therefore not:

Task → Model → Code

but:

Task → Model → Execution → Automated Verification → Repair or Escalation → Re-verification → Human Acceptance

Forge’s Sandbox and Security Controls Govern What a Model Is Allowed to Do, Not Merely What It Knows How to Do

As coding agents evolve from code generators into systems that execute commands, modify files, install dependencies, and run tests, another cost and governance variable emerges: execution permission.

Forge’s current stable sandbox design follows a progressive model. Simple tasks can run directly in the host environment, while tasks requiring isolation can use a local Docker Compose sandbox. The architecture also retains room for additional execution environments.

The stable implementation emphasizes several principles:

• configuration is decoupled from the concrete sandbox implementation;
• sandboxing is not forced by default;
• compatibility with existing interfaces is preserved.

The governing principle is again similar to model-cost optimization:

Not every task requires the highest level of isolation.

Launching a full container environment for a simple static-analysis task introduces unnecessary compute and latency.

On the other hand, allowing an agent to install dependencies, execute untrusted code, operate a browser, or modify an unfamiliar repository without isolation creates obvious security risk.

Future software engineering agent orchestration therefore needs to allocate two resources simultaneously:

how much intelligence to use, and how strong an execution environment to provide.

Model capability, reasoning budget, and sandbox level together form the execution configuration of a task.

Forge’s Own Engineering Reviews Show Why AI-Generated Results Must Remain Subject to Evidence-Based Verification

One of the most dangerous assumptions in AI coding is to equate “the agent completed the implementation” with “the engineering capability is complete.”

Forge’s own project-scaffolding review demonstrates why independent verification remains necessary.

At the design level, Forge had already defined multiple project types, including web, desktop, mobile, API, command-line, library, bot automation, plugin, and data-pipeline projects.

However, engineering review found gaps between the project types defined in the product layer and the actual scaffold support implemented in the backend. Additional specialized templates and end-to-end tests were therefore still required.

This example demonstrates the true boundary of software engineering agents more clearly than any abstract framework.

Models can generate large amounts of code very quickly, but only:

• code review,
• runtime verification,
• project-specific testing,
• security checks,
• and regression testing

can establish that a capability genuinely exists.

The objective of Forge is therefore not to maximize how much code AI can generate.

Its objective is to ensure that AI-generated engineering changes continuously enter a delivery process with evidence, verification, and rollback capability.

Long-Lived Projects Need to Preserve More Than Code: They Need Agent Execution History

As agent involvement expands from minutes to hours, weeks, or even an entire project lifecycle, another issue emerges: the project itself continues to evolve.

Forge’s Works project environment maintains version information, migration records, and version-difference analysis. Project-structure evolution is handled through startup checks, project-open checks, and incremental migration. The migration process also includes backup, failure rollback, and logging so that older projects can continue operating as Forge itself evolves.

From an AI engineering perspective, this means a long-lived project must preserve more than its Git repository.

At minimum, it should retain:

• project requirements;
• model and runtime configuration;
• knowledge bases;
• agent instructions;
• execution logs;
• test and security results;
• model-call and tool-use traces;
• project-version information.

Together, these artifacts form an auditable software engineering workspace.

Databricks’ Unity AI Gateway is moving in a comparable direction. It places model and coding-agent traffic under a unified control plane, governing model and MCP access, cost controls, and usage records while providing visibility into model, identity, and tool behavior.

From Forge Practice to a Broader Conclusion: Enterprises Need a Task-Level Intelligent Compute Orchestrator

Once these capabilities are combined, AI coding cost governance stops being an isolated finance function and becomes part of the software engineering runtime itself.

An ideal task first undergoes classification according to:

Task Type + Engineering Complexity + Risk Level + Verifiability

The system then selects candidate models from the enterprise’s own capability baseline and determines:

• model tier;
• reasoning budget;
• context budget;
• tool scope;
• sandbox level;
• maximum retry count;
• verification rules.

A simple configuration-file change, for example, might use a low-cost model, current-directory context, no sandbox or a lightweight environment, and one deterministic verification step.

A cross-service performance incident may instead require a high-capability reasoning model, repository-level dynamic context, runtime logs, a containerized environment, multiple test cycles, and final human approval.

The broader framework divides this routing process into three stages:

1. capability routing;
2. compute-budget allocation;
3. escalation after failure or increased task complexity.

Routine tasks can begin with lower-cost models and move upward only when verification fails or execution reveals that the problem is more complex than initially estimated.

This is more important than a simplistic policy such as “use cheap models for L1 and expensive models for L5,” because the true complexity of an engineering task often becomes apparent only during execution.

What Enterprises Should Monitor Is How Much Compute Was Required per Successful Task

If enterprises adopt this model, their cost dashboards must also change.

Traditional dashboards usually report model spend, Token consumption, and departmental usage rankings. These metrics remain useful, but they cannot determine whether spending is actually wasteful.

More meaningful indicators include:

First-Pass Completion Rate
The proportion of tasks that pass verification after the initial execution.

Final Completion Rate
The proportion of tasks eventually completed after repair and model escalation.

Cost per Verified Task
Total model, tool, sandbox, and retry cost divided by the number of verified successful tasks.

Human Review Time
Used to determine whether apparently cheap models simply transfer cost to engineers.

Model Escalation Rate
The proportion of tasks that begin with a lower-capability model but eventually require a stronger one.

Retry Rate
Useful for identifying insufficient model capability, incorrect context, or dysfunctional agent loops.

Context Amplification Factor
Used to determine whether agents repeatedly ingest unnecessary information.

Frontier-Model Utilization
Used to detect whether simple workloads are disproportionately consuming the most expensive models.

False Acceptance Rate
The proportion of incorrect implementations that the agent or verification system mistakenly classifies as successful.

Another useful metric is routing regret.

If the production system uses a configuration costing RMB 10, while internal evaluation has shown that a RMB 3 configuration can satisfy the same quality requirement for the same class of tasks, then the RMB 7 difference represents the cost that should actually be optimized.

A mature cost-governance framework therefore records task-level traces covering Tokens, tools, tests, retries, escalations, and final acceptance results.

Enterprises Do Not Need to Build a Complex Routing Platform on Day One

For enterprises beginning to scale AI coding, the right path is not to immediately build an elaborate intelligent routing platform.

The first phase is simply observation.

Engineers can continue using existing workflows while the organization records task type, model selection, Tokens, context, tool calls, test results, and whether the output was ultimately accepted.

The second phase is to establish an internal coding evaluation set.

The enterprise can select several hundred representative tasks from recent pull requests, defects, and development work, run them across different models and reasoning budgets, and validate results using the original tests.

The third phase introduces recommendation-based routing.

The system may begin telling engineers that “historically, a mid-tier model has been sufficient for this class of task,” while still allowing manual overrides.

Only after the organization has accumulated sufficient operational evidence should the fourth phase introduce automatic routing and dynamic escalation.

This is also one of the most important lessons from Databricks’ internal benchmark. Rather than searching for a permanently “best coding model,” Databricks continuously evaluates different models and agent harnesses against its own code and task distribution and uses those results to identify the current cost-quality frontier.

Conclusion: AI Coding Cost Governance Ultimately Means Allocating Exactly Enough Intelligence to Each Task

From Databricks’ real-world coding benchmark to Forge’s engineering practice, a clear trend is emerging: AI coding is moving from an individual developer productivity tool into the enterprise software production system.

At that stage, the most important questions are no longer:

Which model has the cheapest Token price?

Which model ranks first on a public benchmark?

How many Tokens did the engineering organization consume this month?

The more important questions become:

What capability does this task actually require?

How much relevant context and inference compute are necessary?

How can the system prove that the implementation actually satisfies the requirement?

If the task fails, should the agent continue reasoning, retrieve more context, or escalate to a stronger model?

What is the total cost of obtaining a result that is genuinely ready for production?

Forge’s current work around project knowledge, dynamic context, project-level model configuration, quality rules, sandboxing, testing and validation, risk review, and persistent project runtime is fundamentally building the engineering foundation required to answer these questions.

The objective is not to make AI generate as much code as possible.

It is to make model capability part of a software engineering delivery system that is:

configurable, verifiable, auditable, and continuously improvable.

Enterprise AI coding cost optimization should therefore not be interpreted as simply “spending fewer Tokens.”

A more accurate definition is:

Given established quality, security, and delivery requirements, allocate exactly enough intelligence, context, tools, and verification resources to each software engineering task, and continuously use real operational data to identify the lowest-cost configuration that still satisfies the required standard.

Once an enterprise can do this consistently, model cost stops being an unpredictable AI expense and becomes a measurable, configurable, and continuously optimizable engineering production resource.

Related Topic