What’s The ROI Of Software Engineers’ Spending On AI?

TL;DR

Buying for a business?Offer from Amazon

Get business pricing on networking and server gear

• Business-only prices and quantity discounts
• Tax-exempt purchasing
• Multiple users, one account, clear invoices

Create a free Business account →Free to register

As an affiliate, we earn on qualifying purchases.Hide

Data gathered by The Pragmatic Engineer from 27 workplaces and two AI coding platforms points to rising pull request output alongside widely varying AI bills. The report says few companies can yet answer whether the added spending produces a measurable return on investment.

A survey of 27 workplaces by The Pragmatic Engineer found that companies are spending widely different amounts on AI tools for engineers, while many still cannot say whether the outlay delivers a measurable return on investment. Data shared by CodeRabbit showed weekly pull requests (PRs) among its surveyed users rose 113% year over year, but the report does not establish that AI tools alone caused the increase or that more PRs translated into better business results.

The report’s author said an engineer had asked how much companies spend on AI per engineer and how PR output changed before and after adopting the tools. In response, the author interviewed engineering leaders and senior engineers at 27 workplaces and collected aggregate data from CodeRabbit and Gitar. Most company names were withheld at their request, and the findings are a limited sample rather than a representative industry census.

CodeRabbit provided anonymized data on engineers using its AI code review platform. It reported a 113% increase in weekly PRs and a 127% rise in monthly PRs. For a tracked group of 20,000 developers, monthly PRs rose 128% over a year. Separately, Gitar data covering hundreds of customers showed that some teams categorized as “AI-native” ship more than 60 PRs per developer per week. The measures describe output volume; they do not establish how much of that output was generated with AI.

Company spending varied sharply and did not appear to track company size in this sample. A two-person .NET company reported $3,000 per developer per month, while a publicly traded company with 1,500 engineers capped monthly AI spending at $150 per developer. Several organizations use spending limits, though respondents told the author that caps can be bypassed. Some companies also face the end of subsidized subscriptions, which they said could raise bills five- to tenfold if usage and plans stay unchanged.

More Output, Unproven Returns

For engineering leaders, the findings show why tracking tool bills and output counts is not the same as measuring economic return. PR volume can show that more code changes are being submitted, but it does not reveal whether those changes are useful, whether review and maintenance costs rose, or whether products reached customers faster. The report presents higher throughput as a readily available metric, not a complete productivity measure.

Cost exposure also varies with how teams access models. The author reports that API-plan costs peaked for many surveyed companies in June and July, then roughly halved for some as they shifted to open models or changed model use. Subscriptions may make direct per-engineer costs less visible, while subsidies ending could change budgets quickly. Companies deciding whether to expand access therefore face both uncertain benefits and potentially volatile costs.

How the Survey Measured Output

The report focuses on PR throughput because pull requests are recorded in most software development workflows and can be compared over time. The author described the metric as readily available and said it may not yet have been “gamed” by developers. Even so, PR counts are only one indicator: they do not capture code quality, complexity, defects, or the value of each change.

The evidence combines different sources and populations. CodeRabbit supplied anonymized platform data, Gitar shared customer-level categorization, and the spending figures came from interviews with people at 27 workplaces. These sources should not be treated as one controlled experiment. The comparisons do not isolate AI use from other factors such as team size, company culture, workflow changes, or hiring.

““The question of ROI is one that few companies seem able to answer – at least for now.””

— The Pragmatic Engineer report author

What the Numbers Cannot Show

The report does not provide a controlled comparison establishing that AI adoption caused the increases in PR counts. It also does not show whether the additional PRs improved software quality, reduced delivery time, lowered total labor costs, or generated revenue. The author says few companies could answer the ROI question, and the material provided does not supply a common definition or calculation of ROI across the 27 workplaces.

Several figures are based on interviews or platform data whose methods and participant mix differ. The report does not provide enough information to generalize the spending levels to the full software industry. It also describes costs falling for some API users after model and infrastructure changes, but does not quantify comparable savings across all companies. The duration and financial impact of subscription subsidies ending remain unspecified.

Budgets Face New Cost Tests

Companies in the report are weighing spending caps, model choices, and changes to subsidized subscriptions as they set budgets. The author says some organizations are preparing either to optimize costs or absorb potentially much higher bills when subsidies end. The report also describes one high-spending firm building internal GPU and CPU capacity, but the supplied material does not establish when that infrastructure will be operational or how much it will change costs.

The next test for employers will be whether they can connect tool use to outcomes beyond PR counts, while accounting for quality, review work, and total costs. No common measurement or follow-up result is reported yet. Until companies publish comparable evidence, the relationship between AI spending, engineering productivity, and financial returns remains unsettled.

Key Questions

How much are companies spending on AI per engineer?

The 27-workplace survey found a wide range. Examples included $150 per engineer per month at a 1,500-engineer public company and $3,000 per developer per month at a two-person .NET company. These examples are not an industry average.

Did AI cause the increase in pull requests?

The report does not prove causation. CodeRabbit users’ PR counts rose year over year, but the data does not isolate AI use from other changes to teams, tools, or workflows.

Does a higher PR count mean a better return on investment?

Not by itself. PR volume measures submitted changes, but the report does not show whether they improved quality, reached customers sooner, saved money, or increased revenue.

Why could AI bills change for companies?

Spending can depend on model and API usage, internal limits, and subsidized subscriptions. The report says some companies saw API costs fall after changing models, while others expect bills could rise five- to tenfold when subsidies end.

Source: rss