AI’s Software Gains Are Harder to Prove, Uplevel Survey Finds

Confidence is ahead of measurement

The results are part of Uplevel’s report, Would You Recognize AI ROI If You Saw It?, which examines how engineering organizations measure AI’s value, attribute improvements and address obstacles to adoption.

Nearly three quarters of respondents, 74%, said they had extended their pre AI return on investment measurement methods to AI initiatives. Only 4% said they were building measurement approaches specifically for AI, while 29% primarily evaluated AI through business outcomes.

Joe Levy, CEO of Uplevel, said confidence does not show how much improvement comes from AI rather than other influences. He said organizations need a clearer picture of where gains originate, what may be limiting them and whether those gains produce meaningful business outcomes.

Faster coding can create more review work

Speed remains the leading measure of AI value for engineering leaders. The report says 39% identified speed as the primary factor they would consider when deciding whether AI creates value. The preference was similar among managers at 41%, directors at 40% and vice presidents at 40%.

Separately, 39% said AI primarily creates value by helping developers write code faster. However, 29% identified additional time spent reviewing AI generated work as their biggest new source of friction during the past quarter. Uplevel said this was a higher share than for any other challenge.

Amy Carrillo Cotten, Uplevel’s director of client transformation, said faster coding does not automatically produce faster or better delivery. She argued that engineering leaders should examine the full delivery process, including where AI accelerated work may create new bottlenecks.

Organizations are spread across five AI maturity levels

Uplevel describes a five point maturity continuum, ranging from consistent use of AI tools for tasks such as code completion to teams of agents that can deploy software independently with minimal human intervention.

Thirty eight percent of leaders placed their organizations in the earlier stages, covering Level 1 and Level 2. These organizations use AI for tasks such as code completion or have begun applying agents in specific, bounded areas with limited organizational context. Another 23% were at Level 3, where agents are configured with information such as codebases, conventions and runbooks, and engineers increasingly review, correct and direct their output.

Thirty eight percent reported more advanced stages. At Level 4, organizations develop, maintain and share AI agents across teams. At Level 5, they orchestrate agent teams that deploy software independently and restructure development workflows around AI.

The obstacles change as adoption advances

Governance, security and compliance were the most commonly cited obstacle to getting more value from AI, at 24%. Gaps in engineering skills or AI literacy and missing context, including documentation, data or institutional knowledge, each accounted for 21%. Workflow friction and infrastructure limitations each represented 17%.

The leading barrier varied by maturity. Missing context was the biggest impediment at Level 1, at 25%, and Level 2, at 27%. At Level 4, skills and AI literacy gaps reached 26%, while governance, security and compliance concerns reached 28%. At Level 5, the figures were 30% for skills and AI literacy and 26% for governance, security and compliance.

The findings suggest that measuring AI only through faster coding may leave organizations unable to identify where value is created or where new review and governance demands are slowing delivery. Uplevel has published the full survey findings on its website.