--- title: "A Guide to Data-Driven Software Engineering Transformation" description: "Only 5% of companies see ROI from their AI software engineering transformations. Success starts with the right data — before the initiative and during it." type: blog version: 3 version_id: "55dab003-ded5-4030-afa4-9ba1db215f45" generated_at: "2026-06-26T05:58:16.660Z" author: "Lauren Lang" date_published: "2024-06-11T16:51:52.000Z" date_modified: "2026-06-20T00:57:15.635Z" language: en reading_time: "20 min" word_count: 3804 keywords: ["A Guide to Data", "Driven Software Engineering Transformation", "Stay up to date"] url: "https://uplevelteam.com/blog/digital-transformation-engineering" --- # A Guide to Data-Driven Software Engineering Transformation > Only 5% of companies see ROI from their AI software engineering transformations. Success starts with the right data — before the initiative and during it. ## Key Takeaways - Why is engineering transformation so hard to measure? - What should you measure during a software engineering transformation? - Which engineering metrics actually drive transformation progress? - How do engineering leaders get the visibility required? - What identifying the real bottlenecks looks like in practice ## Contents - [Why is engineering transformation so hard to measure?](#why-is-engineering-transformation-so-hard-to-measure) - [What should you measure during a software engineering transformation?](#what-should-you-measure-during-a-software-engineering-transformation) - [Which engineering metrics actually drive transformation progress?](#which-engineering-metrics-actually-drive-transformation-progress) - [How do engineering leaders get the visibility required?](#how-do-engineering-leaders-get-the-visibility-required) - [What identifying the real bottlenecks looks like in practice](#what-identifying-the-real-bottlenecks-looks-like-in-practice) - [FAQs](#faqs) [Back to Resources](https://uplevelteam.com/resources) [Digital Transformation](https://uplevelteam.com/blog/tag/digital-transformation) Jun 20, 2026 # Software Engineering Transformation Starts with Data Only 5% of companies see ROI from their AI software engineering transformations. Success starts with the right data — before the initiative and during it. * * * In 2024, BCG found that [two-thirds of large-scale tech programs](https://www.bcg.com/publications/2024/most-large-scale-tech-programs-fail-how-to-succeed) miss their targets on time, budget, and scope — and only 30% fully achieve their objectives. A more recent study found that [95% of enterprise companies are seeing zero return on investment](https://mlq.ai/media/quarterly_decks/v0.1_State_of_AI_in_Business_2025_Report.pdf) from AI initiatives. Volkswagen's software division Cariad [illustrates this pattern](https://insideevs.com/news/753673/vw-group-cariad-billions-losses-2024/) at scale. Founded in 2020 to build a unified operating system across all VW Group brands, it accumulated over $7.5 billion in operating losses between 2022 and 2024. Insiders reported 17 status meetings per week. Developers made slide decks instead of shipping code. After approximately €14 billion invested, Volkswagen turned to Rivian for the software platform it couldn't build internally. Many organizations struggle to meet targets and achieve sustainable change. After a series of failed initiatives, wasted resources, and missed opportunities, teams revert to familiar practices. Organizational inertia drags transformation out for years instead of months.  **The organizations that do succeed are the ones that choose the right priorities — and can measure progress against them. If "what gets measured gets managed," measuring software engineering transformation can turn a multi-year quagmire into an achievable roadmap.**  The problem: what to measure is different for every company. ## Why is engineering transformation so hard to measure? Software engineering transformation resists standardized metrics because context determines what success looks like, and no two organizations share the same starting point. The idea of measuring progress with a universal framework is a little like measuring health by weight or BMI. (The Rock weighs 260 pounds and his BMI would categorize him as "morbidly obese," which we know is [not the truth](https://www.instagram.com/p/C60WXqtuZwx/?img_index=1).) It paints an incomplete or inaccurate picture in a situation where context matters. "Digital transformation means different things to different people," says Amy Carrillo Cotten, Uplevel’s Director of Client Transformation. AI tools add a specific new layer to this problem. Knowing how many coding licenses are deployed tells you nothing about whether developers are producing better outcomes — the gap between [AI adoption and AI impact](https://uplevelteam.com/blog/ai-capabilities-maturity) is one of the most underexamined constraints in transformation work today. Here’s what makes measuring success particularly difficult: - **One size metrics do not fit all.** What success looks like is unique to your organization and depends on a number of factors like core competencies, architectures, components, and teams involved. Traditional organizations will have different measures of success than digitally native tech companies as they often have more inertia to overcome. - Sustainable transformation goes beyond technology. It's a systemic, cultural shift in [ways of working](https://uplevelteam.com/blog/ways-of-working-engineering) and adopting [modern development practices](https://resources.uplevelteam.com/10x) that support cross-functional collaboration, transparency, agility, and value delivery. Many companies underestimate the investment these changes require and struggle to measure it. - **Enterprise systems are complex.** Companies often underestimate the dependencies of their technical systems, hiring [multiple consultants and vendors](/blog/software-engineering-transformation-consultants) to advise and execute. These siloed efforts create inefficiencies that work against the goals of the transformation and harm visibility of progress as a whole. - **Transformation is never (really) done.** The purpose of transformation is to be agile and responsive to market needs — to continually optimize people, systems, and processes. Defining and measuring success in that context requires a different approach than a project checklist. ## What should you measure during a software engineering transformation? Engineering transformation success can be measured. The key is to establish measurement practices before the initiative is underway — not after it stalls. If "what gets measured gets managed," modern development practices are often left behind when measurement gets treated as a phase-two problem. Transformation becomes a list of projects to complete rather than a state change to track, and the longitudinal data that would reveal real progress never gets collected. Starting to measure development practices before a transformation begins makes the transformation more likely to succeed — and [how that sequencing plays out](https://uplevelteam.com/blog/software-engineering-process-transformation) determines whether what follows sticks. Having the right engineering metrics helps you identify areas for improvement and monitor whether those practices are actually being adopted. **Digital transformation is more about a state change than completing a list of projects, and the longitudinal data can help measure that progress.** The sooner you can establish practices that ensure developers have enough time to do their work and are [working on the right things](https://uplevelteam.com/blog/time-allocation-strategies), the sooner you can [identify the major bottlenecks](https://uplevelteam.com/blog/engineering-bottlenecks) that are the biggest impediments to progress. Those bottlenecks are likely going to be a core component of your transformation roadmap. It's always best to build inspection metrics and incentives into the culture as early as possible. It's never too late to start, but the earlier you start the easier it is to form and grow with the company. Matt Swann Former enterprise CTO and strategic advisor ## Which engineering metrics actually drive transformation progress? Digital transformation in engineering often suffers from trying to do too much at once, leading to inefficiency and lack of direction. While there’s no one way to measure success, it begins by identifying the biggest opportunities and prioritizing them. What technical priorities will help meet business objectives?  [Matt Swann](https://www.linkedin.com/in/mattswann/) explains that “for digital businesses, technical priorities should be mapped clearly to key business priorities. It's critical to change the language from technical to business to ensure the business understands and cares about technical needs.” Pre-transformation, technology priorities and business priorities might not have aligned. When the role of technology is supportive and not strategic, OKRs might look like achieving 99.999% server uptime or reducing IT support ticket resolution time — important goals, to be sure, but they don’t move the needle on a P&L. Digital transformation changes that. Priorities now tie much more closely to what the business needs to achieve, and the metrics that become most important are the ones that reveal the sustainable progress of those priorities: Some metrics will be [lagging indicators](https://uplevelteam.com/blog/why-dora-metrics-are-only-part-of-the-equation), but others can be early signals of where progress is happening — or isn't. Understanding actual vs. predicted performance lets engineering leaders measure progress, adjustbenchmarks, budget resources, and give executive teams the "why" behind the numbers. No framework or set of metrics will be perfect. The right metrics reflect the priorities of your transformation and the directionality and velocity of the change you're trying to make. ## **How do engineering leaders get the visibility required?** Visibility into engineering activity is what makes transformation goals trackable. Without it, progress is estimated rather than measured — and estimates don't hold up in executive reviews or budget cycles. Headcount is an engineering organization's largest budget item. Tracking how that investment is actually being spent — in terms of [time allocation](https://uplevelteam.com/blog/time-allocation-strategies), focus, and [sustainable ways of working](https://uplevelteam.com/blog/deep-work-why-we-measure-in-two-hour-minimum-time-blocks) — requires telemetry, the same way DevOps and cloud monitoring platforms provide telemetry for infrastructure. The right [engineering intelligence platform](https://uplevelteam.com/blog/engineering-intelligence-tools-buyers-guide) gives engineering leaders a way to align team activity directly to business outcomes. Combining standard productivity metrics with harder-to-surface indicators — how developers actually spend their time, where review complexity is building, where AI adoption is generating throughput and where it's hitting a ceiling — produces the picture that actually matters for your transformation goals. * * * [](https://uplevelteam.com/blog/measuring-developer-productivity) [ ##### Developer Productivity Metrics That Matter for AI ](https://uplevelteam.com/blog/measuring-developer-productivity) Measuring developer productivity is hard within large engineering organizations. Here's why that is, and why a holistic approach is necessary. * * * ## **What identifying the real bottlenecks looks like in practice** The pattern Uplevel sees across engineering organizations: teams know something is slowing them down, but the metrics they're tracking don't show [where the constraint actually lives](/blog/engineering-bottlenecks). For example, at a distributed hardware company, AI coding licenses had been in place for over a year. Uplevel's GearUp sprint found that shallow enablement and quality gaps were capping how much teams could benefit from the tools already deployed. Test coverage was inconsistent across teams, with no sustainable program to build on. Based on those findings, the organization launched a structured enablement program — featuring dev-to-dev knowledge sharing, agentic PR reviews, AI-assisted test generation, and expanded coding standards. Test automation coverage increased 30–50% across teams, production incidents declined, and PR cycle time dropped 10%. That outcome started with knowing what to measure. * * * #### Find out where your organization stands before committing to a transformation roadmap. [**How do you StackUp? →**](/stackup) ## FAQs ##### What is software engineering transformation? Software engineering transformation is the process by which an engineering organization changes how it operates — adopting modern development practices, aligning technical work with business goals, and building the measurement infrastructure to track progress over time. It encompasses technology, culture, and organizational structure, and it's ongoing rather than project-bound. ##### Why do most software engineering transformations fail? Most engineering transformations fail because organizations focus on the initiative itself rather than the conditions that make it possible. Unclear priorities, misaligned metrics, and cultural resistance are the most common causes. Establishing measurement practices before the transformation begins — not after it stalls — significantly improves outcomes. ##### How do you measure the success of an engineering transformation? There is no single metric for transformation success. The most useful measures connect technical priorities to business objectives: time allocated to new value versus maintenance work, delivery and quality metrics (DORA), and trend data showing whether practices are actually changing. Progress shows up in directionality over time, not point-in-time snapshots. ##### How does AI affect software engineering transformation? AI tools amplify whatever the underlying engineering system is already doing — which means the health of your fundamentals determines whether AI accelerates progress or accelerates dysfunction. [Measuring what AI is actually changing](https://uplevelteam.com/blog/ai-capabilities-maturity), not just whether it's been deployed, requires a different framework than standard adoption metrics. ##### Which engineering metrics matter most during a transformation? The most useful metrics connect engineering activity to business outcomes. At the team level, leading indicators include focus time, work allocation across value-generating vs. maintenance work, and PR cycle time. At the initiative level, DORA metrics — deployment frequency, lead time, change failure rate, and time to restore — track the health of delivery practices. The right set depends on the specific priorities of the transformation. ##### How long does a software engineering transformation take? Meaningful transformation takes time. Targeted quick wins are possible in 90 days — specific bottlenecks can be addressed and early measurement infrastructure can be established. Sustained change, where the organization has built the capability to continuously adapt, typically requires a year or more of consistent work. Table of Contents - Why is engineering transformation so hard to measure? - What should you measure during a software engineering transformation? - Which engineering metrics actually drive transformation progress? - How do engineering leaders get the visibility required? - What identifying the real bottlenecks looks like in practice - FAQs [](https://uplevelteam.com/blog/author/lauren-lang) [ ### Lauren Lang ](https://uplevelteam.com/blog/author/lauren-lang) Lauren Lang is Director of Marketing at Uplevel. With 10+ years of experience in SaaS and AI/ML, she is passionate about helping tech leaders create and sustain healthy, productive teams. [](https://linkedin.com/in/asklaurenlang) ## Skip the demo. Get real answers on how to maximize AI impact. Take our 10-minute StackUp diagnostic first. Get benchmarked insights on your AI trajectory, then talk to us about the results if it makes sense. [](https://uplevelteam.com/stackup) ## Related resources [](https://uplevelteam.com/resources) Digital Transformation #### How to Choose an Engineering Transformation Consultant Roadmaps aren't enough. Effective software engineering transformation consultants combine measurement, qualitative research, and team capability building. AI Engineering #### Tokenomics for the Deeply Skeptical AI budgets are under scrutiny. Ed Quick joins Uplevel for a practical conversation on how to manage token spend, drive value, and promote experimentation. AI Engineering #### What Makes Engineering Transformation Stick Most engineering transformations stall because they skip stages. Here's the five-stage framework for building AI capability that compounds. --- ## Frequently Asked Questions ### Why is engineering transformation so hard to measure? Software engineering transformation resists standardized metrics because context determines what success looks like, and no two organizations share the same starting point. The idea of measuring progress with a universal framework is a little like measuring health by weight or BMI. (The Rock weighs 260 pounds and his BMI would categorize him as "morbidly obese," which we know isnot the truth.) It paints an incomplete or inaccurate picture in a situation where context matters. "Digital transformation means different things to different people," says Amy Carrillo Cotten, Uplevel’s Director of Client Transformation. AI tools add a specific new layer to this problem. Knowing how many coding licenses are deployed tells you nothing about whether developers are producing better outcomes — the gap betweenAI adoption and AI impact is one of the most underexamined constraints in transformation work today. Here’s what makes measuring success particularly difficult: ### What should you measure during a software engineering transformation? Engineering transformation success can be measured. The key is to establish measurement practices before the initiative is underway — not after it stalls. If "what gets measured gets managed," modern development practices are often left behind when measurement gets treated as a phase-two problem. Transformation becomes a list of projects to complete rather than a state change to track, and the longitudinal data that would reveal real progress never gets collected. Starting to measure development practices before a transformation begins makes the transformation more likely to succeed — andhow that sequencing plays out determines whether what follows sticks. Having the right engineering metrics helps you identify areas for improvement and monitor whether those practices are actually being adopted. The sooner you can establish practices that ensure developers have enough time to do their work and areworking on the right things, the sooner you canidentify the major bottlenecks that are the biggest impediments to progress. Those bottlenecks are likely going to be a core component of your transformation roadmap. It's always best to build inspection metrics and incentives into the culture as early as possible. It's never too late to start, but the earlier you start the easier it is to form and grow with the company. Matt Swann Former enterprise CTO and strategic advisor It's always best to build inspection metrics and incentives into the culture as early as possible. It's never too late to start, but the earlier you start the easier it is to form and grow with the company. Matt Swann Former enterprise CTO and strategic advisor It's always best to build inspection metrics and incentives into the culture as early as possible. It's never too late to start, but the earlier you start the easier it is to form and grow with the company. Matt Swann Former enterprise CTO and strategic advisor It's always best to build inspection metrics and incentives into the culture as early as possible. It's never too late to start, but the earlier you start the easier it is to form and grow with the company. Matt Swann Former enterprise CTO and strategic advisor It's always best to build inspection metrics and incentives into the culture as early as possible. It's never too late to start, but the earlier you start the easier it is to form and grow with the company. Matt Swann Former enterprise CTO and strategic advisor It's always best to build inspection metrics and incentives into the culture as early as possible. It's never too late to start, but the earlier you start the easier it is to form and grow with the company. Matt Swann Former enterprise CTO and strategic advisor It's always best to build inspection metrics and incentives into the culture as early as possible. It's never too late to start, but the earlier you start the easier it is to form and grow with the company. Matt Swann Former enterprise CTO and strategic advisor It's always best to build inspection metrics and incentives into the culture as early as possible. It's never too late to start, but the earlier you start the easier it is to form and grow with the company. Matt Swann Former enterprise CTO and strategic advisor It's always best to build inspection metrics and incentives into the culture as early as possible. It's never too late to start, but the earlier you start the easier it is to form and grow with the company. Matt Swann Former enterprise CTO and strategic advisor It's always best to build inspection metrics and incentives into the culture as early as possible. It's never too late to start, but the earlier you start the easier it is to form and grow with the company. It's always best to build inspection metrics and incentives into the culture as early as possible. It's never too late to start, but the earlier you start the easier it is to form and grow with the company. Matt Swann Former enterprise CTO and strategic advisor Former enterprise CTO and strategic advisor ### Which engineering metrics actually drive transformation progress? Digital transformation in engineering often suffers from trying to do too much at once, leading to inefficiency and lack of direction. While there’s no one way to measure success, it begins by identifying the biggest opportunities and prioritizing them. What technical priorities will help meet business objectives? Matt Swann explains that “for digital businesses, technical priorities should be mapped clearly to key business priorities. It's critical to change the language from technical to business to ensure the business understands and cares about technical needs.” Pre-transformation, technology priorities and business priorities might not have aligned. When the role of technology is supportive and not strategic, OKRs might look like achieving 99.999% server uptime or reducing IT support ticket resolution time — important goals, to be sure, but they don’t move the needle on a P&L. Digital transformation changes that. Priorities now tie much more closely to what the business needs to achieve, and the metrics that become most important are the ones that reveal the sustainable progress of those priorities: Some metrics will belagging indicators, but others can be early signals of where progress is happening — or isn't. Understanding actual vs. predicted performance lets engineering leaders measure progress, adjustbenchmarks, budget resources, and give executive teams the "why" behind the numbers. No framework or set of metrics will be perfect. The right metrics reflect the priorities of your transformation and the directionality and velocity of the change you're trying to make. ### How do engineering leaders get the visibility required? Visibility into engineering activity is what makes transformation goals trackable. Without it, progress is estimated rather than measured — and estimates don't hold up in executive reviews or budget cycles. Headcount is an engineering organization's largest budget item. Tracking how that investment is actually being spent — in terms oftime allocation, focus, andsustainable ways of working — requires telemetry, the same way DevOps and cloud monitoring platforms provide telemetry for infrastructure. The rightengineering intelligence platform gives engineering leaders a way to align team activity directly to business outcomes. Combining standard productivity metrics with harder-to-surface indicators — how developers actually spend their time, where review complexity is building, where AI adoption is generating throughput and where it's hitting a ceiling — produces the picture that actually matters for your transformation goals. Developer Productivity Metrics That Matter for AI Measuring developer productivity is hard within large engineering organizations. Here's why that is, and why a holistic approach is necessary. Developer Productivity Metrics That Matter for AI Measuring developer productivity is hard within large engineering organizations. Here's why that is, and why a holistic approach is necessary. Developer Productivity Metrics That Matter for AI Measuring developer productivity is hard within large engineering organizations. Here's why that is, and why a holistic approach is necessary. Developer Productivity Metrics That Matter for AI Measuring developer productivity is hard within large engineering organizations. Here's why that is, and why a holistic approach is necessary. ### What is software engineering transformation? Software engineering transformation is the process by which an engineering organization changes how it operates — adopting modern development practices, aligning technical work with business goals, and building the measurement infrastructure to track progress over time. It encompasses technology, culture, and organizational structure, and it's ongoing rather than project-bound. Software engineering transformation is the process by which an engineering organization changes how it operates — adopting modern development practices, aligning technical work with business goals, and building the measurement infrastructure to track progress over time. It encompasses technology, culture, and organizational structure, and it's ongoing rather than project-bound. Why do most software engineering transformations fail? Most engineering transformations fail because organizations focus on the initiative itself rather than the conditions that make it possible. Unclear priorities, misaligned metrics, and cultural resistance are the most common causes. Establishing measurement practices before the transformation begins — not after it stalls — significantly improves outcomes. ### Why do most software engineering transformations fail? Most engineering transformations fail because organizations focus on the initiative itself rather than the conditions that make it possible. Unclear priorities, misaligned metrics, and cultural resistance are the most common causes. Establishing measurement practices before the transformation begins — not after it stalls — significantly improves outcomes. Most engineering transformations fail because organizations focus on the initiative itself rather than the conditions that make it possible. Unclear priorities, misaligned metrics, and cultural resistance are the most common causes. Establishing measurement practices before the transformation begins — not after it stalls — significantly improves outcomes. How do you measure the success of an engineering transformation? There is no single metric for transformation success. The most useful measures connect technical priorities to business objectives: time allocated to new value versus maintenance work, delivery and quality metrics (DORA), and trend data showing whether practices are actually changing. Progress shows up in directionality over time, not point-in-time snapshots. ### How do you measure the success of an engineering transformation? There is no single metric for transformation success. The most useful measures connect technical priorities to business objectives: time allocated to new value versus maintenance work, delivery and quality metrics (DORA), and trend data showing whether practices are actually changing. Progress shows up in directionality over time, not point-in-time snapshots. There is no single metric for transformation success. The most useful measures connect technical priorities to business objectives: time allocated to new value versus maintenance work, delivery and quality metrics (DORA), and trend data showing whether practices are actually changing. Progress shows up in directionality over time, not point-in-time snapshots. How does AI affect software engineering transformation? AI tools amplify whatever the underlying engineering system is already doing — which means the health of your fundamentals determines whether AI accelerates progress or accelerates dysfunction.Measuring what AI is actually changing, not just whether it's been deployed, requires a different framework than standard adoption metrics. ### How does AI affect software engineering transformation? AI tools amplify whatever the underlying engineering system is already doing — which means the health of your fundamentals determines whether AI accelerates progress or accelerates dysfunction.Measuring what AI is actually changing, not just whether it's been deployed, requires a different framework than standard adoption metrics. AI tools amplify whatever the underlying engineering system is already doing — which means the health of your fundamentals determines whether AI accelerates progress or accelerates dysfunction.Measuring what AI is actually changing, not just whether it's been deployed, requires a different framework than standard adoption metrics. Which engineering metrics matter most during a transformation? The most useful metrics connect engineering activity to business outcomes. At the team level, leading indicators include focus time, work allocation across value-generating vs. maintenance work, and PR cycle time. At the initiative level, DORA metrics — deployment frequency, lead time, change failure rate, and time to restore — track the health of delivery practices. The right set depends on the specific priorities of the transformation. ### Which engineering metrics matter most during a transformation? The most useful metrics connect engineering activity to business outcomes. At the team level, leading indicators include focus time, work allocation across value-generating vs. maintenance work, and PR cycle time. At the initiative level, DORA metrics — deployment frequency, lead time, change failure rate, and time to restore — track the health of delivery practices. The right set depends on the specific priorities of the transformation. The most useful metrics connect engineering activity to business outcomes. At the team level, leading indicators include focus time, work allocation across value-generating vs. maintenance work, and PR cycle time. At the initiative level, DORA metrics — deployment frequency, lead time, change failure rate, and time to restore — track the health of delivery practices. The right set depends on the specific priorities of the transformation. How long does a software engineering transformation take? Meaningful transformation takes time. Targeted quick wins are possible in 90 days — specific bottlenecks can be addressed and early measurement infrastructure can be established. Sustained change, where the organization has built the capability to continuously adapt, typically requires a year or more of consistent work. ### How long does a software engineering transformation take? Meaningful transformation takes time. Targeted quick wins are possible in 90 days — specific bottlenecks can be addressed and early measurement infrastructure can be established. Sustained change, where the organization has built the capability to continuously adapt, typically requires a year or more of consistent work. Meaningful transformation takes time. Targeted quick wins are possible in 90 days — specific bottlenecks can be addressed and early measurement infrastructure can be established. Sustained change, where the organization has built the capability to continuously adapt, typically requires a year or more of consistent work. --- ## About This Content **Source:** [A Guide to Data-Driven Software Engineering Transformation](https://uplevelteam.com/blog/digital-transformation-engineering) **Author:** Lauren Lang **Published:** June 11, 2024 *This content is provided for informational purposes. Please visit the original source for the most up-to-date information.*