Powering Innovation: ZoomInfo's Developer-Centric Productivity Transformation

At ZoomInfo, we’ve recognized that maintaining our durable competitive advantage in the GTM space extends beyond just having the most accurate data. A crucial edge lies in our ability to rapidly iterate and create valuable services and software for our customers. This agility stems from a culture that places developer experience at the forefront of our priorities.

We understand that encouraging our engineers to experiment with building features that delight our customers requires reducing friction and mental load in the software development cycles. Building software is complex enough without additional organizational overhead. That’s why we’re transforming ZoomInfo Engineering by establishing a supportive dedicated group chartered with changing the way we deliver software; fostering an environment where decoupled product-empowered teams can thrive and iterate quickly to create value.

This is the story of how we’re revolutionizing our approach to software delivery, putting developer satisfaction and productivity at the center of our strategy. By focusing on these crucial elements, we’re not just improving our processes and toolchain – we’re accelerating our ability to deliver innovative solutions that keep ZoomInfo at the cutting edge of the GTM space.

Redefining Developer Productivity in the Modern Era

When approaching the productivity paradigm discussion in the software engineering space, we recognize that developer productivity encompasses much more than just lines of code or frequency of commits. We choose to see developer productivity as the ability of our engineering teams to efficiently deliver high-quality software that drives business value while maintaining a positive and sustainable work environment. This holistic view encompasses not only technical output, but also factors like job satisfaction, collaboration effectiveness, and overall well-being of our engineers.

Our approach is rooted in the understanding that what is not measured cannot be improved. As management expert Peter Drucker famously stated, “You can’t manage what you don’t measure.” This principle has guided our efforts to create a comprehensive framework for assessing and enhancing developer productivity. However, we also recognize the inherent complexities in measuring creative knowledge work like software development. We’re acutely aware that not every metric that can be measured should be used as a key performance indicator (KPI), as this can lead to unintended consequences and behavioral changes that may ultimately hinder rather than help productivity.

Charting a Course for Engineering Excellence

To drive our transformation, we’ve established an internal cross-engineering multidisciplinary Productivity group that brings together experts from software, quality, platform, and DevOps engineering. This team is charged with the mission of elevating our internal engineering productivity across the board. By leveraging diverse perspectives and expertise, we ensure that our initiatives address the multifaceted nature of developer productivity and experience.

Our approach to measuring and improving developer productivity is both nuanced and multidimensional. We understand that solely relying on quantitative metrics from our toolchain—like the number of pull requests, DORA metrics, deployment frequency, failure rates, and others—only gives us part of the picture. While these metrics provide valuable insights into our technical processes, they don’t capture everything that contributes to a flourishing engineering culture.

To fill this gap, we’ve created a robust system to measure qualitative aspects of developer sentiment and satisfaction. Central to this system is our Quarterly Engineering Productivity Pulse survey, which gives us a comprehensive view of our engineers’ experiences in several areas, such as toolchain satisfaction, work environment, team dynamics, and organizational processes.

The Power of Developer Sentiment

We’ve come to understand that developer satisfaction is not just a “nice-to-have”, it is a critical factor in driving both developer experience and productivity. Research has consistently shown that engaged, satisfied developers are more productive, innovative, and likely to stay with the company long-term. By prioritizing developer sentiment front left and center, we’re not only improving individual experiences but also fostering a culture of excellence that propels our entire organization forward.

Our measurement system is designed to capture both quantitative and qualitative data to provide a holistic view of our engineering ecosystem. On the quantitative side, we track a variety of toolchain metrics including pull request volume, DORA metrics (Deployment Frequency, Lead Time for Changes, Change Failure Rate, and Time to Restore Service), deployment success rates, and more. These metrics give us valuable insights into the efficiency and reliability of our technical processes.

However, we recognize that these numbers alone don’t tell the whole story. That’s why we’ve implemented a comprehensive quarterly pulse survey to capture qualitative feedback from our engineers. This survey is carefully crafted to touch on multiple factors that scientific research has shown to be key drivers of developer experience and satisfaction. We explore factors such as toolchain satisfaction, team dynamics, and processes satisfaction, that enhance or detract flow, affect their mental load, and their ability to easily produce quality software and services.

Measuring What Matters: The “DevSat” Score

At the heart of our qualitative measurement system is what we call the “DevSat” score. This overarching metric is derived from three key sentiment questions that assess:

1.Perceived Quality of Tools and Processes aka Ease of delivery

2. Day-to-day Perceived Productivity

3. Tools and Processes General Satisfaction

*Figure 1: Our DevSat scale. The score is measured with a scale of 1-5 where higher is better.*

By distilling these critical aspects into a single score, we can track our progress over time and identify areas that need attention. The DevSat score serves as a north star for our productivity initiatives, ensuring that we’re always focused on improving the aspects of work that matter most to our engineers.

A Survey Built on Science and Adaptability

Our quarterly survey is not just another corporate questionnaire – it’s a carefully designed tool based on cutting-edge research in the developer productivity and experience space. We’ve drawn heavily from recent studies, particularly the work published in “An Actionable Framework for Understanding and Improving Developer Experience”, which outlines a comprehensive framework for understanding and measuring developer productivity.

But we don’t just rely on existing research – we’re constantly evolving our approach. We treat our survey process as an iterative experiment, embracing lean and rapid experimentation principles. After each survey cycle, we gather feedback from engineers, leadership, and other stakeholders to refine and enhance our questions and methodology. This agile approach ensures that our survey remains relevant and effective as our organization and the broader tech landscape evolve.

*Figure 2: (survey question) Technical Debt Deep Dive, Types:*
When respondents express negative sentiments towards technical debt, we prompt them to categorize the types of technical debt they most frequently encounter. This approach provides leaders with a comprehensive view of potential technical debt across key areas, enabling them to prioritize improvements and enhance overall satisfaction. The aggregated data empowers decision-makers to strategically address technical debt issues, ultimately fostering a more efficient and resilient development environment.

Inclusive by Design

Furthermore, we also acknowledge the diverse nature of our engineering organization. We’re not just building web applications – we have teams working on data science, mobile development, infrastructure, and more. Each of these disciplines has its unique toolchains, pipelines, and processes. Our survey is designed to be inclusive of all these different engineering populations, with adaptive question flows that ensure relevance for each respondent.

This inclusivity extends to the name of our initiative as well. We deliberately call it the “Quarterly Engineering Productivity Survey” rather than limiting it to just software developers. This broader framing reflects our commitment to improving productivity and satisfaction across our entire engineering organization.

Figure 3 (survey question): Illustrates an adaptive question flow. Based on your response, you are directed to a set of questions relevant to your specific discipline. For example, if you identify as a front-end developer, you will be routed to questions pertaining to our UI components platform. This tailored approach ensures that each user receives the most appropriate and valuable questions relevant to his role, and enables us to survey all populations at ZoomInfo Engineering where every population receives sections that are adapted to the discipline they are serving.

Transparency and Empowerment

We believe that true transformation can only happen in an environment of openness and trust. That’s why we’ve committed to full transparency with our survey results. All raw data, reports, and analyses are published on our internal wiki, allowing anyone in the organization to dive into the findings and draw their insights.

But we don’t stop at just sharing data. We’ve also developed internal dashboards that allow engineering leaders to explore the survey results over time, enabling them to identify trends and areas for improvement within their teams. This empowers our leaders to take ownership of their team’s experience and drive positive change from the ground up.

Figure 4 (Reporting Results Dynamic Dashboard): Our internal dashboard is not only open and transparent to all engineers for viewing the latest survey results, but it also empowers our leadership to slice and dice the data, see comparative metrics, and read verbatim feedback from their teams. This capability enables them to take action to improve their team’s overall well-being and satisfaction.

Balancing Confidentiality and Psychological Safety

While we’re committed to transparency, we also recognize the importance of maintaining psychological safety for our survey respondents. We’ve implemented strict data processing and presentation policies to ensure that individual responses remain confidential. This approach allows our engineers to provide honest, actionable feedback without fear of repercussions.

From Insight to Action

Collecting data is only the first step. What truly sets our approach apart is how we act on the insights we gain. After each survey cycle, we identify key areas with the greatest potential for positive impact. We then hold “Feedback Jam” sessions, where we invite the entire engineering organization to meet with the teams responsible for driving improvements.

During these sessions, the subject matter team presents its planned initiatives based on the survey feedback and opens the floor for discussion and more input. This collaborative approach not only helps us refine our action plans but also fosters a sense of ownership and engagement across the organization. We’ve found that this practice builds empathy, improves communication, and ultimately leads to more effective solutions.

Figure 5 (Screenshot from a Feedback Jam session.) These session are initiated from the survey insights and analysis, open for everyone to participate and are conducted within a scope that has potential for improvement. We provide the subject matter team and leadership with a platform to interact with engineers, promote connectedness, and offer more context regarding what can be improved. The team also presents the trajectory plan and receives feedback from the engineers. Every session results in a published action plan to address the points raised by the engineers.

Measurable Impact and Continuous Improvement

Our focused productivity initiatives have already led to significant improvements in both quantitative metrics and developer satisfaction. We’ve seen increases in deployment frequency, reductions in lead time for changes, and improvements in our DevSat score. These gains demonstrate the power of a dedicated focus on productivity engineering to drive meaningful improvements in both technical output and team satisfaction.

But we’re not resting on our laurels. The quarterly plan for our productivity group is directly shaped by the feedback we receive from our engineers. This ensures that we’re always working on the issues that matter most to our teams and continuously evolving our approach to meet their changing needs.

Closing the Feedback Loop: Transactional Surveys and Flow Engineering

While our quarterly pulse survey provides invaluable insights, we emphasize the need for more immediate feedback mechanisms. Enter “Flow Engineering” — the approach to optimizing the workflows that drive value creation. By conducting detailed value stream mapping and implementing transactional surveys at key points in developer workflows, we’re closing shorter feedback loops and gaining real-time insights into the developer experience, enabling the responsible teams to connect to the produced experience.

Our first area of this approach targets our CI/CD pipelines. We understand that the success or failure of a pipeline run doesn’t tell the whole story of developer satisfaction. For example; A failed pipeline run that quickly helps the developer to pinpoint the root cause for a quick iteration can be equally satisfying as a successful run. That’s why we’re rolling out a system to gauge satisfaction around journeys, providing immediate feedback to the teams responsible for making improvements.

Harnessing Digital Adoption Platforms for Deeper Insights

To further enhance our understanding of developer satisfaction, we’ve started experimenting with Digital Adoption Platforms (DAPs) to not only train developers for features that can enhance their lives, it is also means to measure satisfaction for specific tool journeys, and relay that feedback to the internal team responsible for the tool; Providing contextualized insights into how developers interact with our systems. By implementing DAPs, we’re not only improving the onboarding experience for new team members but also continuously optimizing our toolchain for seasoned developers who leave feedback that drives improvements.

The Power of Short Feedback Loops

At the core of our approach is the belief in the power of short feedback loops. By enabling our engineering teams to connect directly with their internal customers—fellow engineers—we’re fostering a culture of rapid iteration, continuous improvement, and empathy. This approach allows us to identify and address pain points quickly, ensuring that our development environment evolves in lockstep with the needs of our engineers.

Conclusion: Building a Culture of Excellence

At ZoomInfo Engineering, we’re committed to creating an environment where top engineering talent can thrive. Our comprehensive approach to measuring and improving developer productivity and satisfaction is just one example of how we’re working to become a top-performing software company. We believe that this commitment to developer experience and productivity not only makes ZoomInfo Engineering a great place to work but also drives our ability to deliver innovative solutions to our customers. As we continue on this journey of transformation, we’re excited to welcome new talent who share our passion for engineering excellence and want to be part of a team that’s truly pushing the boundaries of what’s possible in software development.

Bibliography

Greiler, Margaret-Anne, Margaret Storey, Abi Noda, Christian Bird, Eirini Kalliamvakou, and Nicole Forsgren. “An Actionable Framework for Understanding and Improving Developer Experience.” arXiv, May 12, 2022. https://arxiv.org/abs/2205.06352

Noda, Abi. ”A New Approach To Measuring Developer Productivity.” Engineering Enablement (blog), May 19, 2023. https://newsletter.getdx.com/p/measuring-developer-productivity

DX. “12 Developer Productivity Metrics You Need to Measure.” Accessed May 8, 2024. https://getdx.com/blog/developer-productivity-metrics/

Haney, Ryan. “Can Developer Productivity Be Measured?” Stack Overflow Blog, December 7, 2020. https://stackoverflow.blog/2020/12/07/measuring-developer-productivity/

GitHub. “Developer Experience: What Is It and Why Should You Care?” June 8, 2023. https://github.blog/enterprise-software/collaboration/developer-experience-what-is-it-and-why-should-you-care/

Noda, Abi. “How Emotions Affect Perceived Productivity.” Engineering Enablement (blog), June 30, 2023. https://newsletter.getdx.com/p/emotions-and-developer-productivity

DX. “How to Measure and Maximize Developer Productivity.” June 13, 2024. https://getdx.com/blog/developer-productivity/

Pluralsight. “How To Measure and Strengthen Developer Productivity.” July 16, 2024. https://www.pluralsight.com/resources/blog/business-and-leadership/developer-productivity

Szilagyi, Gergely. “Measuring Developer Productivity? A Response to McKinsey.” The Pragmatic Engineer (blog), August 29, 2023. https://newsletter.pragmaticengineer.com/p/measuring-developer-productivity

Szilagyi, Gergely. “Measuring Developer Productivity: Real-World Examples.” The Pragmatic Engineer (blog), January 16, 2024. https://newsletter.pragmaticengineer.com/p/measuring-developer-productivity-bae

Fowler, Martin. “Measuring Developer Productivity via Humans.” Martin Fowler (blog), March 19, 2024. https://martinfowler.com/articles/measuring-developer-productivity-humans.html

Atlassian. “Why Developer Experience Is More Important Than Productivity.” February 6, 2024. https://www.atlassian.com/blog/devops/developer-experience-more-important

McKinsey & Company. “Yes, You Can Measure Software Developer Productivity.” August 17, 2023. https://www.mckinsey.com/industries/technology-media-and-telecommunications/our-insights/yes-you-can-measure-software-developer-productivity

IT Revolution. “Flow Engineering.” June 7, 2024. https://itrevolution.com/product/flow-engineering/

“Flow Engineering: From Value Stream Mapping to Effective Action.” Flow Engineering, https://flowengineering.org/

Related blog:
How Zoominfo is making a Quantum Leap in Infrastructure … (2024). https://engineering.zoominfo.com/how-zoominfo-is-making-a-quantum-leap-in-infrastructure-operations-by-employing-an-internal-developer-platform

Powering Innovation: ZoomInfo’s Developer-Centric Productivity Transformation