What are DORA metrics?

DORA (DevOps Research and Assessment) metrics were first developed at Google. They analyzed data from more than 32,000 DevOps professionals to create their DevOps report, Accelerate: State of DevOps. The metrics that DORA is comprised of are:

1. Deployment Frequency
2. Lead Time for Changes
3. Mean Time to Recover
4. Change Failure Rate

These metrics are a great way to measure and provide objective data on the effectiveness, efficiency and reliability of your software development and deployment practices and also help you identify areas for improvement for your engineering teams.

With these measures in place, DORA metrics can be leveraged by DevOps and software delivery teams to determine whether their engineering teams are “low performers”, “medium performers”, “high performers” or “elite performers” to create a foundation for software delivery excellence across your organization.

DORA Metrics 101: Are you a low or elite performer?

‍

‍

Deployment Frequency

The deployment frequency metric measures the number of deployments your team makes. Once you have an easy and often used deployment pipeline in place, it has a positive impact on Lead Time for Changes and Mean Time to Recover. Ideally, high-performing companies tend to ship smaller and more frequent deployments.

Lead Time for Changes

The lead time for changes metric measures the amount of time it takes for a commit to be deployed to production.

To measure this, two data points are needed: the time of the commit and the time of deployment. The average time between committing code and deploying is then used as an indicator of your team’s lead time for changes in performance.

If a higher measure of lead time shifts, DevOps teams can streamline processes, and break down products and features into smaller, more manageable code.

Mean Time to Recover

The mean time to recover metric measures the amount of time it takes to restore service to your users after a failure. While you’re deploying frequently, bugs or incidents may occur. While it is not reasonable to eliminate unplanned outages or incidents, the lower the time it takes for your team to respond and restore services makes all the difference in becoming an “elite performer” versus a “low performer."

This is an important metric as it leads to engineering teams creating more resilient systems and processes to respond to any incidents quickly.

Change Failure Rate

The change failure rate metric measures the percentage of changes that fail in production. It’s calculated by the number of deployment failures / total number of deployments. In essence, it measures the reliability of your software development and deployment processes.

DORA metrics challenges and pitfalls

DORA metrics are a great way to get a snapshot of your team’s overall software delivery performance, but with any form of measurement, there are caveats to keep in mind.

Maturity of your DevOps team

The first key challenge is asking why you’re considering implementing DORA and what benefits your organization and customers will reap. If you’re in the process of evaluating whether or not implementing DORA is right for your organization, your DevOps team, documentation, systems and processes should be mature, meaning you have an established process, tooling and workflow that your teams have adopted.

However, if the entirety of your DevOps team is only 1 person and you don’t have all your systems in place for CI/CD for instance, then it might not be a good use of time to start implementing DORA at your organization.

Proper Collection of Data

Another challenge to implementing DORA is collecting and tagging data in such a way that’s usable for your teams. It’s critical for the longer term success of your engineering team and should be a high priority for Engineering Managers to execute.

With the Mean Time To Recovery metric, you would need to tag issues as customer outages, so figuring out who’s responsible for that, as well as figuring out your criteria for an outage, and automating into your systems is something to consider.

Differing teams and products

No two teams or products are the same with their unique cadence and differing processes, so it might be difficult for some of your teams to become “elite performers”. As such, consensus among leadership and having proper change management in place to roll DORA metrics across your organization are important

Empower your team

The last thing to keep in mind is that DORA is a process and that it’s about rallying your team(s) to arrive at shipping a reliable product that’s finely tuned with systems and processes in place to attain “high” to “elite” performance. This can manifest itself in dashboards of your DORA metrics that are visible to everyone in the office, in Slack or at company-wide meetings as a regular point of discussion. The key point here is empowering your teams with these systems and metrics in place as guardrails to deliver the best product possible.

How DORA metrics and feature flags work together

Feature flags allow teams to control the deployment of new features or changes to their product. They help teams prevent errors and bugs from making their way to production by giving teams the ability to roll back features, which directly impact metrics like Deployment Frequency, Mean Time to Recover, and Lead Time to Changes. When properly implemented they help teams iterate on new features faster and with less risk when features are deployed behind feature flags.

Learn about DevCycle’s take on DORA Metrics

‍

Conclusion

DORA metrics are a great way to measure the performance of your software development and deployment practices. However, implementing DORA metrics should align with your organization’s business objectives and customers’ needs. Combined with feature flags, both make the delivery of your software and deployment faster and more reliable.