DORA Metrics at DevCycle

DevOps teams are under immense pressure to release code quickly and efficiently, but often these teams are burdened by fragile feature release processes. This can result in code being shipped with vulnerabilities that hackers can exploit.

In fact, DORA’s research shows that the most reliable software teams typically optimize their software for four metrics, including Change Lead Time, Deployment Frequency, Change Failure Rate and Mean Time To Recovery.

To dig into the topic of DORA further, Nik LeBlanc, Director of Engineering at DevCycle and Taplytics recently gave a talk on DORA metrics at DevCycle.

In this blog, we’ll highlight the Nik's talk, giving you the key takeaways and insights from our latest APIs and IPAs event. Watch Nik's talk below, or scroll on to read the about the highlights and bonus insights.

‍

01  What is DORA?

DevOps Research and Assessment, founded in 2014, acquired by Google in 2019. 

Determined Four Key Metrics that set apart high performing teams from low performing teams.

02  What are the Four Key Metrics

So what are the four key metrics they determined that set apart high performing teams for the low performing teams? They are as follows:

Deployment Frequency

How often does a team push changes to production

Elite: ~1 per day
Low: ~1 per 6 months

Lead Time for Changes

How long does it take for a commit to make it to production

Elite: ~1 day
Low: ~1-6 months

Change Failure Rate

How often does a deployment cause an incident on production?

Elite: 0-15%
Low: 46-60%

Time to Restore

How quickly can an incident be mitigated and production restored?

Elite: ~1 hour
Low: ~1 month

These four metrics can be grouped into two categories, speed and quality. Speed is deployment frequency and lead time for changes. Your lead time for changes could be super quick, but your quality could suffer as a result.

03  How To Use The Four Key Metrics

Above is a cycle that shows the typical workflow of the feature production, starting with Step (1) Eureka. Somebody on the product team has an idea, beginning the cycle and putting product brief together.

Next, the product team provides the brief to the engineering team. Engineering says, no problem. We can get it done. We've now entered step (2) Lead Time For Changes. That's how long it will take them to actually get their commits deployed to production. Following this is engineering shipping it, which is Step (3) Deployment Frequency aka how frequently you can ship.

But what could possibly go wrong? Everything will work perfectly right? Usually it doesn't and that's when we hit Step (4) Change Failure Rate, followed by learning from mistakes and getting production restored. Hopefully at this point it'll never happen again once Step (5) is hit with a short Time to Resolution.

If we go through each of these steps in the cycle, we can see the flaws that it tries to demonstrate.

04  Taplytics and DevCycle Compared

Here's how Taplytics and DevCycle compare and fit different gaps for product and engineering teams:

Taplytics

Founded in 2013, a YC Company.

Experimentation and Feature Flag Platform for Product Teams

Legacy Codebase

Continuous Delivery of Release Branches

DevCycle

Founded in 2021, a Taplytics Company.

Feature Flag and Comparative Analysis Platform for Engineering Teams

Greenfield Codebase

Continuous Deployment of merges to main

05  Using Feature Flags to Improve

We have two products, but we work on it as one team. If deploying features or resolving incidents takes time on one product, it keeps us from the other product.

We will only be “elite” once our metrics show a healthy flow of work across all of the products we’re responsible for, as a team. Our team uses feature flags to improve the four key DORA metrics by...

Deployment Frequency

Decoupling releases from deployments.

Lead Time for Changes

Reducing barriers to production by safely deploying unfinished features.

Change Failure Rate

Testing in production with an internal set of users.

Time to Restore

Disabling broken features, or finding bugs faster thanks to smaller batch sizes.

06  How To Track Your Own Metrics

Four Keys Project
https://github.com/GoogleCloudPlatform/fourkeys

Sleuth.io
https://www.sleuth.io/

Swarmia
https://www.swarmia.com/

Roll Your Own
Big Query, Metabase

07  Learn more about DevCycle

Interested in learning more about how DevCycle can help your teams crush your DORA metrics? Book a guided demo with our team or sign-up for a free account.

DevCycle is already being put to use by Fortune 500 customers such as the Royal Bank of Canada, GrubHub Inc. and Warby Parker Inc.