Continuous improvement fo the engineering process and the delivery performance of our teams motivated that endeavor.
We were missing a tool to be able to track the following metrics, identified in the book “Accelerate” by N. Forsgren, J. Humble & G. Kim, because they turn to be revealing indicators of software team delivery performance:
- Deployment Frequency
- Lead Time
- Mean Time To Restore (MTTR)
- Change Failure Rate
Deployment Frequency
The number of deployment over a period of time
Lead Time
Lead time is the time between a commit is submitted for review and the time it gets deployed.
It can be decomposed in two:
- Review Lead Time: from commit submitted for review to approved, basically when a Pull request is opened until the corresponding commit is deployed.
- Deployment Lead Time: from approved to deployed successfully
Mean Time To Restore
The time between when an incident (failure of service) is detected to the time it is resolved.
Change Failure Rate
The ratio of failed deployment over total number of deployments.
Solution
After some research, I did not find any simple existing tool to easily gather the data and display these metrics. Hygieia was a close contender but complex and not easy to adapt in my experience.
So I decided to build a small solution to my problem and here comes: devopstic
It’s composed of a service with a REST api, a Postgres database and uses Grafana to display results.
The observed systems (CI/CD, etc.) send events to devopstic service. The service logic then create the entities like commit and incident based on the event type and compute the adequate values.
The project is open source and available at:
https://github.com/splisson/devopstic
Let me know what you think, any feedback is welcome!
References
“Accelerate” Nicole Forsgren, Jez Humble, Gene Kim https://www.amazon.com/Accelerate-Software-Performing-Technology-Organizations/dp/1942788339/ref=sr_1_1?keywords=accelerate+devops&qid=1560966448&s=gateway&sr=8-1
“Measuring DevOps Success with Four Key Metrics” https://stelligent.com/2018/12/21/measuring-devops-success-with-four-key-metrics/