Tracking engineering performance with analytics

The value of metrics in DevOps cannot be understated, especially if it’s your goal to remove obstacles, demolish bottlenecks, and accelerate your engineering team’s efforts. 

However, if you’re familiar with the work of Gene Kim and have engaged in the software engineering management methods outlined in the book, Accelerate: The Science of DevOps, the three authors (Kim, Forsgren, and Humble), identified the following:

“A recent Forrester (Stroud et al. 2017) report found that 31% of the industry is not using practices and principles that are widely considered to be necessary for accelerating technology transformations, such as continuous integration and continuous delivery, Lean practices, and a collaborative culture (i.e., capabilities championed by the DevOps movement)”

That single statistic highlights the less than ideal level of metrics tracking. 

What are software development metrics?

To be a little clearer, the metrics that Kim, Forsgren, and Humble speak of aren’t usage statistics (views, comments, likes, interactions). They’re metrics collected for and about your engineering or developer teams. 

What developer metrics are worth tracking

According to Atlassian, some of the benefits of tracking developer metrics include application performance improvements, and faster release cycles.

Developer metrics can include:

• Customer support tickets closed
• GitHub pull requests reviewed
• Attention paid to Research and Development
• Other measurements related to software code actions.

All these metrics are related to people within an organization, but they’re not engagement scores. 

Devops metrics, track and report on where an organization is  actually using its developer hours, versus where they should be beingbe invested – to support engineering performance improvements.

Connecting metrics, budgets and engineering performance

There’s no doubt about it – your developer hours are expensive.. Payscale reports 26.83USD as the 2021 average base hourly rate for a software developer, ranging up to as high as 53.00USD per hour. By knowing your metrics, you can make more informed financial decisions and see exactly which projects are consuming your developer budget 

For example, if you need to meet a Service Level Agreement (SLA) with your customers, it’s useful to collect metrics on customer accounts, from your ticketing system. Contrast these metrics with activity records from other sources – including time spent in strategy meetings, time paying down technical debt, and time spent on Research and Development.

Metrics are about culture, not just engineering performance

There’s more to developer metrics than budgets. 

For example, one company with an open source freemium business model may have a culture around transparency in everything they do. Metrics for developers should reflect that by capturing how long developers are spending on public facing requests. If there’s too much code activity measured in Research and Development for internal projects, or for Premium features that not everyone in the community sees, the metrics are revealing a disconnect between the developer culture, and developer’s actions. 

Addressing the attention paid to the open source community requests can improve the businesses’ commitment to transparency.

How to measure engineering performance

Based on what's important, there are some first steps to take to measure your engineering performance: 

1. Look at your bug reporting system.
2. Collect metrics on customer accounts from the bugs or other defect reports.
3. Compare these metrics with activity from time spent in strategy meetings, time paying down technical debt, and time spent on Research and Development.
4. Also compare time spent on any public facing open source content with Research and Development activity.

General advice on developer analytics providers 

• Don’t give the service access to your infrastructure or internal systems without first consulting an independent security audit.
• Read the security audit twice.
• A good metric service can be expensive – make sure they’re responsive to your developer team’s requests for changes.
• A skilled metrics collection service for developers can handle complex requirements. 

Key points on development metrics for software engineering management

Look at cycle time

If you suddenly see issue cycle time increases without any logical reason behind it, then that’s a sign of problems emerging.

Establish flexible reporting

Some of your team may need to see the available results  and create their own reports. Make sure these options are available for them, or request them if they’re not yet available.

Define your measurements

This is a vital step. Make sure you have buy-in from each stakeholder on what each measurement means for your team. For example, customer related metrics is a broad term, and you could narrow it down to measurement of bugs report, bug response time, or tickets marked resolved.

Choose a frequency

A weekly report is not going to suit every team, so calibrate a cadence that matches your needs. For instance, if you have an agile process with a 14-day sprint, have reports scheduled to arrive at the beginning, the midway point, and just before the sprint retrospective.

Next steps for your metrics and engineering performance

Make sure you carry out the necessary due diligence: 

• Review an independent security audit
• Confirm the metrics provider responds to your team
• Request any complex requirements early, to ensure you understand what your provider can and can’t handle.

By managing these points, you’re taking active steps to measure and improve your business software development and customer connection.

You can find more useful resources in the TinyMCE Developer Resources article, and contact us for more information about TinyMCE culture and rich text editing.