Productivity for developers is an important topic for many, the developers who want to feel productive, make great software and deliver outstanding outcomes to their users, and managers who want to know where there team needs help, and who is excelling. Developer productivity metrics such as SLOC and SP/t have been superseded with metrics from DORA by Google and more recently in Nicole Forsgren’s work defining SPACE. I’ll define and discuss each of these below.
Why lines of code is a bad measure of developer productivity
SLOC (Source Lines of Code) measures raw lines of code, and not functionality. Duplicated code is measured as code production, but maximising this is counterproductive because duplicated code is a maintenance headache.
Story-points per sprint (SP/t)
An agile scrum approach to developer productivity, use of story points improves on SLOC by measuring functionality and removing an incentive to duplicate code. However it suffers from the variability and subjectivity of story-point estimations. Subsequent modification of story-point numbers as political pressures are exerted, and as more information comes to hand, further exacerbates the subjectivity of story-point velocities.
DORA (DevOps Research and Assessment) Metrics
DORA metrics move from the individual developer to the team and organisation
- Deployment Frequency – How often an organisation successfully releases to production
- Lead Time for Changes – The amount of time it takes a commit to get into production
- Change Failure Rate – The percentage of deployments causing a failure in production
- Time to Restore Service – How long it takes an organisation to recover from a failure in production
The SPACE of Developer Productivity
The most recent and deepest analysis of this topic is the work by Nicole Forsgren for SPACE. I find this to be the deepest analysis of the complex interactions of individual and organizational performance.
Space stands for
- S = Satisfaction
- P = Performance
- A = Activity
- C = Communication
- E = Efficiency
I encourage you to read this one for yourself at the ACM queue publication here.
Here’s a quote from the introduction to the article:
This article explicates several common myths and misconceptions about developer productivity. The most important takeaway from exposing these myths is that productivity cannot be reduced to a single dimension (or metric!).
Forsgren et al. The SPACE of Developer Productivity. March 6, 2021
Below I’ve selected some sections of the full text to provide a shortened summary.
Satisfaction and well-being
Satisfaction is how fulfilled developers feel with their work, team, tools, or culture; well-being is how healthy and happy they are, and how their work impacts it.
To assess the satisfaction dimension, you might measure the following:
• Employee satisfaction. The degree of satisfaction among employees, and whether they would recommend their team to others.
• Developer efficacy. Whether developers have the tools and resources they need to get their work done.
• Burnout. Exhaustion caused by excessive and prolonged workplace stress.
Performance
The performance of software developers is hard to quantify, because it can be difficult to tie individual contributions directly to product outcomes.
Performance is often best evaluated as outcomes instead of output.
Example metrics to capture the performance dimension include:
• Quality. Reliability, absence of bugs, ongoing service health.
• Impact. Customer satisfaction, customer adoption and retention, feature usage, cost reduction.
Activity
Some of the developer activities that can be measured and quantified relatively easily are:
• Design and coding. Volume or count of design documents and specs, work items, pull requests, commits, and code reviews.
• Continuous integration and deployment. Count of build, test, deployment/release, and infrastructure utilization.
• Operational activity. Count or volume of incidents/issues and distribution based on their severities, on-call participation, and incident mitigation.
These metrics can be used as waypoints to measure some tractable developer activities, but they should never be used in isolation to make decisions about individual or team productivity because of their known limitations.
Communication and collaboration
Software development is a collaborative and creative task that relies on extensive and effective communication, coordination, and collaboration within and between teams.
The following are examples of metrics that may be used as proxies to measure communication, collaboration, and coordination:
• Discoverability of documentation and expertise.
• How quickly work is integrated.
• Quality of reviews of work contributed by team members.
• Network metrics that show who is connected to whom and how.
• Onboarding time for and experience of new members.
Efficiency and flow
Finally, efficiency and flow capture the ability to complete work or make progress on it with minimal interruptions or delays, whether individually or through a system. This can include how well activities within and across teams are orchestrated and whether continuous progress is being made.
Some example metrics to capture the efficiency and flow dimension are:
• Number of handoffs in a process; number of handoffs across different teams in a process.
• Perceived ability to stay in flow and complete work.
• Interruptions: quantity, timing, how spaced, impact on development work and flow.
• Time measures through a system: total time, value-added time, wait time.
Efficiency is related to all the SPACE dimensions.