More on this book
Kindle Notes & Highlights
Read between
September 12 - September 19, 2020
We argue that there is no metric that adequately captures the full space of developer productivity and that attempting to find one is counterproductive. Instead, we encourage the design of a set of metrics tailored for answering a specific goal.
Tracking individual performance can create a morale issue, which perversely could bring down overall productivity. Research has shown that developers do not like having metrics focused on identifying the productivity of individual engineers
Any single productivity metric is intrinsically problematic. Productivity is too broad of a concept to be flattened into a single metric, and confounding factors will exacerbate the challenges with attempting such a flattening.
Productivity is a broad concept with many aspects. The problem is that productivity metrics are poor proxies of the underlying behavior or activity that we want to measure. As poor proxies, they are ripe for misuse.
At Google, we work with teams to figure out how they can leverage metrics to help make data-driven decisions. The process starts with clarifying the research questions and motivation. We then come up with custom metrics targeted toward those specific questions. This kind of thinking is similar to the Goal–QuestionMetric paradigm [1]. We validate these metrics against qualitative research (encompassing techniques such as surveys and interviews) to ensure that the metrics measure the original goal.
We instead narrow down the problem into a concrete research statement and seek metrics that address precisely the question at hand. This allows us to validate each individual metric against a specific goal, rather than against the vague concept of productivity.
There is no single productivity metric for software engineers. Instead, focus on a set of custom metrics targeted to a specific question.
For a manager to actually go from data to intervention, they need to make a creative leap: a manager has to take all of the measures, correlations, and models to ultimately infer a theory for what explains the productivity they’re observing. Making these inductive leaps can be quite challenging, and coming up with a wrong theory means any intervention based on that theory would likely not be effective and may even be harmful.
regardless of how accurately or elaborately one can measure productivity, the ultimate bottleneck in realizing productivity improvements is behavior change. And if our productivity utopia relies on developer insight into their own productivity to identify opportunities for individuals to change, why not just focus on developers in the first place, working with them individually and in teams to identify opportunities for increased productivity, whatever the team and organizational goals? This would be a lot cheaper than trying to measure productivity accurately, holistically, and at scale. It
...more
Great managers, by respecting the humanity of the people they are managing and understanding how their developers are working, are constantly building and refining rich models of their developers’ productivity all the time and using them to make identify opportunities for improvements.
The whole idea of measuring productivity is really just an effort to be more objective about the subjective factors that are actually driving software development work.
The difference between efficiency and effectiveness is usually explained informally as “efficiency is doing things right” and “effectiveness is doing the right things.”
efficiency refers to the utilization of resources and mainly influences the required input of the productivity ratio. Effectiveness mainly aims at the usefulness and appropriateness of the output as it has direct consequences for the customer.
Productivity of knowledge work therefore has to aim first at obtaining quality—and not minimum quality but optimum if not maximum quality.
Comparing the ideal with the actually produced functionality and quality shows the effectiveness of the software development activities; the relation of the ideal to the actual effort gives the efficiency. Both have an influence on productivity.
performance contains profitability and adds factors such as customer perception.
productivity is expressed as the combination of effectiveness and efficiency: a team can be productive only if it is effective and efficient!
The three dimensions in the proposed productivity framework for software engineering are as follows: Velocity: How fast work gets done Quality: How well work gets done Satisfaction: How satisfying the work is
Velocity and quality taken together make up overall work efficiency and effectiveness, while velocity and quality may impact satisfaction in different ways. An increase in velocity may lead to reduced costs (and improve the satisfaction of managers), but at the same time it can lead to increased stress for developers (and reduce their satisfaction and in turn incur future costs).
The velocity dimension captures how productivity is often conceptualized in terms of the time spent doing a task or the time taken (or cost) to achieve a given quantity of work.
Quality may be an internal consideration in a project (e.g., code quality) or external to a project (e.g., product quality from the perspective of the end users). Metrics for quality in a software project could include counts of negative characteristics such as post-release defects or self-reported ratings of delays incurred by technical debt.
Engineering satisfaction is a multifaceted concept, which makes it challenging to understand, predict, or measure. This dimension captures human factors of productivity and has several possible subcomponents, including physiological factors such as fatigue, team comfort measures such as psychological safety, and individual feelings of flow/focus, autonomy, or happiness. Learning or skill development that may positively impact long-term quality, developer retention, or velocity may manifest as an increase in satisfaction. For developers, satisfaction may be impacted by the real or perceived
...more
The following are the main types of lenses we feel are important to consider: Stakeholders : Different stakeholders (e.g., developer, manager, vice president, etc.) may have varied goals and interpretations of any sort of productivity measurement. Before trying to understand and measure productivity, it is essential to identify which stakeholders are of concern and what is important to those stakeholders. It may not be immediately obvious which stakeholders should be considered; a researcher or practitioner may need to carefully elicit which stakeholder perspectives are important. Context :
...more
This highlight has been truncated due to consecutive passage length restrictions.
Unfortunately, going from goals to metrics is not trivial as metrics are typically proxies for specific aspects of a goal.
the goal-question-metric (GQM) approach for understanding and measuring the software process [1, 2] works by first generating “questions” that define goals and then specifying measures that could answer those questions. GQM suggests a systematic approach to do the following: Conceptualize goals aimed at understanding or improving software engineering tools and processes Specify research questions to operationalize those goals Define metrics for understanding or measuring tools and processes
HEART first decomposes a high-level usability goal (such as “my app is awesome”) into subgoals, abstract “signals” that could measure those subgoals (e.g., time spent with app), and specific metrics for those signals (e.g., number of shares or number of articles read in app). In addition to this goals-signals-metrics breakdown, the HEART framework splits usability into five dimensions: happiness, engagement, adoption, retention, and task success.
we propose splitting into goals, questions, and metrics in combination with the productivity dimensions and lenses.
Productivity should be considered along three dimensions: quality, velocity, and satisfaction. These three dimensions complement each other but often are in tension with each other. The dimensions have several possible attributes; measuring them is highly task and situation dependent. Productivity goals may be refined by considering the three dimensions through a set of perspective lenses. The main lenses we suggest include the stakeholders, the development context, the levels, and the time scale.
Research shows that team productivity is actually bounded not by how efficiently individual developers work but by communication and coordination overhead
Finding a way to manage teams that streamlines communication, coordination, and decision-making is therefore key and perhaps more impactful than making individual developers faster.
Given all of these complex factors of organizational culture, one might imagine that a fruitful way to think about productivity from an organizational perspective is to reason about the unintended consequences of norms and policies on individual and team productivity.
Viewing productivity from a market lens acknowledges that the whole purpose of an organization that creates software is to provide value to customers and other stakeholders.
These ever-evolving understandings of an organization’s goal then filter down to new organizational policies, new team-level project management strategies, and new developer work strategies targeted at improving this top-level notion of productivity.
studies of software engineering expertise show that great developers are capable of reasoning about code through all of these lenses
It means you’re not going to find one measure for everything. Individuals, teams, organizations, and markets need their own metrics because the factors that affect performance at each of these levels are too complex to reduce to a single measure.
teams, organizations, and markets need different productivity metrics. Productivities for these different lenses are often in tension.
A multiple-level productivity measurement technique is the macro, micro, and mid-knowledge worker productivity models, which seeks to measure productivity at the factory, individual contributor, and department levels, respectively. This technique measures productivity over time using attributes such as quality, cost, and lost time. The main advantage of these techniques is that they provide a more holistic view of organizational productivity than many other metrics, but at the same time, collecting them can be complex.
Understanding productivity drivers is valuable because it tells organizations what changes they can make to improve knowledge worker productivity. While some productivity drivers are specific to software development, such as code complexity (see also Chapter 8), other drivers probably apply equally well to knowledge workers generally and software developers specifically, such as the need for quiet spaces required for concentration.
While the prior environmental drivers enable productive work through organizational practices, individual work practices measure to what extent knowledge workers will actually implement these practices.
Finally, Palvalin includes a knowledge worker’s well-being at work both as a driver of productivity at work and as an outcome of productivity. A productive knowledge worker is one who enjoys and is enthusiastic about their work, finds meaning and purpose in their work, is not continuously stressed, is appreciated, has a work-life balance, finds the work atmosphere pleasant, and resolves conflicts with co-workers quickly.
task interruptions take time to recover from and lead to errors.
One of the main insights to come from modeling work using the Memory for Goals theory is that the longer an interruption, the more likely it is that errors are to occur, including forgetting to resume the task altogether (and for specific cases, the models can give even more specific and exact predictions). Therefore, the implication of this work is that there is value in avoiding being interrupted.
Across several studies, we found that people were very quick at locating the best possible strategy for dividing their time between tasks. We learn from this work that people are actually pretty good at multitasking, when the relative importance of each task is made clear to them.
In a work environment, observations found that people self-interrupt almost as often as experiencing interruptions by an external source such as a phone call or colleague entering the office
Longer interruptions (or work breaks), such as taking a walk in nature during work hours, have been shown to increase focus and creativity at work [1]. Observational studies have identified that people use a variety of social media and news sites to take breaks to refresh and to stimulate themselves
It is theorized that people who multitask more and who are susceptible to interruptions may have lower ability to filter out irrelevant stimuli
Consistent with the results from interruption experiments, observational studies also reveal that frequent interruptions result in feelings of reduced productivity. However, regular breaks from work are also necessary, and people return from breaks feeling energized and ready to resume their work.
Interruptions can take time from which to recover from and can lead to errors. Shorter interruptions are less disruptive than longer interruptions. Interruptions delivered during a natural break in a task are less disruptive. Interruptions that are relevant to the current task are less disruptive. Resuming a task too quickly can lead to errors being made. All of these characteristics of the resumption lag can be explained by an underlying memory retrieval process. People self-interrupt almost as often as being interrupted by external sources. People often work on multiple tasks at the same
...more
Being happy corresponds to frequent positive experiences, which lead to experiencing positive emotions. Being unhappy corresponds to the reverse: frequent negative experiences leading to negative emotions. Happiness is the difference or balance between positive and negative experiences. This balance is sometimes called affect balance
We have also presented a theory that provides an explanation of how affect impacts programming performance [3]: events trigger affects in programmers. These affects might earn importance and priority to a developer’s cognitive system, and we call them attractors . Together with affects, attractors drive or disturb programmers’ focus, which impacts their performance.
