What Is Impact?

Updated 1 month ago by Jaala

Impact is a way to measure the ‘bigness’  of code changes that are happening, in a way that goes beyond simplistic measurements like LoC.

Impact attempts to answer the question: “Roughly how much cognitive load did the engineer carry when implementing these changes?”

To illustrate, consider the following example:

One engineer makes a contribution of 100 new lines of code to a single file.

Compare that to another engineer’s contribution, which touches three files, at multiple insertion points, where they add 16 lines, while removing 24 lines.

The significance of each contribution can’t be boiled down to just the amount of code being checked in. Even without knowing specifics, it’s likely that the second set of changes were more difficult to implement, given that they involved several spot-edits to old code.

On the left we have someone adding 100 lines of new code to a single file. On the right is an example that represents only 24 lines of new code written, but there’s a bit more going on here:

  • this change required modifying previous work
  • the edits happened in 4 different locations
  • 3 different files were affected

Even without knowing the severity of changes or comparing to historical changes, it’s probably safe to assume that the second contribution was more ‘expensive,’ and therefore carries is higher impact score.

Although change set #1 is technically more code, the added complexity of the work happening in change #2 could arguably make that change set at least as much work, and possibly more — even in the simplistic representation above, it’s clear that there’s more than just lines of code at play.

Impact is a measure of work size that takes this into account. Impact takes the following into account:

  1. The amount of code in the change
  2. What percentage of the work is edits to old code
  3. The surface area of the change (think ‘number of edit locations’)
  4. The number of files affected
  5. The severity of changes when old code is modified
  6. How this change compares to others from the project history

We roll all of this up to create score that more accurately lets us understand how “big” a particular piece of work is. The example above is based on data from our own repository, and when we evaluate the impact of these two changes we got the following results:

  • Change 1: 24 impact score
  • Change 2: 36 impact score

The more involved nature of change #2 yielded a higher impact score, even though only about 1/4 of the amount of raw code was written.

📝 Check out the second half of this blog post: Three ways to use Impact

How did we do?