Software Design X-Rays

Fix Technical Debt with Behavioral Code Analysis

by Adam Tornhill

Are you working on a codebase where cost overruns, death marches, and heroic fights with legacy code monsters are the norm? Battle these adversaries with novel ways to identify and prioritize technical debt, based on behavioral data from how developers work with code. And that’s just for starters. Because good code involves social design, as well as technical design, you can find surprising dependencies between people and code to resolve coordination bottlenecks among teams. Best of all, the techniques build on behavioral data that you already have: your version-control system. Join the fight for better code!

Printed in full color.

Use statistics and data science to uncover both problematic code and the behavioral patterns of the developers who build your software. This combination gives you insights you can’t get from the code alone. Use these insights to prioritize refactoring needs, measure their effect, find implicit dependencies between different modules, and automatically create knowledge maps of your system based on actual code contributions.

In a radical, much-needed change from common practice, guide organizational decisions with objective data by measuring how well your development teams align with the software architecture. Discover a comprehensive set of practical analysis techniques based on version-control data, where each point is illustrated with a case study from a real-world codebase. Because the techniques are language neutral, you can apply them to your own code no matter what programming language you use. Guide organizational decisions with objective data by measuring how well your development teams align with the software architecture. Apply research findings from social psychology to software development, ensuring you get the tools you need to coach your organization toward better code.

If you’re an experienced programmer, software architect, or technical manager, you’ll get a new perspective that will change how you work with code.

1. Why did you decide to write this book?

I’ve spent much of my career trying to improve existing code, and I found it progressively harder as software systems and organizations keep growing in scale. Even though the software industry has improved dramatically over the two decades I’ve been part of it, we do keep repeating avoidable mistakes by isolating our influences to technical fields. I wrote this book to provide that missing link between technical and social sciences. This blend, behavioral code analysis, lets us prioritize technical debt based on the most likely return on investment, evaluate our software architecture based on how well it supports the work we do, bridge the gap between developers and business oriented people, and much more. My goal was to take mainstream software development one step closer to a point where decisions — both technical and organizational — are influenced by data and research from multiple fields. I’m really, really happy with the end result and hope you’ll enjoy it too.

2. What kind of experience would help readers get the most out of this book?

To get the most out of this book you should be an experienced programmer, technical lead, or software architect. The most important thing is that you have worked on larger software projects and experienced the various pains and problems. You don’t have to be a programming expert, but you should be comfortable looking at smaller code samples. Most of the discussions are on a conceptual level and since the analyses are technology-neutral, the book will apply no matter what programming language you work with.

3. What do you hope readers take away from this book?

The key point in the book is to base decisions on data by putting numbers on our gut feelings. Software development, and in particular the people side of it, is notoriously hard to get right. By embracing behavioral code analysis we get to tap into the social side of code and start to measure things that we cannot deduce from the code alone, like communication and coordination needs, Conway’s Law, and long-term complexity trends. I also want to point out that behavioral code analysis doesn’t offer any silver bullets, nor does it intend to replace anything. Instead the techniques in this book are here to complement your existing expertise by focusing your attention on the parts of the system that need it the most.

4. How does this book compare to Your Code As A Crime Scene?

Software Design X-Rays represents the evolution of the ideas from my previous book, Your Code As A Crime Scene. If you read the previous book, you’re already familiar with hotspots and some of the change coupling metrics presented in chapters 2 and 3. These two concepts lay the foundation for the more advanced analyses, and the new book goes deeper into both areas. Most of the material points out new directions that I haven’t covered before. Software Design X-Rays is of course also interdisciplinary and blends software engineering with psychology, but this time there are no direct forensic references.

5. What’s your favorite part of the book?

I like chapter 4 on refactoring patterns since it makes the technical debt detection techniques actionable by providing specific recommendations. I also enjoyed writing chapter 7, Beyond Conway’s Law, that brings some valuable findings from group psychology into the software field. Those findings fill out the missing pieces in Conway’s Law and help guide your organization towards better code. Finally, I have to mention the last chapter which explores preventive and predictive uses of behavioral code analysis. I like that chapter because the resulting information becomes like an extra team member that points out areas of the code in need of our attention as we code along.

• Errata, typos, suggestions

• Source Code (zip file)

Releases:

• P1.0 2018/03/06
• B4.0 2018/02/26
• B3.0 2018/01/05
• B2.0 2017/11/06

• *Introduction
• Prioritize and React to Technical Debt
  • Why Technical Debt Isn’t Technical
    • Questioning Technical Debt
    • The Perils of Quantifying Technical Debt
    • Mine Your Organization’s Collective Intelligence
    • Prioritize Improvements Guided by Data
  • Identify Code with High Interest Rates
    • Measure Interest Rates
    • Prioritize Technical Debt with Hotspots
    • Evaluate Hotspots with Complexity Trends
    • Use X-Rays to Get Deep Insights into Code
    • Escape the Technical-Debt Trap
    • Exercises
  • Coupling in Time: A Heuristic for the Concept of Surprise
    • Uncover Expensive Change Patterns
    • Detect Cochanging Files
    • The Dirty Secret of Copy-Paste
    • The Power of Language-Neutral Analyses
    • Exercises
  • Pay Off Your Technical Debt
    • Follow the Principle of Proximity excerpt
    • Refactor Congested Code with the Splinter Pattern
    • Build Temporary Tests as a Safety Net
    • Turn Hotspot Methods into Brain-Friendly Chunks
    • The Curse of a Successful System
  • The Principles of Code Age
    • Stabilize Code by Age
    • The Three Generations of Code
    • Refactor Your Package Structure
    • Scale from Files to Systems
    • Exercises
• Work with Large Codebases and Organizations
  • Spot Your System’s Tipping Point
    • Is Software Too Hard?
    • Divide and Conquer with Architectural Hotspots
    • Analyze Subsystems
    • Fight the Normalization of Deviance
    • Toward Team-Oriented Measures
    • Exercises
  • Beyond Conway’s Law
    • Software Architecture Is About Making Choices
    • Measure Coordination Needs
    • Code Ownership and Diffusion of Responsibility
    • Analyze Operational Team Boundaries
    • Social Groups: The Flip Side to Conway’s Law
    • Combine Social and Technical Information
  • Toward Modular Monoliths through the Social View of Code
    • Dodge the Silver Bullet
    • Layered Architectures and the Cost of Consistency
    • Monolithic Alternatives: Use Case and Feature-Centric
    • Discover Bounded Contexts Through Change Patterns
    • The Perils of Feature Teams
    • Clean and Dirty Architectures
    • Exercises
  • Systems of Systems: Analyzing Multiple Repositories and Microservices
    • Analyze Code in Multiple Repositories
    • Compare Hotspots Across Repositories
    • Track Change Patterns in Distributed Systems
    • Measure Technical Sprawl
    • Distribution Won’t Cure the Dependency Blues
    • Exercises
  • An Extra Team Member: Predictive and Proactive Analyses
    • Detect Deviating Evolutionary Patterns
    • Catch the Absence of Change
    • Guide On- and Offboarding with Social Data
    • Know the Biases and Workarounds for Behavioral Code Analysis
    • Your Code Is Still a Crime Scene
    • Exercises