small medium large xlarge

Against SEMAT

Thinking Differently about a Proposed Software Engineering Revolution

by Jorge Aranda

Generic image illustrating the article
  You say you want a revolution? Well, you know.... Dozens of really smart people call for a software engineering revolution, and one grad student asks to see the plan.  

You may have heard of a recent call for action that goes by the name of SEMAT (Software Engineering Method and Theory). It is backed by many of the great names in software industry and research; Ivar Jacobson, one of its proponents, presents it as a “revolution” whose goal is “to re-found software engineering as a rigorous discipline.”

What is it all about? The call for action itself is succinct, so it is helpful to reproduce it here for discussion:

Software engineering is gravely hampered today by immature practices. Specific problems include:

  • The prevalence of fads more typical of fashion industry than of an engineering discipline.

  • The lack of a sound, widely accepted theoretical basis.

  • The huge number of methods and method variants, with differences little understood and artificially magnified.

  • The lack of credible experimental evaluation and validation.

  • The split between industry practice and academic research.

We support a process to re-found software engineering based on a solid theory, proven principles and best practices that:

  • Include a kernel of widely-agreed elements, extensible for specific uses

  • Addresses both technology and people issues

  • Are supported by industry, academia, researchers and users

  • Support extension in the face of changing requirements and technology

It seems, judging by the hundreds of supporters of SEMAT so far, that for many people this call for action is a way to build a much-needed consensus in our field. But a close analysis leads me to conclude that it is a misguided movement. Let me explain my reasons (and note that I’ve posted similar criticisms on my blog).

Words Matter

First, it’s important to remember that our choice of words matters. Research shows that the way we frame a problem influences the strategies we use to solve it, and blinds us to other potential solutions. One instance out of many: the name of the games we play biases our behavior. In a 2004 study, players of a board game called “the Wall Street game” were far more cutthroat and less cooperative than players of a board game called “the Community game,” even though the rules for both games were exactly the same!

For decades now, the dominant metaphor for software development, at least in academia and in the trade literature, has been to treat it like an industrial engineering problem. Thus we call what we do “software engineering,” we attempt to define and compartmentalize its processes, we institute measurements and quality control cycles, and we treat the whole thing more or less like an assembly line.

It is a useful metaphor, to an extent. The problem is that the metaphor is so ingrained that we tend to forget that that’s all it is, that knowledge work is not industrial work and that coding is not manufacturing. We convince ourselves that what we do actually is industrial engineering and then we wail in desperation when we see that we’re not up to the standards of other engineering disciplines.

Fads and Progress

In fact, software development has little to do with most established engineering disciplines, and some people in the community (notably, but not exclusively, Tom DeMarco) have spoken against this paradigm. For this reason it is frustrating and disappointing that SEMAT, a supposedly “re-foundational” effort, is still trapped in the old and worn engineering metaphor: it takes for granted that we do “software engineering,” and that we should strive to do what other engineers do instead of wasting our time with “fads” that are more like this season’s frivolous fashion trends than like the serious, professional, piecemeal progress we should be aiming for.

Let’s turn a blind eye to the fact that several of those “fads” were started or promoted by some of the signatories of SEMAT. It is true that fads are prevalent in the software industry. But some of them (for instance, object orientation or extreme programming) end up being pretty good ideas, and some become mainstream. We stifle innovation if we dismiss every grassroots innovation and every exciting new idea as nothing more than a capricious fad just because it doesn’t fit with our preconceptions of what progress should look like.

And what should progress look like? The SEMAT call for action offers several hints, not all of them far-sighted. For example, it tells us that we should evaluate and validate practices experimentally. I’m in favor of controlled experiments in software research, of course, but I don’t think they should be singled out as the only or the most important empirical strategy at our disposal. As a researcher who has applied both experimental and non-experimental methods in software development, I have seen that controlled experiments tend to abstract away important elements of the phenomena we study, whereas non-experimental methods, such as case studies, provide a richness that is necessary and insightful. But since we see that engineers give primacy to experiments, we feel we should imitate them.

Theory and Research

The second half of the call for action is as disappointing as the first, and more puzzling. SEMAT proposes to re-found the field with “a solid theory” that includes “a kernel of widely-agreed elements,” but the proposal is completely silent regarding which theory and which elements it intends to use. It is a jarring thought: a group of people picks up a theory and by decree, bypassing any kind of peer review process, determines that the software field shall be built upon it. To my knowledge there is no scientific field that has prospered under such a shoddy epistemological basis.

The problem is that if we have a kernel of widely-agreed elements, that in itself is our theory, and we don’t need to re-found anything. We do not have a theory precisely because we do not have a kernel of widely-agreed elements—and judging by the list of the signatories of SEMAT, it’s hard to see how such a kernel could arise. A few examples: the list includes Watts Humphrey (creator of SEI’s CMM) and Ken Schwabber (co-creator of SCRUM). It includes formalists like David Harel and pragmatists like Erich Gamma; agilists like Alistair Cockburn and measurement apologists like Capers Jones. The list itself suggests that, more than a kernel of widely-agreed elements, what we already have is several vaguely formulated and competing theories that need to be made explicit, refined, and compared. I believe this is where we should spend our efforts.

The rest of the items in the SEMAT statement are innocuous enough to be unobjectionable. I think everybody would agree that we need stronger theoretical frameworks, more and better links between practice and research, and practices that are flexible and technologically and socially valuable. We can all rally behind such a wishlist. But there’s nothing here to go on, no stake in the ground, no bold re-foundational vision; just a recycled wish to get rid of controversies and to be more like an engineering discipline. A revolution might come to the software field one day, but not today.

Discuss this article with the author and other readers in the magazine forum. -Mike

Jorge Aranda is a Ph.D. student in Computer Science at the University of Toronto. He researches communication and coordination in software teams, the development of shared understanding, group cohesion, and the fitness and convenience of the paradigms that our community uses, sometimes inadvertently, to study these concepts. His academic website is at