One thing, at least, is clear from these reflections: What was defined at the Snowbird meeting remains a work in progress. It wouldn’t be agile otherwise.
When we realized that February 2011 was the 10th anniversary of the Agile Manifesto, we suggested to all of the 17 people who were present at that historic meeting that it would be appropriate for them to put down their thoughts, and send them to us for inclusion in this Agile Issue of PragPub.
Pretty much all of them responded, mostly with some variation of, “Yeah, you and every other technical magazine, blog, podcast, or conference organizer.” So we nagged them some more. In the end, ten of the authors of the Agile Manifesto sent us something. Some wrote full essays, others wrote a few sentences. Some reflected on progress made, others on challenges remaining. Here are their thoughts on the Agile Manifesto’s tenth birthday.
Ten Down, A Thousand to Go
“We shall do a much better programming job, provided that we approach the task with a full appreciation of its tremendous difficulty, provided that we stick to modest and elegant programming languages, provided that we respect the intrinsic limitations of the human mind and approach the task as Very Humble Programmers.” Edsger W. Dijkstra offered that sage advice nearly forty years ago, at the conclusion of his Turing Award lecture in 1972. Apparently, as an industry, we’re slow learners.
The ensuing programming languages over the years, from Modula 2 and Ada to Java and C#, were neither modest nor elegant. Instead of respecting and working with our innate limitations, methodologies seemed to assume that people could make great decisions despite a complete absence of fact; that we could could plan months and years in advance with prescient clarity; that we could create working code that would, as if by magic, answer every need of an unseen and unknown customer. We thought we could even write code that did what exactly what we intended, more or less the first time. Well, at least as soon as it compiled successfully. Maybe with a few print statements for good measure.
Time passed, and some thirty years after Dijkstra’s gentle polemic, the Agile Software Development movement reached critical mass in Snowbird, Utah. The seventeen of us interested parties who gathered on that date had different views on how software development should best be conducted, but tacitly agreed on one very important point: the need, as Dijkstra put it, to respect the intrinsic limitations of the human mind and approach the task as Very Humble Programmers.
A bunch of us (for some large value of bunch) recognized what it meant to be “humble,” in the sense that Dijkstra meant it. To take small bites, and not try and do too much at once, or to try to prognosticate too far out. To verify and validate what we’d written, not trusting our own overrated coding prowess. To openly and honestly discuss what we’d do next, not assuming that we knew better than anyone else. We recognized that software development is like trying to run through a very large dark cave with a very small weak flashlight. Raw speed will just kill you faster if you’re headed over a steep cliff.
Now, exactly ten years to the month after the catalytic coining of the word “agile,” where do we stand? Did the world at large catch on that programming was a might trickier than it seemed, and that you needed the support of modest and elegant tools, flexible and adaptable methods, and experienced, enthusiastic team members and customers in order to succeed?
Meet the New Boss, Same as the Old Boss
Let’s start with education. Or rather, the lack of it. The better software engineering techniques promoted by agile methods still aren’t widely taught in college and university. There are some bright spots, and some excellent programs that actually “get it,” but we’re far from universal acceptance. There are plenty of degree programs where graduates have never worked in teams, never used version control, and never written a unit test. So academia isn’t quite on board yet. What about the corporate world?
Last summer, I had the good fortune to visit a “very advanced” agile shop. These folks really did embrace agile methods with a discipline, completeness, and a zealous fervor that would be hard to match. In many ways, they could have been a poster child for agile methods. For such a magnificent, large-scale operation to succeed, perhaps we had won the war after all. Perhaps the agile ideals really had taken hold, and manifested themselves here in this largish company, freed from the shackles of mindless dogma that had held previous generations of programmers in thrall.
But these weren’t productive developers freed from mindless process dogma. They were agile slaves. The dogma they followed was ours, and they followed it well. And as with many organizations in a similar position, they saw some promising results. Continuous integration, refactoring, unit tests, pair programming—all these techniques yielded some benefit. But they weren’t thinking, they weren’t reacting, they weren’t being agile. When problems came up, they addressed them with all the grace and elegance of a deer caught in the terrifying blaze of alien headlights. They knew how to do agile; they didn’t know how to be agile.
And that’s part of the continuing challenge. We’ve made great headway in getting better basic techniques adopted. Years ago you still found shops that didn’t use any version control system at all—just source code files on a shared disk, and “last one in wins.” That’s much less common now. Unit testing and its descendants are increasingly accepted and practiced, and even pair programming, often viewed with malignant suspicion by facilities managers and stakeholders alike, isn’t quite as foreign and scary as it once was.
But folks need to understand what it means to be agile; how to successfully solve the new problems that arise constantly. There’s no formula for that, and never will be. As Dr. Patricia Benner puts it so eloquently, “Practices can never be completely objectified or formalized because they must ever be worked out anew in particular relationships and in real time.” That’s a big part of the picture we haven’t got quite right yet: helping people work out new practices in particular contexts that work well for them. That’s the secret to real agility; it’s not about doing pair programming, or having stand-up Scrum meetings. Anyone can dogmatically follow the practices prescribed by others.
But true agility goes beyond the dogma, beyond the practices. Agility is about adapting; adapting your process, your language, your tools, your team, and yourself to respond to the situation at hand. We need to be positioned to be able to do that in order to fully realize the promise of agility, and maybe even move beyond it.
So what’s next? I humbly suggest we take the direction for the future from Dijkstra: stick to modest and elegant programming languages, and acknowledge and respect the intrinsic limitations of the human mind.
Java is neither modest nor elegant. But what about Ruby, Clojure, or Erlang? Are these languages modest or elegant enough? Can we come up with better programming languages, languages that make it easy to do the right thing? How many programming languages do you know? If it’s only two or three, grab another dozen before you answer.
As far as our limitations go, I’ve hit on a few of those in the PragPub “Guru Meditation” column (named for the old Amiga OS error messages.) Our brains aren’t wired optimally for this sort of work in general, and not well for agile methods in particular. But that’s just a start. Here are a few other interesting topic areas that we could pursue to go beyond agile:
The Dreyfus Model. Experts are not merely smarter novices. There are many changes on the road from novice to expert, from how you approach problem solving to how and what you communicate. Yet on most teams, we tend to treat all team members equally. This is unfair to both novice and expert, in different ways. We can be more effective by using the lessons of the Dreyfus Model of skill acquisition to guide teaching, learning, and teamwork.
Communications. In the 2011 State of the Union address, President Obama spoke of education, competitiveness, and inspiration. But the number one word that listeners heard? “Salmon.” The president made a joke about overlapping government responsibilities (which involved salmon), and that’s what stuck with the audience. People don’t always hear what you want them to hear, and we don’t communicate requirements, technical needs, or opportunities all that well. We can do better. From interviewing and working with users to help them determine their needs, to teaching the dangers of technical debt and the need to refactor, we can use what we know of cognitive limitations to communicate the important stuff more effectively.
The Nature of Thinking. But improving communications is just the tip of the iceberg. We can improve our creativity and inventiveness—two critical components for successful software development. I touch on these topics in Pragmatic Thinking & Learning, but that’s only a start. A lot of the research in cognitive- and neuroscience can be applied to improve software development and other “thinking” professions. So far, we ignore most of it, because some of the lessons seem counter-intuitive to what we’ve been taught. But our brains are plastic and pliable, so we can change how our brains work, for better or worse. Let’s deliberately try for better.
Something to think about.
To Agility... and Beyond!
I took the Manifesto in one direction with my To Agility... and Beyond! talk. The point there was that the Manifesto assumes that there is someone “out there” who knows (or at least will tell you) what to do next. What do you do if that isn’t the case? Increasingly it seems to me that this will be the case for interesting development. How do you develop when it’s everybody’s job to figure out what to do next?
This is not the only direction the Manifesto could go. In general, I’d like to see it talk about increasing responsibility and accountability, productive conflict, and personal growth, but that’s probably because these are the issues I struggle with. :)
We Really Changed the World—and Yet…
I was excited to be at Snowbird for at least two reasons. First of all, it was a meeting of important and interesting people, some of whom I knew, some of whom I did not, most of whom I at least knew of and had reason to admire. I had been noisy about Agile but did not and do not consider myself to be an inventor of it, so I felt honored to be there.
I was also excited because I’ve always been interested in doing good software for people who need it, and Agile—especially Extreme Programming—holds out the prospect of doing good work in a context of delivering great results to someone who actually wants them. I had found the ideas and practices to be incredibly energizing, even at my then-advanced age.
It seemed to me that we had a handle on something great, and that we were going to change the world. And, in fact, I guess we did.
Today, there are a lot of people doing Agile, and a lot more saying they are doing Agile while really doing something else. What we see, however, is that if an organization makes a good-faith effort to do what we have suggested, they do get benefit. There are over 100,000 “Certified ScrumMasters,” so surely there must be a million people trying to do what we offered, and I believe that most of them are probably getting some benefit.
Frankly, though, I had imagined more. I had hoped that many people would adopt these ideas, and I had imagined a significant step upward in project success among those who did adopt them. I had imagined the industry really moving up a notch. What happened is that many have adopted the ideas, at least in name, but that few of them have attained anything like the benefit that is possible.
That leaves me pleased in my mind, and less than pleased in my heart.
So what to do? I try to review and adjust course every year and maybe as this is a decade marker for the Manifesto I should take a harder look.
If there’s something new and amazing on the horizon, I do not yet see it. Lean and Kanban aren’t it. Fuzz your eyes right, and they are the same as Agile, or Agile is the same as them if you prefer. Mostly cut from the same cloth, that’s what I’m saying. Still, if something newly great comes along, surely I’ll have a hard look and dig in if I can.
Meanwhile, my first mission will remain the same: eradicate all errors on the Internet, at least regarding Agile. I’ll surely continue my appearances and rants on as many mailing lists as will have me. I do observe that I’m laying back a bit, letting other younger, less bruised individuals take more of the lead. But I don’t expect that I’ll stop.
In the past 18 months or so, I’ve primarily been working with relative newcomers to Agile, teaching “Certified ScrumMaster” courses. That has been fun, most of the time, and fairly lucrative, but it is not fulfilling. Many of the students are in organizations where they cannot possibly succeed, and many more are just in the class because they were sent to it. It furrows into my heart.
In the time remaining before me, I want to work more with people who truly want to excel. I don’t imagine for a moment that I can teach them all they need to do to excel, but I do believe they can profit from working with me and observing how I do this stuff. And we’ll all have a good time.
Finally, I’ve been “working” on a book for quite a while. It is time to bear down and finish it. I am confident that it’ll be interesting to people, and more important, it will say something that I want to say.
Change Begins with Me
I’m all over the “agile sucks” thing. In general, the collective “we” suck at software development, agile or otherwise. Most of the folks paying for software have not a clue whether they are getting their money’s worth or not. There’s no Car & Driver or Consumer Reports showing the stark contrasts that exist between software teams/products. Or John Stossel showing the incredible waste in one software dev organization versus another.
I’ve pretty much given up on preaching to companies how to change their ways. Often, the software bit of what they do is so far down the chain of bureaucratic flotsam as to matter very little. I’ll happily work with small teams to help them solve real problems the best we can, and do it as “agile” as we can—in context, of course. As long as we have the chance to have an impact and change what needs changing in order to succeed, that’s where you will find me toiling away.
I’ve pretty much resolved to change... me. Yup, just me. I want to be the best that I can be at software development and producing deployed code. To that end, I am thankful for Ruby and all the good published works on pragprog—I confess, I am an addict, with a stack “hits” I am still waiting to ingest.
Maybe through changing my ways to be as good as they can be, I’ll change those around me. Together, other teams might take notice of what we’re doing and think about their own journey, starting at themselves. Just maybe.
Part of me knows that this is a cop-out. Part of me says that I should strive to help shine light on the dark recesses of the truths behind software development in the wild. The other part of me says, let the market do its thing. For now, the libertarian in me is winning. To each their own. You can believe that scrum in a vacuum will save the day, you can believe that agile sucks, you can believe that waterfall is for features and agile is for bugs (yeah, that’s a real quote). Whatever. All I ask is that you question with boldness everything that you and your software organization do. Since you all might be zombies, you should even allow outsiders to question with boldness what you do. Following that, you’re free to continue or change as you see fit.
After all, agile is a state of mind, an attitude—nothing more, nothing less.
It’s About People
Agile is a neat term. Someone once said that all this would have never happened if we called it the “Fragile Manifesto.” It certainly achieved one of its goals, to drive a stake into the heart of Waterfall. Using the wrong process was nothing but frustrating and demeaning.
However, agility required people to retool their practices, tooling, and most significantly—their skills, beliefs, and habits. The desire is strong and the vision is clear, but the path is not straight, and the effort and courage to achieve agility is phenomenal. I think we have started a quiet revolution toward people no longer being thought of as resources. Instead, paraphrasing an old IBM slogan, “People are our most valuable asset,” I would say, “we are our people.”
I Went for the Skiing
When I first learned about extreme programming back in 1999 I was very excited about it. It provided logical solutions to many of the problems I had experienced over the prior two decades of software development. I was not quite sure about pair programming and test-driven development; that all seemed promising but a bit… extreme. But after a little experience I found both practices productive, and fun.
In 2000, while I was with Object Mentor, we were early adopters and advocates of extreme programming. We knew that we had discovered something that could help many development teams get out of their rut. Organizations hired us to help them change. We had visions of walking in the door, explaining XP and having everyone embrace it like we did. Wow, what a shock: they were not ready to change.
Current practice was not delivering quality software on time, but XP was too extreme. Just look at the name. “Will we need elbow pads and helmets?” our clients would joke. Developers and product managers of the day thought that the solution to their current woes was to do more upfront work, not less. The idea of fast feedback along with evolving design and requirements was too foreign. Well, we persisted and helped many organizations get started. XP was new then. Agile did not even exist by that name.
In the beginning of 2001 Bob Martin invited me to join him at Snowbird for a meeting with the guys that started XP and some of the other “Lightweight” development processes. My answer was, “Yes, I want to go (skiing).”
The snow was fantastic, with two feet of fresh powder and plenty of avalanche danger. Hanging out with the software development personalities was affirming, inspiring, and thought-provoking. There were many competing and complementary perspectives discussed: code-centric, model-centric, and people-centric. Rather than trying to define the one true software development methodology and practice, someone had the bright idea that we should make a statement of the things we have in common. With some good discussion, careful word tweaking, and the snow falling, the four values were distilled.
Before we could hit the slopes, we needed one more thing, a name. Lightweight was the banner we flew under on our way to Snowbird, but who wants to be known as a lightweight? Good point; we could not have that. After a while we settled on Agile Software Development. Of course organizations would want to be agile. You can explain that to your CEO or customer far more easily than proclaiming you are extreme.
We had a name, we made a statement, but we were not sure if anyone would know or care. Over the following years we found that people did care. Many creative and passionate people were attracted to Agile, and Agile continues to grow in popularity and depth. After ten years, you can see that much has changed, but much remains the same.
Many developers are still unaware of Agile, or only know the misconceptions. The idea that more upfront work is needed is deeply ingrained in the software development mindset. Code bases are a mess, dragging teams and products to a standstill.
Happily, people are discovering agile everyday. Embedded development, where I spend much of my time, is awakening to Agile with interest—and skepticism. The questions and concerns are essentially the same that non-embedded application developers cite, though on the surface they sound very different. Dependence on hardware is a big conceptual hurdle for embedded developers, but non-embedded developers have dependencies on UIs and databases. Those sound like very different problems, but they have similar solutions in the technical practices of XP and Object Oriented Design.
Incremental delivery is another conceptual hurdle. How can you deliver a washing machine incrementally? Embedded development is not the only place where product completeness requires that the top ten (priority 1) features must be in the product for it to be useful or marketable. But even when value cannot be delivered incrementally, we can still benefit from measurable, visible, and incremental progress.
There is at least one new problem ten years after. Now we have organizations that have a few Scrum Masters and proclaim themselves Agile, but that continue to spend months in analysis and design, and similar amounts of effort in test and fix. They have stories and iterations, but ignore relative effort estimates and velocity. Code is deteriorating. Tests are not written. And they wonder why Agile is not working for them.
Change is difficult, but the only way to improve is to change. I show teams the careful way of working, with clean code, tight feedback, and plenty of tests. They say “That takes so long. We want to know the fast way, show us the fast way,” as if I am holding back some secret. The careful way is the fast way. Get good at careful; that’s how you can go fast. My hope for the next ten years is that more people accept this tough lesson. Now it’s time to get out the ski gear for a tenth anniversary trip to Snowbird. I’m going for the skiing and looking forward to the next ten years. Will there be another revolution? Maybe, but I know for sure there will be more evolution.
Hard Times Call for Agility
Arie van Bennekum
Ten years ago I was part of the initiative that led to the Agile Alliance. At that time I was already totally into DSDM (since 1994) and what we now call “agile” had become part of my way of thinking. Not only in software projects, but also in other kind of projects and even outside work —things like prioritizing, prototyping, and iterating to converge to a well-fitting solution.
At the customer site the business case way of thinking, including prioritization techniques like MoSCoW, were understood, but the urgency was not felt. I remember for me it felt like being on trial again, explaining to the customer how he could and should save lots of money and time in using agile methods…. And the answer was always “Well, this is not really for us,” or “Well, it works well but it is to difficult for us.”
Now we have a credit crunch torturing the world. Business and government feel they really have to execute projects in a slim way. Nowadays where I live the need for doing the right things instead of doing things right surfaces a lot. Nevertheless, old ways of thinking in project management are still in the lead. “Just squeeze your project members and you will get there” seems to be the line.
Agile has evolved. DSDM has grown into Atern. It covers business-case thinking in a fast-changing and agile environment. Agile principles in projects need to be applied more than ever. We have many miles ahead of us on this road....
I Hoped to Derail Their Evil Plans
Stephen J. Mellor
I was astonished to be invited to what became the meeting that originated the Agile Manifesto because my work had always been based around building models. The Structured Development for Real-Time Systems trilogy with Paul Ward and the pair of Object-Oriented Analysis books with Sally Shlaer all focused on analysis and design, placing much less emphasis on the coding and testing favored by agile practitioners.
Moreover, I had recently read Kent Beck’s Extreme Programming and I was horrified by the lack of emphasis on up-front thinking, abhorrence of models and the deprecation of documentation. However, it was clear that lightweight, extreme, or agile approaches were gaining traction, so I resolved to attend the meeting. The fact that it was in the Rockies in winter had nothing to do with it.
I introduced myself as a spy who hoped to derail their evil plans. While I found many of the positions taken regarding process to be sound (talking to the customer, for example, or timeboxing), the rejection of models was surely unjustified. Yes, there had been an overemphasis on “big design up front,” but surely models had some value. Yes, writing documents (and models) had come to be seen as an end in itself, but surely the correct response was not to throw the productivity gains from modeling out with the dirty documentation.
So just why were models so bad? Because they don’t execute, I was told. Yet my work over the preceding decade had been all about building executable models. Over the few days of the meeting I had a similar conversation with almost all of the signatories, sometimes more than once.
This intellectual passing-in-the-night came about because we had differing ideas about what the word “model” means. Some signatories viewed models (if they thought about them at all) as sketches, to be drawn on napkins for communication purposes and then thrown away. What drew the most ire was the view of a model as a blueprint to be thrown over the wall to developers who did as they were told. I held neither of those views. I thought models ran.
Though we were executing models in 2001, it was with our own action language. I heard repeatedly at the meeting that you could not write a program to say “Hello World!” in the Unified Modeling Language (UML). And although you could, in fact, say “Hello World!” in UML, it certainly wasn’t easy. In fact, it was an advertisement for writing code. This had to be addressed if modeling was to become widely viewed as executable.
In 1997, when the UML was accepted as a standard, there were but seven actions you could carry out in a model, and one of those was “uninterpreted string.” (You can certainly forgive the signatories for thinking that models couldn’t execute!) I proposed a standard Action Language for UML. Some folk immediately suggested standardizing on Java. Or Smalltalk. Or....
But an existing programming language is insufficient because it works at too low a level of abstraction. It therefore mixes (to some degree) application concepts with their implementation. For example, when we build a particular data structure (a list of Customers, say, with pointers to lists of Accounts), we have made a decision about implementation. In the UML, we would instead express this is as an association, but we would not specify its implementation.
Moreover, when we come to operate on this data in code, we have to take into consideration the implementation we have selected. For example, to sum up all the account balances for a specific customer, we are required to traverse the linked list. It would be simpler to say (somehow) “sum all of the account balances for this customer.”
It’s not so much that we want to be able to execute models in UML—we could just add code to achieve that. It’s that we want the higher productivity that comes from operating at a higher level of abstraction away from the implementation. We also want to be able to change our minds about the implementation without changing the description of the problem. The UML model would have the same application meaning if we put Customers and Accounts into a database implementation.
Over the following twelve years, we (the members of the Object Management Group, the organization that standardized UML, and especially Ed Seidewitz from Model Driven Solutions) have produced a series of standards that define the kinds of actions permissible in UML; defined a subset of UML for execution; defined the semantics of that subset precisely; and (finally!) defined a standard Action Language for UML.
In that language, the example above would be written:
where “reduce” is an operation that “reduces” the collection of balances produced by traversing from myCustomer to all their accounts by applying a binary operation, in this case ‘+’ repeatedly until only a single value remains. Of course, that can then be translated into an implementation using linked lists. Or a database. Or anything else. There are no lists in the specification, nor any traversal of them, just a very compact statement of what needs to be done. (By the way, one design requirement for the language was that it be easily accessible to Java programmers, so much of the syntax looks like Java. The example here was chosen not to obfuscate but to illustrate the power of moving away from the implementation.)
The other signatories were kind enough, back in 2001, to write the Manifesto using the word “software” (which can include executable models), not “code” (which is more specific.) So I felt able, in good conscience, to become a signatory to the Manifesto while continuing to promote executable modeling. Ten years on we have a standard action language for agile modeling.
We rarely see the words “agile” and “model” in the same sentence, but they are not at all in conflict. Rather, modelers can learn a lot from the agile folk (building tests for models early, for example) and those following an agile process can benefit from the increased productivity and eased customer communication that come from sharing executable diagrams with their pair-modeling customers. Surely that’s a win for everyone.
For more detailed information, including critical references, look here.
How Will It Survive?
We wanted people to read code and learn from it. We wanted a canon for our industry. We asked that people program together and reflect on the experience, to learn a new vocabulary from the experience, a vocabulary of patterns, a vocabulary of collaboration.
Now everyone wants to collaborate with programmers. Programmers have a new-found reputation for getting things done. It’s refreshing. But it’s diluting the canon, our young canon. How will it survive?
You Can’t Do Agile
Mike Swaine contacted me for my thoughts on “Agile at 10.” My initial thoughts were slightly churlish—there’s no such thing as Agile, and whatever there is, it isn’t 10 years old.
First, there’s this growing tendency to treat the word “Agile” as a noun. People say “we’re doing agile.” But friends, agile is not a noun. It’s an adjective, meaning to be able to move quickly and easily. When we first coined the term back in our Snowbird meeting as a way of describing what we did, we felt that the word “agile” also carried overtones of grace.
So, for me at least, you really can’t use “agile” without having it qualify some kind of noun—it’s the process, or the thinking, or the interactions that are agile. And that’s something I’d like us all to refocus on as we reflect back and plan forward. Being an agile software developer means doing things in an agile manner, not just naming things that you do “agile.” And that’s what makes this all so tough, but also so effective. You can’t just buy an agile toolkit, or stick up agile posters, or pay for some agile consultant. Ultimately what makes the things you do agile is the way you do them. And, in my experience, if you continue to work to make the doing of those things quicker and easier (over the long term), you’ll be making yourself more agile.
Think about that for just a second. You can’t do “agile” and make yourself better. You can only do things better and in the process become more agile.
The other problem with “Agile at 10” is that agile processes are not really ten years old. When we seventeen middle-aged white guys met in Snowbird in February, 2001, we didn’t invent agility. All we did was give a name to a set of values that encompassed the way a whole bunch of people (including ourselves) were already producing software. I dare say much of the work at Xerox PARC 25 years before our meeting would be considered agile. I know that much of the software development that was being done in the 90s was. So, in a way, all we did was to provide the seed crystal of a name—the subsequent growth came from all those thousands of teams that had already been doing things in an agile way. Our seed gave them a name and a set of four values on which to hang their existing practices.
I’m proud that much of our industry has used the Agile Manifesto as the starting point from which to explore better ways of creating software. I’m impressed by people who have used the term to help convince their companies to try new techniques. But mostly, I’m just blown away by the enthusiasm with which the people in our industry constantly strive to do things better. These are the people we should be celebrating on the 10th anniversary of the Snowbird meeting. Thank you all for continuing to make it interesting and fun.