Agile -- Fake Solution to an Important Problem

News

Software Life Cycle Models Recommended Links Parkinson Law Is DevOps a yet another "for profit" technocult Prototyping Spiral model
Cargo cult programming Science, PseudoScience and Society Lysenkoism Information Technology Hype Software Fashion Bootlickocracy  CMM (Capability Maturity Model)
Brooks law Conway Law Relationship of Brooks Law and Conway Law Software realism vs software idealism The Mythical Man-Month The True Believer The Peter Principle
Program Understanding OSS development model Project Management Reverse Engineering Links SE quotes Humor Etc

As military generals know the plans of battle usually are abandoned after the first encounter with reality. But that does not mean that they are useless. Much depends of the talent of particular military leader.

Similarly prototyping is a perfect method of getting a viable system, but it is often too expensive. That's why palliatives like Agile proposed are essentially a shortcut to the real thing -- prototyping.

Agile also deny the existence of software architecture.  This is actually a fatal drawback that  makes is dangerous techno-cult.  It evloved based on  easier extreme programming fad. Most noise about XP was around 2000-2003. XP proponents quickly realized that just holding  conferences, seminal and publishing books can be very profitable activity. In this sense XP is an early examples of "for profit" techno-cult. XP gradually faded and I did not noticed any books about it published in 2008. But then is re-emerged as a part of Agile software Development parts of which later metamorphosed into DevOps  -- a more modern (as of 2017 I would say "current")  among   "for profit" software development techno-cults. Like with any religion there  are some rational parts but  they are outshined  by irrational. 

Wikipedia mentioned that extreme programming is a now considered to be a form of "agile development" (that's an interesting term, because opposite to agile development is probably "stagnant software development" :-). This proximity gives "agile development" term bad breath despite some sound ideas in the agile manifesto (which by-and-large looks like plagiarism of  open source software development methodology). 

One important sign of techno-cult is rejection of previous achievements and attempt to ignore them and pretend this this is novel approach.  Both Extreme programming and Agile development fits the bill.  Books  about  agile usually solidly ignorer history of software development methodologies and books that does not fit the proposed "software development nirvana". 

IBM was at one time very active in milking Agile  cows via its Rational franchise (note the title practice leader for agile development below -- we can observe emergence of Agile clergy):

In a keynote presentation and in correspondence afterward, IBM's Per Kroll, chief architect for IBM Rational Expertise Development & Innovation, cited the company's proliferation of agile practices among some of its approximately 35,000 developers.

"I think in general we have 1,000-plus people [using] agile in the IBM community," with an additional 2,000 persons trained in agile practices, Kroll said. He said, though, that he did not anticipate that all 35,000 developers would adopt agile.

On Monday at the conference, IBM's Scott Ambler, practice leader for agile development, spoke on the state of the agile space. Agile development focuses on developing software in iterations of usually two weeks. This allows for more flexibility than the traditional waterfall methodologies.

"Agile is taking many things that we’ve been using for a lot of years before agile was around [and] adding a lot of new things and putting a very good spin on it and making development more human," said Kroll. Concepts like iterative development and continuous integration are not new, but agile accounts for factors like human and collaborative aspects, he said. "We firmly believe, and our executives firmly believe, that the most successful organizations of tomorrow will be the ones [able] to adopt agile principles."

IBM has roughly 150 agile projects, he said. But each team needs to determine its own agile use based on their context. Agile means a lot of things to different teams, he said.

A key factor is that IBM helps customers and its own teams with applying agile complex environments, facing such issues as application complexity, scalability, and compliance issues, said Kroll.

That's sign that transformation of IBM into money milking and outsourcing behemoth, which exist to enrich shareholders and thin layer of top executives. Classic IBM which was the company which created IBM/360 series mainframes and personal PC was destroyed by infamous Lou "the last thing IBM needs right now is a vision" Gerstner . He was one of the first neoliberal honchos at the helm of large and respectable software/hardware company. Coming, as you can expect, from the financial company -- American Express -- which, if we allow to bypass political correctness, is somewhat close to a legitimized usury company; or at least strives to emulate this harpies by pushing debt on its customers (do you want $10K at zero percent for one year, 28% after ?)  in a hope that they will not repay it in full and will be milked for months to come. He switched IBM from providing technical value to providing "shareholder value" (and first of all for himself; in 2002 when he left IBM he was worth $630 million). Of course "shareholder value" is a classic neoliberal mantra that hides the idea of redistribution of wealth up, but at this time it was very diffluent to fight, as his term (1993-2002) coincided with the triumphant march of neoliberalism all over the globe (the USSR collapsed in 1991).  But I digress.

Among  perversions introduced by Agile software development methodology we can mentions pair programming, fussiness of interactions with customers (which, typically, are far from knowing what they really want, and given too much influence tend to screw the development by introducing too many changes in the process), disreragd for architecture and documentation, and too much emphasis on their superiority with waterfall model (which is not an achievement and in certain cases is not given).  Actually waterfall software development serves as a scapegoat in Agile technocult.  

The problem they are trying to solve is real -- lack of communication and understanding each other code in large softwaredevelopment projects lead to more and more nasty bugs and delays in complex software projects, so "forcing reading each other code" might be worthwhile exercise, if done intelligently (reading of some developmenrs code can tach you nothing and actually is harmful for one;s mental helath ;-).

The key word here is "intelligently". "Pair programming" is definitely not an intelligent way to enhance communication between key developers and improve knowledge of each other code. It is a pervert "social engineering"  practice which openly insulting to one's intelligence and the level of "management enforced" social interactions. for talanted developers (which are way too few in any large software project and as such are the most valuable commodity) this not only waist of time, but also somewhat denigrating experience. To me it looks like a parody of Brooks  idea of "surgical team" (see The Mythical Man-Month). 

Looks like those "ballroom" generals who promote Agile really does not understand that real challenges of really big and complex software project: the scarcity of the top talent.

Looks like those "ballroom" generals who promote Agile really does not understand that real challenges of really big and complex software project: the scarcity of the top talent.
Agile used to be a fashionable software development methodology (but in comparison with only methodology called software prototpying it looks like  Cargo cult programming)  with elements of "for profit" technocult probably until 2015 or so. So in was in play for 15 years or so !  After 2015 it was by-and-large superseded by DevOps hoopla, which produces higher profits for high priests of this technocult (books, consulting, lecture, conferences). 

Agile methodology exhibits typical properties of the past management fads (see Software Fashion and Information Technology Hype)  including but not limited to:

  1. Elaborate new jargon which obscure rather then clarify as is as close to Corporate bullshit, as one can get, 
  2. A set of gurus of the new cult, exclusion and active suppression of negative evidence,
  3. Expensive external consultants with "make money fast" mentality, who specialize in its implementation, bold claims of a measurable business improvement via measurement of fake metrics such as velocity (miracles of the new pseudo religious cult).

In other words it promises "Nirvana" for software development. But in reality software development is the activity is highly dependent of the level of talent of both the leader of the project and individual developers, as well as motivation and morale of the whole team. The latter is a huge problem in neoliberal software development firms with thier excessive accent of quantification of result and performance reviews as well as outsourcing and offshoring of development (often to Idia)  In this sense software development team behaves more like a military unit in the battle.  And it is well know fact that disgruntled military units sometimes shoot their commanding  officers (Vietnam).

Another problem of Agile is giving users of software too much freedom for changing specifications. While some degree of flexibility is necessary, because in software development  "appetite comes with the food", and while up to certain point user input is justified,  exceeding this level is very costly and mostly counterproductive.

Contrary Agile naive perception, the users of software are not positive heroes of this software development drama. In this play usually there is no positive heroes, or if they exist they tend to be among software developers, who often sacrifices their health, social life and family (if they have one)  to deliver short lived artifacts -- the software systems with high quality and, occasionally, even on time. Like solders after battle they usually getting nothing for their valiant efforts. Generals -- software project managers often get all the glory in such cases. Who BTW most often are either corrupt careerists, ready to sacrifices people for their petty goals of promotion, or just incompetents, who are not up to the task and were saved by valiant  efforts of rank-and-file developers, half of them would be not needs if they were a little bit more intelligent. 

From Wikipedia (please  not that Wikipedia is incorrect -- waterfall software development  was rejected around the time of the publication of Mythical Man month and its defects were known in late 60th)

Incremental software development methods have been traced back to 1957.[2] In 1974, a paper by E. A. Edmonds introduced an adaptive software development process.[3] Concurrently and independently the same methods were developed and deployed by the New York Telephone Company's Systems Development Center under the direction of Dan Gielan. In the early 1970s, Tom Gilb started publishing the concepts of Evolutionary Project Management (EVO), which has evolved into Competitive Engineering.[4] During the mid to late 1970s Gielan lectured extensively throughout the U.S. on this methodology, its practices, and its benefits.

So-called lightweight agile software development methods evolved in the mid-1990s as a reaction against the heavyweight waterfall-oriented methods, which were characterized by their critics as being heavily regulated, regimented, micromanaged and overly incremental approaches to development.

Proponents of lightweight agile methods contend that they are a return to development practices that were present early in the history of software development.[2]

Early implementations of agile methods include Rational Unified Process (1994), Scrum (1995), Crystal Clear, Extreme Programming (1996), Adaptive Software Development, Feature Driven Development, and Dynamic Systems Development Method (DSDM) (1995). These are now collectively referred to as agile methodologies, after the Agile Manifesto was published in 2001.[5]

Agile Manifesto

In February 2001, 17 software developers[6] met at the Snowbird, Utah, resort, to discuss lightweight development methods. They published the Manifesto for Agile Software Development[1] to define the approach now known as agile software development. Some of the manifesto's authors formed the Agile Alliance, a nonprofit organization that promotes software development according to the manifesto's principles.

The Agile Manifesto reads, in its entirety, as follows:

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
That is, while there is value in the items on the right, we value the items on the left more.[1]

The meanings of the manifesto items on the left within the agile software development context are described below:

The Agile Manifesto is based on twelve principles[8]:

In 2005, a group headed by Alistair Cockburn and Jim Highsmith wrote an addendum of project management principles, the Declaration of Interdependence,[9] to guide software project management according to agile development methods.

Pair programming, an agile development technique used by XP. Note information radiators in the background.

There are many specific agile development methods. Most promote development, teamwork, collaboration, and process adaptability throughout the life-cycle of the project.

Agile methods break tasks into small increments with minimal planning and do not directly involve long-term planning. Iterations are short time frames (timeboxes) that typically last from one to four weeks. Each iteration involves a cross functional team working in all functions: planning, requirements analysis, design, coding, unit testing, and acceptance testing. At the end of the iteration a working product is demonstrated to stakeholders. This minimizes overall risk and allows the project to adapt to changes quickly. An iteration might not add enough functionality to warrant a market release, but the goal is to have an available release (with minimal bugs) at the end of each iteration.[10] Multiple iterations might be required to release a product or new features.

Team composition in an agile project is usually cross-functional and self-organizing, without consideration for any existing corporate hierarchy or the corporate roles of team members. Team members normally take responsibility for tasks that deliver the functionality an iteration requires. They decide individually how to meet an iteration's requirements.

Agile methods emphasize face-to-face communication over written documents when the team is all in the same location. Most agile teams work in a single open office (called a bullpen), which facilitates such communication. Team size is typically small (5-9 people) to simplify team communication and team collaboration. Larger development efforts can be delivered by multiple teams working toward a common goal or on different parts of an effort. This might require a coordination of priorities across teams. When a team works in different locations, they maintain daily contact through videoconferencing, voice chat, e-mail, etc.

No matter what development disciplines are required, each agile team will contain a customer representative. This person is appointed by stakeholders to act on their behalf[11] and makes a personal commitment to being available for developers to answer mid-iteration questions. At the end of each iteration, stakeholders and the customer representative review progress and re-evaluate priorities with a view to optimizing the return on investment (ROI) and ensuring alignment with customer needs and company goals.

Most agile implementations use a routine and formal daily face-to-face communication among team members. This specifically includes the customer representative and any interested stakeholders as observers. In a brief session, team members report to each other what they did the previous day, what they intend to do today, and what their roadblocks are. This face-to-face communication exposes problems as they arise. "These meetings, sometimes referred as daily stand-ups or daily scrum meetings, are held at the same place and same time every day and should last no more than 15 minutes. Standing up usually enforces that rule."[12]

Agile development emphasizes working software as the primary measure of progress. This, combined with the preference for face-to-face communication, produces less written documentation than other methods. The agile method encourages stakeholders to prioritize "wants" with other iteration outcomes, based exclusively on business value perceived at the beginning of the iteration (also known as value-driven).[13]

Specific tools and techniques, such as continuous integration, automated or xUnit test, pair programming, test-driven development, design patterns, domain-driven design, code refactoring and other techniques are often used to improve quality and enhance project agility.

Light Agile Development (LAD) is a flavor of agile methodology that applies hand picked techniques from the wider range of agile practices to suit different companies, development teams, situations and environments. Another key aspect of LAD is that it tends to be user-centric, focusing primarily on the user experience and usable software interfaces and uses agile methodologies to deliver them. Most real-world implementations of Agile are really LAD in practice, since a core value of the method is flexibility, sensibility, and focus on getting stuff done.[citation needed]

In agile software development, an information radiator is a (normally large) physical display located prominenly in an office, where passers-by can see it. It presents an up-to-date summary of the status of a software project or other product.[14][15] The name was coined by Alistair Cockburn, and described in his 2002 book Agile Software Development.[15] A build light indicator may be used to inform a team about the current status of their project.

Comparison with other methods

Methods exist on a continuum from adaptive to predictive.[16] Agile methods lie on the adaptive side of this continuum. Adaptive methods focus on adapting quickly to changing realities. When the needs of a project change, an adaptive team changes as well. An adaptive team will have difficulty describing exactly what will happen in the future. The further away a date is, the more vague an adaptive method will be about what will happen on that date. An adaptive team cannot report exactly what tasks they will do next week, but only which features they plan for next month. When asked about a release six months from now, an adaptive team might be able to report only the mission statement for the release, or a statement of expected value vs. cost.

The diagram below displays the differences between agile and waterfall development processes. By delivering working, tested, deployable software on an incremental basis, agile development delivers increased value, visibility, and adaptability much earlier in the life cycle, significantly reducing project risk.[17]


Top Visited
Switchboard
Latest
Past week
Past month

NEWS CONTENTS

Old News ;-)

[Oct 26, 2017] Amazon.com Customer reviews Extreme Programming Explained Embrace Change

Rapid Development by Steve McConnell is an older and better book. The Mythical Man-Month remains valuable book as well, albeit dated.
Notable quotes:
"... Having a customer always available on site would mean that the customer in question is probably a small, expendable fish in his organization and is unlikely to have any useful knowledge of its business practices. ..."
"... Unit testing code before it is written means that one would have to have a mental picture of what one is going to write before writing it, which is difficult without upfront design. And maintaining such tests as the code changes would be a nightmare. ..."
"... Programming in pairs all the time would assume that your topnotch developers are also sociable creatures, which is rarely the case, and even if they were, no one would be able to justify the practice in terms of productivity. I won't discuss why I think that abandoning upfront design is a bad practice; the whole idea is too ridiculous to debate ..."
"... Both book and methodology will attract fledgling developers with its promise of hacking as an acceptable software practice and a development universe revolving around the programmer. It's a cult, not a methodology, were the followers shall find salvation and 40-hour working weeks ..."
"... Two stars for the methodology itself, because it underlines several common sense practices that are very useful once practiced without the extremity. ..."
"... The second is the dictatorial social engineering that eXtremity mandates. I've actually tried the pair programming - what a disaster. ..."
"... I've also worked with people who felt that their slightest whim was adequate reason to interfere with my work. That's what Beck institutionalizes by saying that any request made of me by anyone on the team must be granted. It puts me completely at the mercy of anyone walking by. The requisite bullpen physical environment doesn't work for me either. I find that the visual and auditory distraction make intense concentration impossible. ..."
"... One of the things I despise the most about the software development culture is the mindless adoption of fads. Extreme programming has been adopted by some organizations like a religious dogma. ..."
Oct 26, 2017 | www.amazon.com

Mohammad B. Abdulfatah on February 10, 2003

Programming Malpractice Explained: Justifying Chaos

To fairly review this book, one must distinguish between the methodology it presents and the actual presentation. As to the presentation, the author attempts to win the reader over with emotional persuasion and pep talk rather than with facts and hard evidence. Stories of childhood and comradeship don't classify as convincing facts to me.

A single case study-the C3 project-is often referred to, but with no specific information (do note that the project was cancelled by the client after staying in development for far too long).

As to the method itself, it basically boils down to four core practices:

  1. Always have a customer available on site.
  2. Unit test before you code.
  3. Program in pairs.
  4. Forfeit detailed design in favor of incremental, daily releases and refactoring.

If you do the above, and you have excellent staff on your hands, then the book promises that you'll reap the benefits of faster development, less overtime, and happier customers. Of course, the book fails to point out that if your staff is all highly qualified people, then the project is likely to succeed no matter what methodology you use. I'm sure that anyone who has worked in the software industry for sometime has noticed the sad state that most computer professionals are in nowadays.

However, assuming that you have all the topnotch developers that you desire, the outlined methodology is almost impossible to apply in real world scenarios. Having a customer always available on site would mean that the customer in question is probably a small, expendable fish in his organization and is unlikely to have any useful knowledge of its business practices.

Unit testing code before it is written means that one would have to have a mental picture of what one is going to write before writing it, which is difficult without upfront design. And maintaining such tests as the code changes would be a nightmare.

Programming in pairs all the time would assume that your topnotch developers are also sociable creatures, which is rarely the case, and even if they were, no one would be able to justify the practice in terms of productivity. I won't discuss why I think that abandoning upfront design is a bad practice; the whole idea is too ridiculous to debate.

Both book and methodology will attract fledgling developers with its promise of hacking as an acceptable software practice and a development universe revolving around the programmer. It's a cult, not a methodology, were the followers shall find salvation and 40-hour working weeks.

Experience is a great teacher, but only a fool would learn from it alone. Listen to what the opponents have to say before embracing change, and don't forget to take the proverbial grain of salt.

Two stars out of five for the presentation for being courageous and attempting to defy the standard practices of the industry. Two stars for the methodology itself, because it underlines several common sense practices that are very useful once practiced without the extremity.

wiredweird HALL OF FAME TOP 1000 REVIEWER on May 24, 2004
eXtreme buzzwording

Maybe it's an interesting idea, but it's just not ready for prime time.

Parts of Kent's recommended practice - including aggressive testing and short integration cycle - make a lot of sense. I've shared the same beliefs for years, but it was good to see them clarified and codified. I really have changed some of my practice after reading this and books like this.

I have two broad kinds of problem with this dogma, though. First is the near-abolition of documentation. I can't defend 2000 page specs for typical kinds of development. On the other hand, declaring that the test suite is the spec doesn't do it for me either. The test suite is code, written for machine interpretation. Much too often, it is not written for human interpretation. Based on the way I see most code written, it would be a nightmare to reverse engineer the human meaning out of any non-trivial test code. Some systematic way of ensuring human intelligibility in the code, traceable to specific "stories" (because "requirements" are part of the bad old way), would give me a lot more confidence in the approach.

The second is the dictatorial social engineering that eXtremity mandates. I've actually tried the pair programming - what a disaster. The less said the better, except that my experience did not actually destroy any professional relationships. I've also worked with people who felt that their slightest whim was adequate reason to interfere with my work. That's what Beck institutionalizes by saying that any request made of me by anyone on the team must be granted. It puts me completely at the mercy of anyone walking by. The requisite bullpen physical environment doesn't work for me either. I find that the visual and auditory distraction make intense concentration impossible.

I find revival tent spirit of the eXtremists very off-putting. If something works, it works for reasons, not as a matter of faith. I find much too much eXhortation to believe, to go ahead and leap in, so that I will eXperience the wonderfulness for myself. Isn't that what the evangelist on the subway platform keeps saying? Beck does acknowledge unbelievers like me, but requires their exile in order to maintain the group-think of the X-cult.
Beck's last chapters note a number of exceptions and special cases where eXtremism may not work - actually, most of the projects I've ever encountered.

There certainly is good in the eXtreme practice. I look to future authors to tease that good out from the positively destructive threads that I see interwoven.

A customer on May 2, 2004
A work of fiction

The book presents extreme programming. It is divided into three parts:
(1) The problem
(2) The solution
(3) Implementing XP.

The problem, as presented by the author, is that requirements change but current methodologies are not agile enough to cope with this. This results in customer being unhappy. The solution is to embrace change and to allow the requirements to be changed. This is done by choosing the simplest solution, releasing frequently, refactoring with the security of unit tests.

The basic assumption which underscores the approach is that the cost of change is not exponential but reaches a flat asymptote. If this is not the case, allowing change late in the project would be disastrous. The author does not provide data to back his point of view. On the other hand there is a lot of data against a constant cost of change (see for example discussion of cost in Code Complete). The lack of reasonable argumentation is an irremediable flaw in the book. Without some supportive data it is impossible to believe the basic assumption, nor the rest of the book. This is all the more important since the only project that the author refers to was cancelled before full completion.

Many other parts of the book are unconvincing. The author presents several XP practices. Some of them are very useful. For example unit tests are a good practice. They are however better treated elsewhere (e.g., Code Complete chapter on unit test). On the other hand some practices seem overkill. Pair programming is one of them. I have tried it and found it useful to generate ideas while prototyping. For writing production code, I find that a quiet environment is by far the best (see Peopleware for supportive data). Again the author does not provide any data to support his point.

This book suggests an approach aiming at changing software engineering practices. However the lack of supportive data makes it a work of fiction.
I would suggest reading Code Complete for code level advice or Rapid Development for management level advice.

A customer on November 14, 2002
Not Software Engineering.

Any Engineering discipline is based on solid reasoning and logic not on blind faith. Unfortunately, most of this book attempts to convince you that Extreme programming is better based on the author's experiences. A lot of the principles are counterintuitive and the author exhorts you just try it out and get enlightened. I'm sorry but these kind of things belong in infomercials not in s/w engineering.

The part about "code is the documentation" is the scariest part. It's true that keeping the documentation up to date is tough on any software project, but to do away with documentation is the most ridiculous thing I have heard.

It's like telling people to cut of their noses to avoid colds. Yes we are always in search of a better software process. Let me tell you that this book won't lead you there.

Philip K. Ronzone on November 24, 2000
The "gossip magazine diet plans" style of programming.

This book reminds me of the "gossip magazine diet plans", you know, the vinegar and honey diet, or the fat-burner 2000 pill diet etc. Occasionally, people actually lose weight on those diets, but, only because they've managed to eat less or exercise more. The diet plans themselves are worthless. XP is the same - it may sometimes help people program better, but only because they are (unintentionally) doing something different. People look at things like XP because, like dieters, they see a need for change. Overall, the book is a decently written "fad diet", with ideas that are just as worthless.

A customer on August 11, 2003
Hackers! Salvation is nigh!!

It's interesting to see the phenomenon of Extreme Programming happening in the dawn of the 21st century. I suppose historians can explain such a reaction as a truly conservative movement. Of course, serious software engineering practice is hard. Heck, documentation is a pain in the neck. And what programmer wouldn't love to have divine inspiration just before starting to write the latest web application and so enlightened by the Almighty, write the whole thing in one go, as if by magic? No design, no documentation, you and me as a pair, and the customer too. Sounds like a hacker's dream with "Imagine" as the soundtrack (sorry, John).
The Software Engineering struggle is over 50 years old and it's only logical to expect some resistance, from time to time. In the XP case, the resistance comes in one of its worst forms: evangelism. A fundamentalist cult, with very little substance, no proof of any kind, but then again if you don't have faith you won't be granted the gift of the mystic revelation. It's Gnosticism for Geeks.
Take it with a pinch of salt.. well, maybe a sack of salt. If you can see through the B.S. that sells millions of dollars in books, consultancy fees, lectures, etc, you will recognise some common-sense ideas that are better explained, explored and detailed elsewhere.

Ian K. VINE VOICE on February 27, 2015
Long have I hated this book

Kent is an excellent writer. He does an excellent job of presenting an approach to software development that is misguided for anything but user interface code. The argument that user interface code must be gotten into the hands of users to get feedback is used to suggest that complex system code should not be "designed up front". This is simply wrong. For example, if you are going to deploy an application in the Amazon Cloud that you want to scale, you better have some idea of how this is going to happen. Simply waiting until your application falls over and fails is not an acceptable approach.

One of the things I despise the most about the software development culture is the mindless adoption of fads. Extreme programming has been adopted by some organizations like a religious dogma.

Engineering large software systems is one of the most difficult things that humans do. There are no silver bullets and there are no dogmatic solutions that will make the difficult simple.

Anil Philip on March 24, 2005
not found - the silver bullet

Maybe I'm too cynical because I never got to work for the successful, whiz-kid companies; Maybe this book wasn't written for me!

This book reminds me of Jacobsen's "Use Cases" book of the 1990s. 'Use Cases' was all the rage but after several years, we slowly learned the truth: Uses Cases does not deal with the architecture - a necessary and good foundation for any piece of software.

Similarly, this book seems to be spotlighting Testing and taking it to extremes.

'the test plan is the design doc'

Not True. The design doc encapsulates wisdom and insight

a picture that accurately describes the interactions of the lower level software components is worth a thousand lines of code-reading.

Also present is an evangelistic fervor that reminds me of the rah-rah eighties' bestseller, "In Search Of Excellence" by Peters and Waterman. (Many people have since noted that most of the spotlighted companies of that book are bankrupt twenty five years later).

Lastly, I noted that the term 'XP' was used throughout the book, and the back cover has a blurb from an M$ architect. Was it simply coincidence that Windows shares the same name for its XP release? I wondered if M$ had sponsored part of the book as good advertising for Windows XP! :)

[Oct 03, 2017] Silicon Valley is turning into a series of micromanaged sweatshops (that's what "agile" is truly all about)

Notable quotes:
"... We've seen this kind of tactic for some time now. Silicon Valley is turning into a series of micromanaged sweatshops (that's what "agile" is truly all about) with little room for genuine creativity, or even understanding of what that actually means. I've seen how impossible it is to explain to upper level management how crappy cheap developers actually diminish productivity and value. All they see is that the requisition is filled for less money. ..."
Oct 03, 2017 | discussion.theguardian.com

mlzarathustra , 21 Sep 2017 16:52

I agree with the basic point. We've seen this kind of tactic for some time now. Silicon Valley is turning into a series of micromanaged sweatshops (that's what "agile" is truly all about) with little room for genuine creativity, or even understanding of what that actually means. I've seen how impossible it is to explain to upper level management how crappy cheap developers actually diminish productivity and value. All they see is that the requisition is filled for less money.

The bigger problem is that nobody cares about the arts, and as expensive as education is, nobody wants to carry around a debt on a skill that won't bring in the bucks. And smartphone-obsessed millennials have too short an attention span to fathom how empty their lives are, devoid of the aesthetic depth as they are.

I can't draw a definite link, but I think algorithm fails, which are based on fanatical reliance on programmed routines as the solution to everything, are rooted in the shortage of education and cultivation in the arts.

Economics is a social science, and all this is merely a reflection of shared cultural values. The problem is, people think it's math (it's not) and therefore set in stone.

[May 17, 2017] Who really gives a toss if it's agile or not

Notable quotes:
"... According to sources, hundreds of developers were employed on the programme at huge day rates, with large groups of so-called agile experts overseeing the various aspects of the programme. ..."
"... I have also worked on agile for UK gov projects a few years back when it was mandated for all new projects and I was at first dead keen. However, it quickly become obvious that the lack of requirements, specifications etc made testing a living nightmare. Changes asked for by the customer were grafted onto what become baroque mass of code. I can't see how Agile is a good idea except for the smallest trivial projects. ..."
"... The question is - is that for software that's still in development or software that's deployed in production? If it's the latter and your "something" just changes its data format you're going to be very unpopular with your users. And that's just for ordinary files. If it requires frequent re-orgs of an RDBMS then you'd be advised to not go near any dark alley where your DBA might be lurking. ..."
"... Software works on data. If you can't get the design of that right early you're going to be carrying a lot of technical debt in terms of backward compatibility or you're going to impose serious costs on your users for repeatedly bringing existing data up to date. ..."
"... At this point, courtesy of Exxxxtr3333me Programming and its spawn, 'agile' just means 'we don't want to do any design, we don't want to do any documentation, and we don't want to do any acceptance testing because all that stuff is annoying.' Everything is 'agile', because that's the best case for terrible lazy programmers, even if they're using a completely different methodology. ..."
"... It's like any exciting new methodology : same shit, different name. In this case, one that allows you to pretend the tiny attention-span of a panicking project manager is a good thing. ..."
"... Process is not a panacea or a crutch or a silver bullet. Methodologies only work as well as the people using it. Any methodology can be distorted to give the answer that upper management wants (instead of reality) ..."
"... under the guise of " agile ?". I'm no expert in project management, but I'm pretty sure it isn't supposed to be making it up as you go along, and constantly changing the specs and architecture. ..."
"... So why should the developers have all the fun? Why can't the designers and architects be "agile", too? Isn't constantly changing stuff all part of the "agile" way? ..."
May 17, 2017 | theregister.co.uk
Comment "It doesn't matter whether a cat is white or black, as long as it catches mice," according to Chinese revolutionary Deng Xiaoping.

While Deng wasn't referring to anything nearly as banal as IT projects (he was of course talking about the fact it doesn't matter whether a person is a revolutionary or not, as long as he or she is efficient and capable), the same principle could apply.

A fixation on the suppliers, technology or processes ultimately doesn't matter. It's the outcomes, stupid. That might seem like a blindingly obvious point, but it's one worth repeating.

Or as someone else put it to me recently in reference to the huge overspend on a key UK programme behind courts digitisation which we recently revealed: "Who gives a toss if it's agile or not? It just needs to work."

If you're going to do it do it right

I'm not dismissing the benefits of this particular methodology, but in the case of the Common Platform Programme , it feels like the misapplication of agile was worse than not doing it at all.

Just to recap: the CPP was signed off around 2013, with the intention of creating a unified platform across the criminal justice system to allow the Crown Prosecution Service and courts to more effectively manage cases.

By cutting out duplication of systems, it was hoped to save buckets of cash and make the process of case management across the criminal justice system far more efficient.

Unlike the old projects of the past, this was a great example of the government taking control and doing it themselves. Everything was going to be delivered ahead of time and under budget. Trebles all round!

But as Lucy Liu's O-Ren Ishii told Uma Thurman's character in in Kill Bill : "You didn't think it was gonna be that easy, did you?... Silly rabbit."

According to sources, alarm bells were soon raised over the project's self-styled "innovative use of agile development principles". It emerged that the programme was spending an awful lot of money for very little return. Attempts to shut it down were themselves shut down.

The programme carried on at full steam and by 2014 it was ramping up at scale. According to sources, hundreds of developers were employed on the programme at huge day rates, with large groups of so-called agile experts overseeing the various aspects of the programme.

CPP cops a plea

Four years since it was first signed off and what are the things we can point to from the CPP? An online make-a-plea programme which allows people to plead guilty or not guilty to traffic offences; a digital markup tool for legal advisors to record case results in court, which is being tested by magistrates courts in Essex; and the Magistrates Rota.

Multiple insiders have said the rest that we have to show for hundreds of millions of taxpayers' cash is essentially vapourware. When programme director Loveday Ryder described the project as a "once-in-a-lifetime opportunity" to modernise the criminal justice system, it wasn't clear then that she meant the programme would itself take an actual lifetime.

Of course the definition of agile is that you are able to move quickly and easily. So some might point to the outcomes of this programme as proof that it was never really about that.

One source remarked that it really doesn't matter if you call something agile or not, "If you can replace agile with constantly talking and communicating then fine, call it agile." He also added: "This was one of the most waterfall programmes in government I've seen."

What is most worrying about this programme is it may not be an isolated example. Other organisations and departments may well be doing similar things under the guise of "agile". I'm no expert in project management, but I'm pretty sure it isn't supposed to be making it up as you go along, and constantly changing the specs and architecture.

Ultimately who cares if a programme is run via a system integrator, multiple SMEs, uses a DevOps methodology, is built in-house or deployed using off-the-shelf, as long as it delivers good value. No doubt there are good reasons for using any of those approaches in a number of different circumstances.

Government still spends an outrageous amount of money on IT, upwards of £16bn a year. So as taxpayers it's a simple case of wanting them to "show me the money". Or to misquote Deng, at least show us some more dead mice. ®

Prst. V.Jeltz

Re: 'What's Real and What's for Sale'...

So agile means "constantly adapating " ? read constantly bouncing from one fuckup to the next , paddling like hell to keep up , constantly firefighting whilst going down slowly like the titanic?

thats how i read it

Dogbowl
Re: 'What's Real and What's for Sale'...

Ha! About 21 years back, working at Racal in Bracknell on a military radio project, we had a 'round-trip-OMT' CASE tool that did just that. It even generated documentation from the code so as you added classes and methods the CASE tool generated the design document. Also, a nightly build if it failed, would email the code author.

I have also worked on agile for UK gov projects a few years back when it was mandated for all new projects and I was at first dead keen. However, it quickly become obvious that the lack of requirements, specifications etc made testing a living nightmare. Changes asked for by the customer were grafted onto what become baroque mass of code. I can't see how Agile is a good idea except for the smallest trivial projects.

PatientOne
Re: 'What's Real and What's for Sale'...

"Technically 'agile' just means you produce working versions frequently and iterate on that."

It's more to do with priorities: On time, on budget, to specification: Put these in the order of which you will surrender if the project hits problems.

Agile focuses on On time. What is delivered is hopefully to specification, and within budget, but one or both of those could be surrendered in order to get something out On time. It's just project management 101 with a catchy name, and in poorly managed 'agile' developments you find padding to fit the usual 60/30/10 rule. Then the management disgard the padding and insist the project can be completed in a reduced time as a result, thereby breaking the rules of 'agile' development (insisting it's on spec, under time and under budget, but it's still 'agile'...).

Doctor Syntax
Re: 'What's Real and What's for Sale'...

"Usually I check something(s) in every day, for the most major things it may take a week, but the goal is always to get it in and working so it can be tested."

The question is - is that for software that's still in development or software that's deployed in production? If it's the latter and your "something" just changes its data format you're going to be very unpopular with your users. And that's just for ordinary files. If it requires frequent re-orgs of an RDBMS then you'd be advised to not go near any dark alley where your DBA might be lurking.

Software works on data. If you can't get the design of that right early you're going to be carrying a lot of technical debt in terms of backward compatibility or you're going to impose serious costs on your users for repeatedly bringing existing data up to date.

Doctor Syntax
Re: 'What's Real and What's for Sale'...

"On time, on budget, to specification: Put these in the order of which you will surrender if the project hits problems."

In the real world it's more likely to be a trade-off of how much of each to surrender.

FozzyBear
Re: 'What's Real and What's for Sale'...

I was told in my earlier years by a Developer.

For any project you can have it

  1. Cheap, (On Budget)
  2. Good, (On spec)
  3. Quick.( On time)

Pick two of the three and only two. It doesn't which way you pick, you're fucked on the third. Doesn't matter about methodology, doesn't matter about requirements or project manglement. You are screwed on the third and the great news is, is that the level of the reaming you get scales with the size of the project.

After almost 20 years in the industry this has held true.

Dagg
Re: 'What's Real and What's for Sale'...

Technically 'agile' just means you produce working versions frequently and iterate on that.

No, technically agile means having no clue as to what is required and to evolve the requirements as you build. All well and good if you have a dicky little web site but if you are on a very / extremely large project with fixed time frame and fixed budget you are royally screwed trying to use agile as there is no way you can control scope.

Hell under agile no one has any idea what the scope is!

Archtech
Re: Government still spends an outrageous amount of money on IT

I hope you were joking. If not, try reading the classic book "The Mythical Man-Month".

oldtaku
'Agile' means nothing at this point. Unless it means terrible software.

At this point, courtesy of Exxxxtr3333me Programming and its spawn, 'agile' just means 'we don't want to do any design, we don't want to do any documentation, and we don't want to do any acceptance testing because all that stuff is annoying.' Everything is 'agile', because that's the best case for terrible lazy programmers, even if they're using a completely different methodology.

I firmly believe in the basics of 'iterate working versions as often as possible'. But why sell ourselves short by calling it agile when we actually design it, document it, and use testing beyond unit tests?

Yes, yes, you can tell me what 'agile' technically means, and I know that design and documentation and QA are not excluded, but in practice even the most waterfall of waterfall call themselves agile (like Kat says), and from hard experience people who really push 'agile agile agile' as their thing are the worst of the worst terrible coders who just slam crap together with all the finesse and thoughtfulness of a Bangalore outsourcer.

Adrian 4
It's like any exciting new methodology : same shit, different name. In this case, one that allows you to pretend the tiny attention-span of a panicking project manager is a good thing.

When someone shows me they've learned the lessons of Brooke's tarpit,. I'll be interested to see how they did it. Until then, it's all talk.

jamie m
25% Agile:

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.

Working software is the primary measure of progress.

kmac499
Re: 25% Agile:

From Jamie M

Working software is the primary measure of progress.

Brilliant few word summary which should be scrawled on the wall of every IT project managers office in Foot High Letters.

I've lived through SSADM, RAD, DSDM, Waterfall, Bohm Spirals, Extreme Programming and probably a few others.

They are ALL variations on a theme. The only thing they have in common is the successful ones left a bunch of 0's and 1's humming away in a lump of silicon doing something useful.

Doctor Syntax
Re: 25% Agile:

"Working software is the primary measure of progress."

What about training the existing users, having it properly documented for the users of the future, briefing support staff, having proper software documentation, or at least self documenting code, for those who will have to maintain it and ensuring it doesn't disrupt the data from the previous release? Or do we just throw code over the fence and wave goodbye to it?

Charlie Clark
Re: Limits of pragmatism

In France "naviguer à vue" is pejorative.

Software development in France* which also gave us ah yes, it may work in practice but does it work in theory .

The story is about the highly unusual cost overrun of a government project. Never happened dans l'héxagone ? Because it seems to happy pretty much everywhere else with relentless monotony because politicians are fucking awful project managers.

* FWIW I have a French qualification.

Anonymous Coward

Agile only works if all stakeholders agree on an outcome

For a project that is a huge change of operating your organisation it is unlikely that you will be able to deliver, at least the initial parts of your project, in an agile way. Once outcomes are known at a high level stakeholders have something to cling onto when they are asked what they need, if indeed they exist yet. (trying to ask for requirements for a stakeholder that doesn't exist yet is tough).

Different methods have their own issues, but in this case I would have expected failure to be reasonably predictable.

You wont have much to show for it, as they shouldn't at least, have started coding to a business model that itself needs defining. This is predictable, and overall means that no one agrees what the business should look like, let alone how a vendor delivered software solution should support it.

I have a limited amount of sympathy for the provider for this as it will be beyond their control (limited as they are an expensive government provider after all)

This is a disaster caused by the poor management in UKGOV and the vendor should have dropped and ran well before this.

Anonymous Coward

I'm a one man, self employed business - I do some very complex sites - but if they don't work I don't get paid. If they spew ugly bugs I get paniced emails from unhappy clients.

So I test after each update and comment code and add features to make my life easy when it comes to debugging. Woo, it even send me emails for some bugs.

I'm in agreement with the guy above - a dozen devs, a page layout designer or two, some databases. One manager to co-ordinate and no bloody jargon.

There's a MASSIVE efficiency to small teams but all the members need to be on top of their game.

Doctor Syntax
"I'm in agreement with the guy above - a dozen devs, a page layout designer or two, some databases. One manager to co-ordinate and no bloody jargon."

Don't forget a well-defined, soluble problem. That's in your case, where you're paid by results. If you're paid by billable hours it's a positive disadvantage.

Munchausen's proxy
Agile Expertise?

" I'm no expert in project management, but I'm pretty sure it isn't supposed to be making it up as you go along, and constantly changing the specs and architecture."

I'm no expert either, but I honestly thought that was quite literally the definition of agile. (maybe disguised with bafflegab, but semantically equivalent)

Zippy's Sausage Factory
Sounds like what I have said for a while...

The "strategy boutiques" saw "agile" becoming popular and they now use it as a buzzword.

These days, I put it in my "considered harmful" bucket, along with GOTO, teaching people to program using BASIC, and "upgrading" to Office 2016*.

* Excel, in particular.

a_yank_lurker
Buzzword Bingo

All too often a sound development ideas are perverted. The concepts are sound but the mistake is to view each as the perfect panacea to produce bug free, working code. Each has its purpose and scope of effectiveness. What should be understood and applied is not a precise cookbook method but principles - Agile focuses communication between groups and ensuring all on the same page. Others focus more on low level development (test driven development, e.g.) but one can lose sight the goal is use an appropriate tool set to make sure quality code is produced. Again, is the code being tested, are the tests correct, are junior developers being mentored, are developers working together appropriately for the nature of the projects are issues to be addressed not the precise formalism of the insultants.

Uncle Bob Martin has noted that one of the problems the formalisms try to address is the large number of junior developers who need proper mentoring, training, etc. in real world situations. He noted that in the old days many IT pros were mid level professionals who wandered over to IT and many of the formalisms so beloved by the insultants were concepts they did naturally. Cross functional team meetings - check, mentor - check, use appropriate tests - check, etc. These are professional procedures common to other fields and were ingrained mindset and habits.

Doctor Syntax
It's worth remembering that it's the disasters that make the news. I've worked on a number of public sector projects which were successful. After a few years of operation, however, the contract period was up and the whole service put out to re-tender.* At that point someone else gets the contract so the original work on which the successful delivery was based got scrapped.

* With some very odd results, it has to be said, but that's a different story.

goldcd

...My personal niggle is that a team has "velocity" rather than "speed" - and that seems to be a somewhat deliberate and disingenuous selection. The team should have speed, the project ultimately a measurable velocity calculated by working out how much of the speed was wasted in the wrong/right direction.

Anyway, off to get my beauty sleep, so I can feed the backlog tomorrow with anything within my reach.

Notas Badoff
Re: I like agile

Wanted to give you an up-vote as "velocity" vs. "speed" is exactly the sleight of hand that infuriates me. We do want eventual progress achieved, as in "distance towards goal in mind", right?

Unfortunately my reading the definitions and checking around leads me to think that you've got the words 'speed' and 'velocity' reversed above. Where's that nit-picker's icon....

bfwebster
I literally less than an hour ago gave my CS 428 ("Software Engineering") class here at Brigham Young University (Provo, Utah, USA) my final lecture for the semester, which included this slide:

Process is not a panacea or a crutch or a silver bullet. Methodologies only work as well as the people using it. Any methodology can be distorted to give the answer that upper management wants (instead of reality)

When adopting a new methodology:

Understand the strengths and weaknesses of a given methodology before starting a project with it. Also, make sure a majority of team members have successfully completed a real-world project using that methodology.

Pete 2
Save some fun for us!

> under the guise of " agile ?". I'm no expert in project management, but I'm pretty sure it isn't supposed to be making it up as you go along, and constantly changing the specs and architecture.

So why should the developers have all the fun? Why can't the designers and architects be "agile", too? Isn't constantly changing stuff all part of the "agile" way?

[May 15, 2017] Why Your Users Hate Agile

No methodology can substitute good engineers who actually talk to and work with each other. Good engineers can benefit from a better software development methodology, but even the best software development methodology is powerless to convert mediocre developers into stars.
Notable quotes:
"... disorganized and never-ending ..."
"... Agile is to proper software engineering what Red Bull Flugtag is to proper aeronautic engineering.... ..."
"... As TFA points out, that always works fine when your requirements are *all* known an are completely static. That rarely happens in most fields. ..."
"... The problem with Agile is that it gives too much freedom to the customer to change their mind late in the project and make the developers do it all over again. ..."
"... If you are delivering to customer requests you will always be a follower and never succeed. You need to anticipate what the customers need. ..."
"... It frequently is. It doesn't matter what methodology you use -- if you change major features/priorities at the last minute it will cost multiple times as much. Yet frequently customers expect it to be cheap because "we're agile". And by accepting that change will happen you don't push the customers to make important decisions early, ensuring that major changes will happen, instead of just being possible. ..."
"... The problem with all methodologies, or processes, or whatever today's buzzword is, is that too many people want to practice them in their purest form. Excessive zeal in using any one approach is the enemy of getting things done. ..."
"... On a sufficiently large project, some kind of upfront design is necessary. ..."
"... If you insist on spinning back every little change to a monstrously detailed Master Design Document, you'll move at a snail's pace. As much as I hate the buzzword "design patterns", some pattern is highly desirable. ..."
"... there is no substitute for good engineers who actually talk to and work with each other. ..."
"... If you don't trust those people to make intelligent decisions (including about when things do have to be passed up) then you've either got the wrong people or a micromanagement fetish ..."
"... The problem the article refers to about an upfront design being ironclad promises is tough. Some customers will work with you, and others will get their lawyers and "systems" people to waste your time complaining about every discrepancy, ..."
"... In defense everything has to meet spec, but it doesn't have to work. ..."
"... There is absolutely no willingness to make tradeoffs as the design progresses and you find out what's practical and necessary and what's not. ..."
"... I'll also admit that there is a tendency to get sloppy in software specs because it is easier to make changes. Hardware, with the need to order materials, have things fabbed, tape out a chip, whatever, imposes a certain discipline that's lacking when you know you can change the source code at anytime. Being both, I'm not saying this is because hardware engineers are virtuous and software engineers are sloppy, but because engineers are human (at least some of them). ..."
"... Impressive stuff, and not unique to the space shuttle. Fly-by-wire systems are the same way. You're talking DO-178B [wikipedia.org] Level A stuff. It works, and it's very very expensive. If it was only 10x the cost of normal software development I'd be amazed. I agree that way too much software is poorly planned and implemented crap, and part of the reason is that nobody wants realistic cost estimates or to make the difficult decisions about what it's supposed to do up-front. But what you're talking about is aerospace quality. You couldn't afford a car or even a dishwasher made to those standards. ..."
Jun 05, 2013 | Slashdot

"What developers see as iterative and flexible, users see as disorganized and never-ending.

This article discusses how some experienced developers have changed that perception. '... She's been frustrated by her Agile experiences - and so have her clients.

"There is no process. Things fly all directions, and despite SVN [version control] developers overwrite each other and then have to have meetings to discuss why things were changed. Too many people are involved, and, again, I repeat, there is no process.' The premise here is not that Agile sucks - quite to the contrary - but that developers have to understand how Agile processes can make users anxious, and learn to respond to those fears. Not all those answers are foolproof.

For example: 'Detailed designs and planning done prior to a project seems to provide a "safety net" to business sponsors, says Semeniuk. "By providing a Big Design Up Front you are pacifying this request by giving them a best guess based on what you know at that time - which is at best partial or incorrect in the first place." The danger, he cautions, is when Big Design becomes Big Commitment - as sometimes business sponsors see this plan as something that needs to be tracked against.

"The big concern with doing a Big Design up front is when it sets a rigid expectation that must be met, regardless of the changes and knowledge discovered along the way," says Semeniuk.' How do you respond to user anxiety from Agile processes?"

Shinobi

Agile summed up (Score:5, Funny)

Agile is to proper software engineering what Red Bull Flugtag is to proper aeronautic engineering....

Nerdfest

Re: doesn't work

As TFA points out, that always works fine when your requirements are *all* known an are completely static. That rarely happens in most fields.

Even in the ones where it does it's usually just management having the balls to say "No, you can give us the next bunch of additions and changes when this is delivered, we agreed on that". It frequently ends up delivering something less than useful.

MichaelSmith

Re: doesn't work (Score:5, Insightful)

The problem with Agile is that it gives too much freedom to the customer to change their mind late in the project and make the developers do it all over again.

ArsonSmith

Re: doesn't work (Score:4, Insightful)

...but they can be trusted to say what is most important to them at the time.

No they can't. If you are delivering to customer requests you will always be a follower and never succeed. You need to anticipate what the customers need. As with the I guess made up quote attributed to Henry Ford, "If I listened to my customers I'd have been trying to make faster horses." Whether he said it or not, the statement is true. Customers know what they have and just want it to be faster/better/etc you need to find out what they really need.

AuMatar

Re: doesn't work (Score:5, Insightful)

It frequently is. It doesn't matter what methodology you use -- if you change major features/priorities at the last minute it will cost multiple times as much. Yet frequently customers expect it to be cheap because "we're agile". And by accepting that change will happen you don't push the customers to make important decisions early, ensuring that major changes will happen, instead of just being possible.

ebno-10db

Re: doesn't work (Score:5, Interesting)

"Proper software engineering" doesn't work.

You're right, but you're going to the other extreme. The problem with all methodologies, or processes, or whatever today's buzzword is, is that too many people want to practice them in their purest form. Excessive zeal in using any one approach is the enemy of getting things done.

On a sufficiently large project, some kind of upfront design is necessary. Spending too much time on it or going into too much detail is a waste though. Once you start to implement things, you'll see what was overlooked or why some things won't work as planned. If you insist on spinning back every little change to a monstrously detailed Master Design Document, you'll move at a snail's pace. As much as I hate the buzzword "design patterns", some pattern is highly desirable. Don't get bent out of shape though when someone has a good reason for occasionally breaking that pattern or, as you say, you'll wind up with 500 SLOC's to add 2+2 in the approved manner.

Lastly, I agree that there is no substitute for good engineers who actually talk to and work with each other. Also don't require that every 2 bit decision they make amongst themselves has to be cleared, or even communicated, to the highest levels. If you don't trust those people to make intelligent decisions (including about when things do have to be passed up) then you've either got the wrong people or a micromanagement fetish. Without good people you'll never get anything decent done, but with good people you still need some kind of organization.

The problem the article refers to about an upfront design being ironclad promises is tough. Some customers will work with you, and others will get their lawyers and "systems" people to waste your time complaining about every discrepancy, without regard to how important it is. Admittedly bad vendors will try and screw their customers with "that doesn't matter" to excuse every screw-up and bit of laziness. For that reason I much prefer working on in-house projects, where "sure we could do exactly what we planned" gets balanced with the cost and other tradeoffs.

The worst example of those problems is defense projects. As someone I used to work with said: In defense everything has to meet spec, but it doesn't have to work. In the commercial world specs are flexible, but it has to work.

If you've ever worked in that atmosphere you'll understand why every defense project costs a trillion dollars. There is absolutely no willingness to make tradeoffs as the design progresses and you find out what's practical and necessary and what's not. I'm not talking about meeting difficult requirements if they serve a purpose (that's what you're paid for) but being unwilling to compromise on any spec that somebody at the beginning of the project pulled out of their posterior and obviously doesn't need to be so stringent. An elephant is a mouse built to government specifications.

Ok, you can get such things changed, but it requires 10 hours from program managers for every hour of engineering. Conversely, don't even think about offering a feature or capability that will be useful and easy to implement, but is not in the spec. They'll just start writing additional specs to define it and screw you by insisting you meet those.

As you might imagine, I'm very happy to be back in the commercial world.

Anonymous Coward

Re: doesn't work (Score:2, Interesting)

You've fallen into the trap of using their terminology. As soon as 'the problem' is defined in terms of 'upfront design', you've already lost half the ideological battle.

'The problem' (with methodology) is that people want to avoid the difficult work of thinking hard about the business/customer's problem and coming up with solutions that meet all their needs. But there isn't a substitute for thinking hard about the problem and almost certainly never will be.

The earlier you do that hard thinking about the customer's problems that you are trying to solve the cheaper, faster and better quality the result will be. Cheaper? Yes, because bugfixing that is done later in the project is a lot more expensive (as numerous software engineering studies have shown) Faster? Yes, because there's less rework. (Also, since there is usually a time = money equivalency, you can't have it done cheap unless it is also done fast. Higher quality? Yes, because you don't just randomly stumble across quality. Good design trumps bad design every single time.

... ... ...

ebno-10db

Re: doesn't work (Score:4, Interesting)

Until the thing is built or the software is shipped there are many options and care should be taken that artificial administrative constraints don't remove too many of them.

Exactly, and as someone who does both hardware and software I can tell you that that's better understood by Whoever Controls The Great Spec in hardware than in software. Hardware is understood to have physical constraints, so not every change is seen as the result of a screw-up. It's a mentality.

I'll also admit that there is a tendency to get sloppy in software specs because it is easier to make changes. Hardware, with the need to order materials, have things fabbed, tape out a chip, whatever, imposes a certain discipline that's lacking when you know you can change the source code at anytime. Being both, I'm not saying this is because hardware engineers are virtuous and software engineers are sloppy, but because engineers are human (at least some of them).

ebno-10db

Re: doesn't work (Score:2)

http://www.fastcompany.com/28121/they-write-right-stuff

This is my evidence that "proper software engineering" *can* work. The fact that most businesses (and their customers) are willing to save money by accepting less from their software is not the fault of software engineering. We could and did build buildings much faster than we do today, if you are willing to make more mistakes and pay more in human lives. If established industries and their customers began demanding software at that higher standard and were willing to pay for it like it was real engineering, then maybe it would happen more often.

Impressive stuff, and not unique to the space shuttle. Fly-by-wire systems are the same way. You're talking DO-178B [wikipedia.org] Level A stuff. It works, and it's very very expensive. If it was only 10x the cost of normal software development I'd be amazed. I agree that way too much software is poorly planned and implemented crap, and part of the reason is that nobody wants realistic cost estimates or to make the difficult decisions about what it's supposed to do up-front. But what you're talking about is aerospace quality. You couldn't afford a car or even a dishwasher made to those standards.

donscarletti

Re: doesn't work (Score:3)

260 people maintaining 420,000 lines of code, written to precise externally provided specifications that change once every few years.

This is fine for NASA, but if you want something that does roughly what you need before your competitors come up with something better, you'd better find some better programmers.

[May 25, 2013] World's Biggest 'Agile' Software Project Close To Failure

May 25, 2013 | Slashdot

00_NOP writes "'Universal Credit' - the plan to consolidate all Britain's welfare payments into one - is the world's biggest 'agile' software development project. It is now close to collapse, the British government admitted yesterday. The failure, if and when it comes, could cost billions and have dire social consequences. 'Some steps have been taken to try to rescue the project. The back end – the benefits calculation – has reportedly been shifted to a "waterfall" development process – which offers some assurances that the government at least takes its fiduciary duties seriously as it should mean no code will be deployed that has not been finished. The front end – the bit used by humans – is still meant to be "agile" – which makes some sense, but where is the testing? Agile is supposed to be about openness between developer and client and we – the taxpayers – are the clients: why can't we see what our money is paying for?'"

[Dec 22, 2012] Ask Slashdot How To Gently Keep Management From Wrecking a Project

Slashdot

Re:solve your problem small (Score:5, Funny)

rwa2 (4391) , Sunday December 23, @01:40AM (#42373645) Homepage Journal

Ooh, ooh, I just got through basic Scrum / Agile training, so I know the answer to this!

Get everyone committed to using Scrum methodology for your project before they really know what it means. Then assign yourself as Product Owner and your boss as Scrum Master for the duration of the project. Then send all the other chickens to Scrum training, so they can find out that they're chickens. They'll spend all the rest of their time rooting out all of the other chickens in your organization to keep them from clucking up your very important work.

After your project gets delivered on time without interference, and everyone can give themselves a pat on the back from running interference. Everyone gets a promotion! (which is all they really wanted in the first place)

[Jun 1, 2011] Making Agile Work for You in TFS 2010–MSDN Magazine June 2011

TechNet
In a recent issue of MSDN magazine, I wrote about our team's journey to Agile software development and I wanted to share that it was released. I've written a lot about experiences our team has gone through in trying to move from a waterfall style of development and to a more iterative approach using Agile techniques – powered by TFS 2010. Using Agile inside of Microsoft is, to speak softly, like trying to turn the Titanic 90 degrees in 60 seconds – its "tough" if not nearly impossible. The reason I say this is that it works great for "a" team – it really begins to become a challenge when you are partnering with other teams to build software as many teams have their own "process." The team I lead is fighting it and doing our best to move to an Agile model.

Inside Microsoft, you find a lot of "Agilebut" or "SCRUMbut" engineering teams. If you aren't familiar with this term it simply means that you are doing some form of Agile but not these parts of Agile. I find that many of the teams are saying that "we don't write functional specs" so we are Agile which is ironic because no where does it say you have to stop writing specs. In fact, Agile doesn't have a "rule" of no documentation but instead just says it values people interactions over documentation. This is the Agilebut methodology that is rampant across the world.

The one thing I will tell you, which is completely my opinion, is that aligning to a specific type of Agile development process and locking into it is your first mistake. The competition between several types of Agile, for example, SCRUM vs. Kanban, is absolutely nuts in my opinion. If you are adopting SCRUM, you must do everything SCRUM says or your not SCRUM?

Let's look at the definition of Agile and talk about it -

"Able to move quickly with suppleness, skill, and control"

The key point to look at there is the fact that it says to move quickly, but with control and skill. Agile teams have a huge amount of discipline when they are executing properly. The first mistake I find is that you are making is trying to find a specific-type of Agile development to align to. Instead, find what you like about any of the methodologies – whether is in XP (Extreme Programming), SCRUM, Kanban, – and make them a part of your Agile process. We are closely aligned to SCRUM but … we don't have SCRUM Masters, or Product Owners.

We have Program Managers, Developers, and Testers. We aren't changing our titles so that we are now using SCRUM. Guess who runs our daily stand-ups? Program Managers. Guess who drives our interactions with customers? Program Managers.

Adaptive nature is the best strategy I've found to allow Agile to fit into your team, your organization and that is exactly what we are practicing each day.

For our software development team, we document our process in a document called the "Master Agreements" that we continually update and discuss. It is the agreement that drives understanding of our process, our dynamics, etc. and is a summary of the methods we use for development.

It will consist of various different parts of different types of Agile, a buffet of sorts to steal a term I saw at TechEd 2011.

I hope that you take the opportunity to read the article I wrote, if for anything, to learn how we slowly but methodically moved our team to Agile. If anything, notice how we don't use parts of the TFS agile process templates –> the last thing you need to do is to shove a square peg into a circle.

The team needs to focus on doing what is natural and align their tools to that process which, in the article I wrote, shares some of how we did this.

The ironic thing is that anything I wrote in this article is potentially going to change by this time next year because our team cares more about learning, adapting, and improving individually, as a team, and as an organization.

Enjoy!

Thanks,

-Chris