Agile and Other Development Methods

Introduction

Agile methods are evolved to adopt changes quickly during the development. This approach is just opposite to plan driven traditional approaches. The main focus of agile methods is early releases of working software that responds to changes promptly using collaborative techniques, code refactoring, test driven development and customer involvement.

As per Agile Manifesto (http://www.agilemanifesto.org):

Individuals and interactions over processes and tools

Working software over comprehensive documentation

Customer collaboration over contract negotiation

Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

Agile methods follow these principles (http://agilemanifesto.org/principles.html):

• Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
• Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.
• Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
• Business people and developers must work together daily throughout the project.
• Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
• The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
• Working software is the primary measure of progress.
• Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
• Continuous attention to technical excellence and good design enhances agility.
• Simplicity--the art of maximizing the amount of work not done--is essential.
• The best architectures, requirements, and designs emerge from self-organizing teams.
• At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

These are the basic principles that every agile method tries to follow in order to overcome the common problems occurring during and after any software\project development. Every agile method has defined its own set of practices based upon above mentioned principles. The intention behind evolution of these methods is to serve customer better and to make development process simpler. We will discuss on agile methods and compare it with other traditional development methods.

Background

In general we follow two types of software development method – Plan Driven Traditional method (waterfall, spiral, iterative etc) and undefined method (we treat them as without any process).

Most common problems that we face during and after software/project development are:

Frequent changes from client side

In traditional development model, first we collect and freeze the requirement based upon discussions with client and then we do the development along with testing. At last we send the release to the client. But in most of the cases we find that client’s expectation varies at the delivery time of the software. So, we put our efforts to map the expectations with the approved requirement and try to convince them on all out of scope functionalities. Some time we convince them to include these in next phase or some time we do it right away to make them happy.

In few cases we receive regular frequent small changes from Client and in this scenario it’s almost impossible to apply defined classical processes. So, we start implementing new changes after delivery on ad hoc basis. Gradually, this creates more chaos situation and project leads to nearly unmanageable state.

Dependency on resource

Since different team is responsible for different phase of development, there are more dependencies on respective resource.

Documentation takes lot of time

Often we find documentation consumes lot of time. It’s very hard to create and update for short duration project or project with short deadlines.

Impact Analysis very complex

We always emphasize to create document that must cover all the important info and fact. But after putting all these relevant fact and figures it becomes very large document. Hence although we create all required documents but when we require to do impact analysis it becomes a cumbersome job for all of us.

All above problems lead to think upon all other possible alternatives.

Comparison with other existing methods

	Waterfall	Spiral	Iterative	Agile
Defined Processes	Required	Required	Required	Planning & Closure only
Final product	Determined during planning	Determined during planning	Set during project	Set during project
Project Cost	Determined during planning	Partially variable	Set during project	Set during project
Project Completion Date	Determined during planning	Partially variable	Set during project	Set during project
Responsiveness to environment	Planning only	Planning primarily	At end of each iteration	Throughout
Team flexibility, creativity	Limited – cookbook approach	Limited – cookbook approach	Limited – cookbook approach	Unlimited during iterations
Knowledge transfer	Training prior to project	Training prior to project	Training prior to project	Teamwork during project
Probability of success	Low	Medium Low	Medium	High

(Source: SCRUM Development process by Ken Schwaber)

Limitations of Agile Method

Limited Support for distributed environments

Face to face communication is an essential characteristic of agile processes. The emphasis on co-location in practices advocated by agile processes does not fit well with the drive by some industries to realize globally distributed software development environments. Development environments in which team members and customers are physically distributed may not be able to accommodate the face-to-face communication advocated by agile processes. In such cases, one can at least approximate face-to-face communication using technologies such as video-conferencing, but these technologies are expensive and not as effective as one would hope.

Suggestion to tackle it: Martin Fowler, a famous author and international speaker on software architecture, specializing in object-oriented analysis and design, UML, patterns, and agile software development methodologies, including extreme programming, has work upon this problem and he has shared his experience in this article (http://martinfowler.com/articles/agileOffshore.html). All we need to customize and use some creativity on existing agile practices to address it.

Limited support for building reusable artifacts

A case against agile process is that it target specific problems in the software and is deficient in producing generalized solutions.

Suggestion to tackle it: Building reusable artifacts is a very controversial topic. It’s very hard to develop something and to say it is reusable. If any one has developed anything and declared it as reusable is not enough. I have come across many times using so called reusable artifact in my projects and needed drastic modification in it and later felt to create new one as per my own requirement.

Limited support for developing safety-critical software

It is said that agile quality control mechanisms like pair programming, informal reviews and refactoring etc are not sufficient to assure users about the safety of the final product.

Suggestion to tackle it: I have used TDD, pair programming, informal reviews and refactoring and achieved great success in final product. I am repeating once again that if require we are free to customize or modify the existing process.

Limited support for development involving large teams

This is well known limitation to agile processes. It is believed that agile process is more suitable for small collocated teams.

Suggestion to tackle it: Its all assumptions and I found many people and organizations have used it successfully in large projects and large distributed team.

Agile Benefits

We wouldn’t try to follow agile practices if we didn’t think it was better in some way. Following are the essential benefits of agile:

1. Early Return on Investment: An agile process delivers valuable results every single iteration. This provides the basis of early return on investment. The early delivery of value has an additional benefit. By delivering value early, the time value of that investment is increased. Revenue or cost savings can be realized sooner and therefore more funds are feed up for other investment.

2. Rapid Learning: This is one of the clearest advantages of an agile approach: rapid deliver helps you learn what you truly want and need because you can see actual results instead of "intermediate" results. If you are building software, you can see a working system and get real concrete feedback about it. Users, consumers, and other stakeholders can participate more effectively in determining what you are building.

3. Satisfied Stakeholders: Agile methods provide multiple roads to satisfaction for customers, users, business people, bureaucrats (okay, maybe not _all_ bureaucrats), team members, managers, shareholders, and interested passer-by. There are three primary mechanisms by which this occurs: engagement, trust-building and feedback-control.

4. Increased Control: Agile methods enable better control of work.

5. Responsiveness to change: The fact that agile methods increase return on investment, accelerate learning, increase stakeholder satisfaction, and enable better control of work are all an interesting result of this final benefit: responding to change.

What we follow at present and do we need it

As I told and discussed initially that either we follow plan driven method or we follow nothing.

Normally in case of large project we able to follow the processes. Though we always try to follow processes in all case but all of us know the limitations of this effort. Some times we get small project or frequent small changes, and thus we inadvertently leave the processes in a side to achieve the target timely and try to make client happy.

So now one obvious question arises in everyone’s mind- will the agile method solve it? And my answer is- I can’t say. I am not sure. Then, what is the use of this discussion?

First of all, the intention of this discussion is not to provide you a solution that would always work for all projects. But it is just to make you aware of other existing alternatives. We don’t need to use the same old methodology in case of all projects. Instead we must choose methodology from the existing lists as per the nature of the project.

Also when we will study and do the comparison among different development methods based upon past and current experience then probably we could be able to decide and follow some mixed practices. Below are some best practices of agile process:

• Unit Testing and Test-Driven Development ensures that bugs and errors are found quickly and early so that it would be cheaper to fix.
• On-site customer and functional testing ensure the analysis and specification of the system is up-to-date and precise with business requirements.
• Pair programming allows two developers working together on one computer, which increases the chances of finding bugs and leads to a simpler design
• Refactoring and “once and only once” increases design consistency and adds more simplicity and flexibility to the structure. This ensures that the system is well-designed and easy to change.
• Regular releases gives the customer feedback and forces the team to make the “release to production” and maintenance phases as cheap as possible.
• Daily small status meeting ensure the progress is in right track
• Use of UML diagrams help to address long documentation and cumbersome impact analysis of design change

All of us are free to decide and customize the development methods depending upon the nature of project. One can mix SCRUM and XP practices or can mix some other agile methods. It’s not so simple to create or adopt agile method\customized method indeed it requires good creativity, deep analysis and open-mindedness.

Challenges in Adopting Agile Methods

Though, agile methods are much hyped these days but it’s not so easy to adopt in the organization.

People with Old mind set or Closed Mindedness (who doesn't adopt or learn new thing easily) are big challenge to implement it anywhere. In some cases people specialized in one skill set create problem as agile method needs participation of everyone, so having specialization in more than one field could result better. Sometimes many organizations use some limited set of options and doesn't want to change it. It could be a Fear Change.
Few times I feel a person with Outdated Skills also creates problem or who love to document a lot for every piece of project. Since all these items can be addressed with UML design artifacts only and agile methods focus on code development over creating UML design artifacts, so big documentation can be reduced to small one and people/organization who loves long documentation doesn't like it.

And the most important one is that most of the time people try to adopt all agile practices in one go. Last point, we should keep in mind that Office Politics is also a big challenge to adopt and use it.

In summary below are the challenges in adopting agile methods:

• Fear of change
• Specialized skills
• Outdated skills
• Documentation heavy mind set
• Do-it-all-at-once attitude
• Serial Thinking
• Closed Mindedness
• Office Politics
• Black and White mindset

Conclusion

If you are not getting troubles or satisfied with the development method which is followed by your organization, then there is no need to opt for agile methods. The final goal is to meet client’s expectation and satisfaction. Through Agile methods our endeavor is to respond the changes quickly even late in development and feel complete satisfaction after the delivery of any software/project.

Role of Processes In Software Development

Introduction

The ultimate goal of any Software delivery is Client Satisfaction. If Client is satisfied with the delivered software or project then it doesn’t matter which technology was used or whether any process was followed or what architecture, design and agile methodologies was used. But if the Client is unhappy then all applied processes, all latest technologies used to develop it and all applied design patterns/architecture methodologies seem useless. Then, why do we all emphasize to use them or follow them all the time? What is the importance of talking about these stuffs and how much these are relevance for real-time Software development?

Background

First of all we should understand that Processes are not merely responsible for timely delivery of Software with high quality, but also to set us free from lots of high expectations of Client. All of us know there is no limit of expectations. If the delivery doesn’t contain any problem and Client is satisfied with it, we are lucky. But if it is opposite then we are in great trouble. Because once Software has been delivered, every valid reason for this situation is not more than lame excuses. And often we face this kind of situations.

We will not discuss about the common design or architecture problems here but we will see later what leads us to make common mistakes while creating design and architecture for any application.

Let’s discuss on the common reasons that make any Client unhappy.

Delivered Software doesn’t behave as per the actual requirements

Its very sad part and even hard to realize for IT world, 50% software fails just because of inadequate requirement management. This kind of problem falls under Inception and Elaboration phase. Inception phase provides only very vague idea of the requirements like business rationale of the project and scope of the project etc. During Elaboration phase it is insisted to capture and freeze the requirements with mutual agreement and written approval by the Client and Stakeholders. So, after the delivery of Software Client could not disagree on its behavior. Only he can suggest some enhancements or modifications. All these suggestions can be easily converted to new requirements for the next phase.

Delivered Software doesn’t behave as per the expectations

Normally this kind of problem occurs due to ignorance of risks involved in the project or inadequate analysis for some of the common problems involved in almost applications like actual load on the application, number of concurrent users, expected performance, security, level of dependencies on external applications etc. Often development team starts development without taking care of these points. Valid reasons could be applied here to justify the reasons for all these ignorance. But we should understand that when we get un-success everyone cares on every minute mistake and on the contrary when we get success no one cares even about big flaws (till it works smoothly). If requirements have been captured and finalized with proper risk analysis, this problem will rarely occur. Now this problem comes under risk analysis and mitigation plan. During elaboration phase all major risks should be identified and communicated to Client/Stakeholders and proper mitigation should be prepared by the concerned person. All these help to identify the scope\boundary of project and dependencies of project on external systems and environment.

It took more time than proposed delivery date

Many times we are compelled to start development work without collecting and finalizing requirement, without proper planning of project, without any risk & mitigation plan for the project. That leads the development to very haphazard position and finally delays in achieving the deadlines. The reason behind this kind of situations is undue pressure from Client side. Only one thing can help here and that is just to try to convince the Client for minimum use of processes and win the trust and faith of client on use of processes.

Lots of problem coming at the time of actual use

If all high risks have not been identified and QA has not been done let’s be ready to face this situation. Sometimes due to some external factors (like hosting of other applications on same server that causes the performance issue for our application or our application uses some shared external database and due to some changes in database it breaks our application etc) lots of problems occur in our application. I do agree that these situations can occur even after risk & mitigation plan and QA. But chances are very less. And at this point of time at least we can convince the Client with adequate justification. One important thing is always follow best practices and try to apply design and development methodologies only as per the nature and requirement of the application.

Any Enhancement and Modification shakes complete application

As I told initially, my endeavor here is not to identify the common mistakes of design and architecture. Yes, it’s true that these are also one aspect of reason for this situation. In absence of Project Management process or Development Management process this situation occurs. In a big and complex project no one can do impact analysis of any new request for change or enhancement without following these processes.

Conclusion

If we would follow the processes we shall be able to minimize the risks or at least we would have some valid reason for the worse situations. And the most important thing is when stakeholders/Client will also play his defined role in all processes, he will more easily be got convinced to understand and agree upon the reasons behind such situations. So processes always keep us in safe situation and help both stakeholders and Client mutually understand and visualize about actual problem (what), accurate factor (why) and true solution (how) during the worst situation. The biggest challenge to follow any process is to make the stakeholders\Client also aware of the processes and its importance from his point of view.