When I reflect on the various software projects I have either worked on, or managed in the last two decades, it soon becomes apparent that all my successful projects have had some things in common. They are:
- Individuals and interactions over processes and tools
- Working software over comprehensive documentation
- Customer collaboration over contract negotiation
- Responding to change over following a plan
The four items mentioned above happen to be the core elements of "The Agile Manifesto”. In this blog, I want to focus on the second item, which is "working software over comprehensive documentation." Scott Ambler of IBM says that the critical points to consider in deciding documentation requirements for a software project are:
- “Document stable things, not speculative things.
- Take an evolutionary approach to documentation development, seeking and then acting on feedback on a regular basis.
- Prefer executable work products such as customer tests and developer tests over static work products such as plain old documentation (POD).
- You should understand the total cost of ownership (TCO) for a document, and someone must explicitly choose to make that investment.
- Well-written documentation supports organizational memory effectively, but is a poor way to communicate during a project.
- Documentation should be concise: overviews/roadmaps are generally preferred over detailed documentation.
- With high quality source code and a test suite to back it up, you need a lot less system documentation.
- Comprehensive documentation does not ensure project success; in fact, it increases your chance of failure.
- Models are not necessarily documents, and documents are not necessarily models.
- Documentation is as much a part of the system as the source code.
- Your team’s primary goal is to develop software; its secondary goal is to enable your next effort.
- The benefit of having documentation must be greater than the cost of creating and maintaining it.
- Developers rarely trust the documentation, particularly detailed documentation because it usually is out of sync with the code.
- Ask whether you NEED the documentation, not whether you want it.
- Update documentation only when it hurts.”
Reasons why you should document:
To turn conceptual ideas into working code - I have found it very useful to put my thoughts down on paper in the form of a high-level logical flow of the functions or parts of a system that I plan to implement. I usually do this immediately after a brainstorming session with my team to ensure that I have not only captured all the essential features that must be implemented, but also to verify that there are no holes in the handling of errors and exceptions.
To establish unambiguous interface contracts -the famous line from "Meditation XVII" by the English poet John Donne - "no man is an island entire of itself; every man is a piece of the continent, a part of the main." - holds true for software systems as well. No software application can exist in isolation without human and/or data interactions. These interactions typically referred to as "application programming interfaces" (APIs) must be clearly defined and documented so that the different parties - humans and/or machines - involved in the data exchange understand the types of information they are likely to receive from, and expected to send to another human or software application.
To be in compliance with regulatory requirements -software must satisfy the documentation requirements imposed by either the government or the industry. Unfortunately many of these requirements are formulated by bureaucrats as a knee-jerk response to absolve them of any liability should the software malfunction. As software engineers it is incumbent upon us to thoroughly read and understand the regulatory requirements so that any documentation that is produced is just right for compliancy and not an overkill.
To identify operational steps after the software has stabilized - this document usually provides instructions for installing and running the software. It specifies the infrastructure required to run the software, schedules for backing up data, steps for scaling up or down the infrastructure, and procedures on how to handle system failures.
To help users effectively utilize all features of the software - also called a user's manual; this is a reference guide to the people who will be using the software. It should be written in a way that allows them to quickly find out how to accomplish a certain task via the software.
To provide overviews to stakeholders and the team - these documents which articulate the objectives of a software project come in two flavors - one for the business; the other for members of the implementation team. The former highlights the business objectives, the latter focuses on topics like the construction environment, source code control, database entities and descriptions of established (not speculative) models.
Reasons why you should not document:
The business stakeholders require it but it doesn't add any value - these documents usually consist of Gantt charts, architecture models, detailed sequence and use-case diagrams which the stakeholders like to "sign off" on, because it gives them the illusion of being in control, and bolsters their mistaken belief that the project will succeed, because how can it not, when the team has been able to generate such a plethora of detailed information even before a single line of code has been written? When you are faced with a situation like this, my advice is to clearly communicate the cost of creating such documents; highlight the lack of any useful function that these documents provide since we cannot document the unknown, and for the software project to succeed we must ever be prepared to handle changes as the system evolves during its lifecycle; and last but not least let the stakeholder know that he/she might be responsible for the failure of the project by insisting on creating documents that are speculative in nature.
Ambler lists the following as "questionable" reasons for creating documentation,
- "The requester wants to be seen to be in control.
- The requester mistakenly thinks that documentation has something to do with project success. Nothing could be further from the truth.
- The requester wants to justify his/her existence.
- The requester doesn’t know any better.
- Someone wants reassurance that everything is okay."
Software construction is innovative and creative in nature; any attempt to commoditize and turn into a manufacturing-style process usually does not work. My advice - be innovative, be rational and logical, keep it simple; there is no document that describes the software better than well-written code; there is no better indicator of progress than seeing "working software" evolve; document only when it is necessary and only when it hurts.
