Reports

[jsburbidge]

For half a year or so between working on more major projects, several years ago, I was stuck^H^H^H^H^H charged with implementing a number of reports on the back end. These had broad similarities in that they were written in Java, but they had different internal clients and ran in slightly different environments.

As separate projects, they had to have formal Requirements documents (System Requirements Specifications) - it was maybe five years before my employer decided to formally, if ineffectively, embrace Agile - and if they took more than 80 hours of work they also had to have a formal Design document and go through a much more formal approval process. The aim, accordingly, was to keep projects under 80 hours, if necessary by lying...

I'll take a representative one. It was a report in Excel format mandated by Compliance, which meant it was going to happen no matter what. The original requirements document had some vaguenesses and lacunae, and there were privileges for database access I needed to get, so it took about a week to get a first pass at the report back to the Business Analyst This exposed some more vaguenesses which needed to be specified (i.e. differences from the implementation) and it took a few more days of tweaking code and running tests - one nice thing about reports is that the developer can easily do his own testing - to get it back to him...

...only to be told that the client had new requirements. Additional columns, and some expansions to the records to be displayed.

This cycle was repeated three or four times more.

Deployment was a headache; the testing and production systems used different names for stored procedures, used different versions of some packages, and ran at a different OS patchlevel.

It took four months, elapsed, to reach production deployment of the application. At some point we undoubtedly went over the 80 hour mark, so reporting to management of time allocated was blurred into inaccuracy. (It was precisely to discourage the use of developer's cycles in casual changes with no chance to set priorities between projects which had led to the 80-hour cutoff for minor projects in the first place. As far as I could tell it was completely ineffective.)

I'm sure this took longer still; I had been looped in as an available resource only after the SRS had been written and approved in its first version.

Part of this was the fault of the Busuness Analyst. My own experience in an earlier context had indicated that in a watetfall-style world you have to be quite aggressive in finding out what users really want. And he had to be pushed into updating the requirements document to reflect the revised requirements, which is necessary to provide a basis for future maintenance. A bigger part of the fault was the business, for not thinking through what they wanted in the first place. A final part was that documentation of environments, credentials, and permissions was very poor and had to be extracted by specific e-mails passed up the chain.

This is a reason why larger businesses have jumped into a pseudo-Agile bandwagon: the metrics look better for piecewise delivery which is happening anyway. On this model, "delivery" took four plus months. In an Agile model, the first cut of the program would have been a sprint in itself, and each iteration would have been a distinct thing in itself. The Compliance department wouldn't have received their finished report any faster, but on the reporting that went up through development management it would have shown as a sequence of short successful sprints, mixed in with a couple that were blocked by external forces (from the point of view of immediate management, though internal to the organization).

Under Agile, e-mail records, from which I was able to reconstruct this, would have been reduced, which is a good in itself: daily scrums would have eliminated the need for e-mail In many cases (though I'm not sure it would have sped up e.g. getting usernames and passwords assigned). However, only e-mails between myself and the business analyst would have been affected. I doubt that the compliance department, in a different building, would have sent a knowledgeable representative over to be involved in a daily scrum; and the same for the key person in the database group, on a different floor. Actually implementing a true daily scrum with all stakeholders would have been difficult.

The dragged-out requirements were a waste of time and effort. If the client had been clear up front about what they wanted, with about the same amount of work they could have had a final report in a couple of weeks, and with no need to munge the hours to keep it below the radar, with a bit of trailing work getting it into production. An incompletely carried-through Agile approach would just have formally institutionalized a wasteful and pointless misuse of resources.

There are places where iterative development makes sense - for example, one with a large base of target users and a limited ability to get general feedback out of them except by pushing out better approximations on a regular basis. Or where a complex interactive application is the deliverable. But an internal client should be able to put together an exact model of a report right out of the gate; you can do it by messing around in Excel with sample data.

In a true Agile environment, of course, the client would be directly involved in daily meetings, face-to-face, and the tightening of the feedback loop would probably act to bring the time for development down sharply. But I have a hard time imagining that degree of involvement from a distant department in a large organization for this sort of project.