This is brilliant: an article by Martin Fowler in IEEE Software magazine from a few years back (via Julian On Software) really nails the issue of agility and complexity by referencing, oddly enough, a speech given by economist Enrico Zaninotto at the XP 2002 conference. Fowler says:
One aspect I found particularly interesting was his [Zaninotto’s] comment that irreversibility was one of the prime drivers of complexity. He saw agile methods, in manufacturing and software development, as a shift that seeks to contain complexity by reducing irreversibility — as opposed to tackling other complexity drivers. I think that one of an architect’s most important tasks is to remove architecture by finding ways to eliminate irreversibility in software designs.
Most of my customers are large financial and insurance organizations that still use very waterfall methods for development. The requirements and functional design take months to develop, and have concrete poured firmly over them as soon as they are complete. In other words, the irreversibility starts at the requirements stage, long before development even starts. Of course, since a technical design follows the requirements stage and in turn is solidified before development begins, the irreversibilty is built into this stage as well: any changes have to go back through (potentially) several layers of approval and redesign, which impacts project schedules and contracts.
Fowler referenced an example where a database administrator made it easy to change the database schema and migrate the data for a project; as Fowler put it, “the database schema is no longer architectural” since it could be changed on the fly to accommodate the requirements of the project, rather than being a pre-supposed part of the design.
When we used to do this, it was called “coding by the seat of our pants”; now it’s Agile!
This is a very interesting thread…and it’s like a ball of yarn because if we have a go at the rigid SDLC that are entrenched then we also have to take on the architecture regimes also. You point out an example of the “architecture” getting in the way of data migration. I would go so far as to say that enterprise architecture groups usually act as an impediment to effective change and process improvement. The other way to look at it is that you know you’re on the right track with new processes and technology, e.g. SOA, if you are disrupting the regimes that have made software development so bad for so long. Just another wave of disintermediation that seems to have followed the HTTP protocol wherever it went as far as I am concerned…
Maybe “architecture” has gone too far into design — since when does a physical data model need to be part of enterprise architecture? By pushing decisons that aren’t really part of the architecture down into software design where they belong, we build a bit more reversibility, and therefore agility, into the process.