The topic of industrialization has been on my mind a lot these days. By 'industrialization' I loosely mean the principles that have driven large scale manufacturing during the industrial revolution. These principles include the division of labor, specialization of skill, the use of production lines, increased usage of numeric metrics of quality, etc.
While I concede that the comforts of our modern life are largely the product of the industrial process, I find myself rebelling against the idea these days. The reason for my rebellion is that I am being exposed to 'industrial processes' for the first time in my life as a programmer, and I find them very stifling.
How do industrial processes enter the life of a programmer? Through the theory of Software Engineering, which prescribes sets of rules and metrics for programmers in large organizations to follow. This theory tries to apply the principles of division of labor, specialization, quantitative metrics for quality, etc. to software development.
For instance, in the Software Engineering world the Developers, Quality Engineers, Release Engineers, Service Engineers, Program Managers, Product Managers, etc. serve distinct roles in the development of software. There are numerical metrics like code coverage numbers, bug counts, etc., to keep track of the quality of the output. There are elaborate plans which attempt to ensure that each 'resource' is optimally utilized and the organization is maximally efficient. Everything is standardized, from coding style, to the use of text editors, to the color of your underwear. ( OK, just kidding about that last one ;-) )
I am all for being organized, coordinated, and having metrics for success. Any individual who wishes to accomplish anything in the world needs these qualities. However, these qualities are very personal and the problem comes with their institutionalization. Once this happens, the enshrined practices tend to become the standard of judgement. There is less incentive for major innovation and creativity because they cannot be appreciated by backward looking metrics.
This institutionalization of common sense and good judgement may make sense for certain organizations like those building software which handles financial transactions, or those writing software for medical equipment. These companies tend to tackle the innovation problem by separating innovation into labs (which don't build products) and engineering departments (which actually build the stuff). The two are connected by host of people whose titles I am not aware of, but probably include marketing, sales, product management, etc.
But I don't think that this kind of industrialization makes sense for companies who are trying to push the envelope of what computers and technology can do. These kinds of companies need Programmers (with a capital P) more than anything else. In my books a Programmer is a polymath: he likes dreaming up new product ideas which address a real need, likes defining the requirements of these products, likes desiging solutions meeting those requirements, likes actually building things, likes to see how his products work in the real world, likes taking that feedback to improve what was built and fill the gaps in his knowledge, and likes to experiment with new methods and practices that improve his efficiency.
For me, Programmers work on a small --non-industrial-- scale. They understand that creativity and inspiration are bottom up processes which take their own time and are not things to be forced. They are passionate craftsmen who love the end-to-end practice of delivering solutions that work and are relevant. They are thrilled by the process of creation. They don't care for metrics and processes which are not relevant to the job at hand. They are infact constantly striving to improve their technique for delivering products that work. (See my post on practice and play for a discussion of technique). They are not bound by convention.
Programmers of this description cannot work in industrial houses. They will be miserable if they do because they will be complete misfits. It is the classic battle of the machine versus the soul in which there are no winners. The best a Programmer can do when he finds himself in this battle is to gracefully surrender and move to a place which values his individuality and passion. Industries don't.