Wednesday, October 31, 2007

On the Nature of Software Development...

It began some years and several companies ago. I was happily working as a grunt programmer at a large company, but most of my formal education was in the natural sciences, rather than software or even engineering. My on-the-job accomplishments therefore came largely from brute force. I seemed to be surrounded by contemporaries who knew a great deal more than me, and who achieved much with far less effort than I expended.

However, I was lucky to have a particularly talented boss whose degree was in Physics, like my own. When describing how to create robust software, he spoke to me in our shared language. One should minimize the entropy of the system. We meant the word in its Physics sense, not in its Shannon or information theory sense. Literally, to shrink the number of accessible states into which the software can fall.

The signal to noise ratio in that exchange was extremely high. I can only speculate how long it would have taken me to grok what he was saying if I hadn’t studied Physics. Over time, I came to believe that my training in science instead of engineering was only an imaginary handicap, which abated quickly. After a few more years in industry, I felt that I was better prepared to develop software than some of my colleagues that had that initial edge.

I currently work as a software architect at a large corporation, and I am sometimes asked why I am so young. (I choose to take that as a compliment, rather than as an indictment of my worthiness.) I think that the successes I now enjoy directly stem from the way of thinking that comes from studying natural science. Along the way, I’ve worked for a few startups, some successful and some laughable failures. The intent of this blog is to capture and share some of the discoveries and lessons learned along the way.

Software development itself is a human activity, and if a comprehensive theory of it were to be developed, it would probably be done by economists, or sociologists, or psychologists. However, programmers’ output is not “virtual” at all. To me, software is visceral and vibrant. Great programs are beautiful sculptures, or musical compositions if your brain is put together that way. I think there is something to be gained by viewing software development as a hard science, more like Physics than philosophy.

In this blog, I’d like to explore how natural science, as a way of thinking, helps software architecture. I don’t propose that one can find the Hamiltonian of a development team. But I do think we can borrow ideas from Physics to help understand software and the way it gets made. For example, Newtonian mechanics works great in the low velocity limit, but breaks down at speeds approaching c. Might some analogous statement be made about what environments are suitable for agile programming?

If you are like me, gentle reader, we like to build things and make them work. Software architecture requires soft skills to educate and persuade our fellow human beings, who like ourselves are afflicted with ego and other weaknesses. No less so, it requires hard skills to engineer systems that must obey the laws of Physics, which are as relentless as they are splendid. And if I may be so bold to express this way: we are blessed to work in a field whose nature is both human and divine.