But, that's because you already knew a thing or two about compilers, or IPC, or generic programming. If that's the kind of thing that you do in your day job, then I'm happy for you, and a little jealous. A lot of software has to do with modeling poorly understood domains, where there is little agreed-upon common vocabulary or precedent. Some days I get confused about exactly what my job is, because I've strayed into something quite new.
If that describes you too, then a normative-only spec doesn't reveal the gist of the problem to be solved. Many of the so-called specs that we write concern "wicked" problems, that is, problems that are so intractable that they can only be stated by solving them. I've often seen requirements documents go on for pages and pages with the guise of rigor, but which fail to convey enough context for why their enumerated and cross-referenced shalls are good ideas.
During the actual process of composing such a document, a good writer will actually discover and answer these "why" questions. Options are weighed and decisions are made. But if the reasoning isn't recorded, then the spec degenerates into solipsist snippets of conversations between the author and himself. That's hardly communication.
At work, I try to champion putting more non-normative context into our documents. Unfortunately, I'm really good at losing arguments. So, I've started to wonder if any lessons from Mathematics or Physics can shore up my reasoning. I can think of three reasons why normative-only specifications are like Mathematical proofs.
- They hide the many wrong turns and false starts that their authors suffered while working.
- They are themselves the end product of the effort, not necessarily a means to an end. (Implementing the spec, or applying the theorem, is often left to others.)
- Specs and proofs are both terse, but they convey enough information for their consumers, as long as their consumers have the expertise and context to read them.
I am going to give what I will call an elementary demonstration. [But] elementary does not mean easy to understand. Elementary means that nothing, very little, is required to know ahead of time in order to understand it, except to have an infinite amount of intelligence. It is not necessary to have knowledge, but to have intelligence in order to understand an elementary demonstration.In this sense, our normative software specs need to be elementary. Unfortunately, I have never met a software spec writer as gifted as Feynman. Neither have you.
A while back, a pretentious document crossed my desk. It dotted every "i" and crossed every "t." Yet, after several readings, I still had no idea what the point was. There was a whole bunch of language that felt like it was adding fault tolerance features to an existing distributed product, but at the same time there was no redundant hardware to make changes worthwhile. So, naturally, I started to inject myself into hallway conversations to get the gist of the project.
It turns out that the existing product relied on some third-party software that came with some pretty onerous licensing fees. The whole point of the project was to rework how the system was deployed to save money. If you could revisit a few architectural decisions, then the expensive software could be localized in fewer places, thus saving a few bucks. Knowing that one little gem would have made the spec comprehensible, but it was nowhere to be found in the text because it was "non-normative."
Of course, you already know the punchline of the story. After spending a bunch of cash on the new project, somebody finally had the idea to call up the third-party software vendor and try to negotiate a more reasonable price. They agreed, and the re-architecture effort turned out to be a very expensive way to save money. By every criterion, whether dollars spent or risk incurred or opportunities lost, the phone call was the better option. But since not enough people were trusted with the real objective of the project, the optimal solution was discovered too late.
In hindsight, calling up the vendor seems obvious, but it wasn't. For those developers, who seldom interacted non-technically, it actually tipified out-of-the-box thinking. Creativity can't be scheduled into the spec-writer's block of time, because nobody can think of everything. The deliberate decision to exclude non-normative information from the original spec enabled this mistake
The best solution to a wicked problem is not commanded; it often emerges as it is solved. Had enough people understood what they were being asked to do, somebody would have made that phone call sooner. Consider Eric Raymond's assertion that "given enough eyeballs, all bugs are shallow." It's interesting to speculate whether there exists some parallel to Linus's Law. Perhaps enough eyeballs tame all wicked problems.
I suppose rationale doesn't matter when the spec itself is the work product. A compiler writer doesn't really have to know why Ada has two different functions for mod and remainder. One would just have to know the rules for each, and implement them correctly. But many of our software documents are not work products themselves; they are just the means why which communities build work products.
Much of what we do is wicked.
The analogy to nature came to mind as I thought about this software development question. I can hone my mathematical skills by reading and doing mathematics. Terse proofs are suitable in Mathematics, because anything you need to know is already out there for the taking. (Can you tell whether I'm a Platonist?) But Physics doesn't work that way. Or, at least, it doesn't work that way in my brain. One can't do new physics without doing experiments, and the most exciting experiments are done in unfamiliar domains.
So, mathematical proofs are like normative specifications, but physical principles are like domain expertise. That surely can't be guessed a priori. Now, suspend your objections for a minute. Yes, Physics has proofs. Yes, Mathematics is at least a quasi-empirical science. Yes, the analogy is bad. But all analogies are bad. I'm not really trying to make a point about Mathematics and Physics here; I'm trying to explore how to write better software documents.
For example, I think one can reason their way towards the notion that there are more real numbers than there are rationals. (I'm not saying everyone is Cantor, but I am saying that bright people can follow his conclusions, and that he didn't need to build a machine to discover them.) This contrast with Physics. I don't think anybody thought up the quantum Hall effect before it was discovered experimentally, even though both quantum mechanics and the Hall effect were already laying around. Physics is wicked.
The second most famous equation in Physics is due to Newton, F = ma. With calculus, we economically write the more general form F = dp/dt, where p represents the momentum of the object. In all our ordinary experience, momentum is the product of the object's mass and velocity, p = mv. Newton's second law tells us that forces impart changes to an object's momentum.
Embracing the analogy, we can say that F = dp/dt is a normative specification for how the universe works. Strictly speaking, you don't need to convey much else. You don't need an appendix containing the history of Leibniz and Newton. As long as you already know calculus and the definition of p, you're off to the races.
Knowing calculus is a technical skill. If you grok fluxions, then you understand what F = dp/dt really means. Given some measurements, you can make some meaningful, quantitative remarks about an object's behavior.
Maybe it's technical skill like being able to read UML. If you look at the arrows on a class diagram (whether inheritance or composition), then you can tell what depends on what in the picture. If there are cycles, then you might be able to say something intelligent about the quality of the design.
Einstein realized that some pretty weird things happen when objects move at speeds that are non-trivial fractions of the speed of light. Our humdrum definition of momentum grows some wrinkles. Our familiar p = mv turns out to be the low velocity approximation of the more general p = mv/sqrt(1-v^2/c^2).
The normative formula, F = dp/dt is still quite correct! Even with Einstein's discovery, I wouldn't need to rewrite my normative spec. But it's going to be misunderstood by anybody who only knows the low velocity definition of p. When you start to pull people out of the domain in which they are most comfortable, normative isn't enough.
So, if "spec" is too pretentious for what we do, what's the alternative? Our documents need to be more humble. It's a happy coincidence of English that humble and human are such similar words, because when trying to evolve solutions to wicked problems, it's all about people communicating.
Specs are perfectly good work products that describe what something has to do, and they have a crucial role in software development. But not every document is an end unto itself. Many are living documents. What could we call those documents, which people use as tools to build a work product?
Narratives.
Specs are ends of well-defined efforts. Narratives are means of attacking wicked problems. In literature, the best narratives uncover profound truths about the human condition. In software development, the best narratives collect the relevant principles needed to attack the problem at hand.
Einstein elucidated the principle that the speed of light is constant, regardless of the speed of the observer. This principle drove a rethinking of the definition of momentum, but it only matters for objects outside of our familiar experiences. However, when attacking problems where the principle is relevant, we'd better make it clear. I'm going to start calling what I do in my day job narrative-writing instead of spec-writing, and see where that leads.
