Are standards best defined by documents, reference implementations or referencesource code? Perhaps unit tests are the answer!
Patrick Logan (Making it Stick) refers to
an article by John Udell musing on whether standard (open source) implementations are more valuable than the standard specification documents.
I think that well documented unit tests would be better still. I often meditate on the saying "the proof of the pudding is in the tasting". Well-written unit tests serve as good pudding tasters. The purpose of standards is to specify interfaces. Unit Tests concisely define how a system behaves under various conditions. They have the added benefit that they can be run against a candidate implementation and directly measure the compliance to the standard.
Unit tests are also compatible with the closed-source parts of the world. They only interact with the interface of a system. The unit tests should define all behaviours that together are sufficent to meet the standard. If the test suite passes, then the implementation details are irrelevant, and no revelation of inner workings is necessary. The test suite can help in determining whether the problem lies in an implementation as opposed to a user or external component relying on implementation side-effects.
Unit tests can also be successfully integrated into documentation. It is much easier and clearer to do so than intregating reference implementation code into the documentation, which tends to include distracting details. An excellent example of this integration is the doctest module included with Python. In fact with Python+doctest you can incorporate documentation, unit tests together with a reference implementation.
Doctests (unit tests intregated into documentation) generally consist of a few annotated examples of normal cases, plus the important edge cases. The doctest approach encourages frequent examples. Examples are the often the best way to communicate an idea or requirement, and resolve a lot of the ambiguity inherent in normal language. On that note, I really should lead by example and provide an example of doctests as examples...
"""Demonstrate doctests as specifications. """
__author__ = 'Tim Wegener '
import types
import doctest
def fibonacci(n):
"""Return the n-th Fibonacci number.
The function is used like so:
>>> fibonacci(7)
13
By definition, the following hold:
>>> fibonacci(0)
0
>>> fibonacci(1)
1
For other values of n greater than 1, F(n) = F(n-1) + F(n-2)
>>> fibonacci(2)
1
>>> fibonacci(3)
2
The result is undefined for negative and non- integers.
>>> fibonacci(-1)
Traceback (most recent call last):
ValueError: result is undefined for negative integer n
>>> fibonacci(1.4)
Traceback (most recent call last):
ValueError: result is undefined for non-integer n
>>> fibonacci(1j)
Traceback (most recent call last):
ValueError: result is undefined for non-integer n
"""
if not isinstance(n, int):
raise ValueError("result is undefined for non-integer n")
if n < 0:
raise ValueError("result is undefined for negative integer n")
if n == 0:
result = 0
elif n == 1:
result = 1
else:
f_n_minus_2 = 0
f_n_minus_1 = 1
result = 0
for i in range(2, n+1):
result = f_n_minus_1 + f_n_minus_2
f_n_minus_2 = f_n_minus_1
f_n_minus_1 = result
return result
def run_tests():
"""Run all doctests in the module."""
import doctest
doctest.testmod()
# Define behaviour when this module is called as a script.
if __name__ == '__main__':
run_tests()