<HTML>
<HEAD>
<TITLE>Does Free Software Production in a Bazaar Obey the Law of Diminishing Returns?</TITLE>
<META NAME="AUTHOR" CONTENT="Jacques Chester">
<H1>Does Free Software Production in a Bazaar Obey the Law of Diminishing Returns?</H1>
</HEAD>
<P>
<HR>
<P>
<BODY>
<H1>Abstract</H1>
<I>Free Software is defined. Brook's Law is introduced as a limiting factor in
software production. A parallel is drawn between Brook's Law and the Law of
Diminishing Returns. Eric S. Raymond's "Bazaar" model is introduced as a possible
exception to the rule. Statistics gathered from an existing Free Software project are
used to demonstrate the nonadherence of "Bazaars" to the law of diminishing
returns. A possible flaw in Raymond's hypothesis is identified, and an economic
alternative is offered. </I>
<H1>Contents</H1>
Those items with an asterisk next to them are included in this copy
of the essay.
<P>
<OL>
<LI>Abstract *
<LI>Contents *
<LI>Introduction *
<OL>
<LI>Free Software Defined *
</OL>
<LI>Brook's Law and The Law of Diminishing Returns *
<LI>The "Bazaar" Introduced *
<LI>The GNOME Project *
<LI>Examination and Discussion of GNOME Data
<LI>Conclusion
<LI>Endnotes *
<LI>Bibliography
<LI>Appendices
<OL>
<LI>Acknowledgements and Thanks
<LI>The Opensource Definition
<LI>Caveats
<LI>Economic Principles in Free Software
</OL>
</OL>
<P>
<HR>
<P>
<H1>Introduction</H1>
<P>
This essay seeks to examine the rapidly emerging production method
of "Free Software" or "Opensource Software". It outlines analogues
between Economic theory and Software Engineering in order to bring
economic analysis to bear on the area.
<P>
Specifically, it aims to provide a quantitative analysis of what has
until now been primarily examined in a qualititative way. Free
Software has existed in one form or another since the very early days
of computing, but very little attention has been paid to it until
recently. Many Free Software projects have achieved significant or
even dominant positions in their marketplace, and more firms are starting
to utilise or release Free Software.
<H2>Free Software Defined</H2>
Definitions of Free Software (also know as "OSS", for Open Source
Software) are typically framed in two perspectives: Ideological and
Legal.
<P>
The two major ideological principles underlying Free Software are
the protection of user/programmer choice, and the belief that
the best solutions must be shared.
<P>
The first principle arose from
Richard M. Stallman's dismay at the rise of proprietary software
as the dominant format. Stallman believes that proprietary (also
known as closed-source) software is a violation of the individual's
right to choose other packages. He argues that access to the
sourcecode grants freedoms to modify and augment without being
"locked in" to one company's whims. Further, he argues that sourcecode
access gives users a choice to go their own way, in defiance
of the company's wishes should they be detrimental.
<P>
The second principle, that good solutions should be shared, arises
from the so-called "Hacker Culture". Within this culture, brainpower
is seen as a limited resource, which should not be wasted on
unnecessarily reinventing the wheel. It is reasonable, therefore,
that all solutions (embodied in sourcecode) should be available
for anyone to use. The corrollary is that withholding solutions (or
source) is effectively <I>evil</I>, inasmuch as it is <I>wasteful</I>
of resources.
<P>
The legal foundations of Free Software stem from a careful blend of
Contract and Copyright laws. Free Software Licenses use the principles
of Contract law to create their terms. Typically, these ensure that
an author's version of the source is always available and that any
modification made by anyone is likewise available under the same terms.
Some licenses go so far as to impose these terms onto software where
opensource code has been added as an external source.
<P>
If the user doesn't agree to the License, the law of Contract renders
the terms of the License effectively powerless without punitive terms.
However, at this point, standard Copyright laws take effect, and the
user is granted no rights whatsoever. There is significant coercion,
then, to accept the terms of the License on a contractual basis.
<P>
To place Free Software in an economic framework is considerably more
difficult, but quite profitable. There are a range of issues and
outcomes that emerge naturally from the application of elementary
economic thought to a Free Software "economy". The main body of this
essay assumes that just such a framework has been established. However,
the description of such a framework is long and outside the scope of the
main body of this essay, soh a short treatise on the elementary
economic framework of Free Software can be found in Appendix IV.
<H1>Brook's Law and The Law of Diminishing Returns</H1>
<BLOCKQUOTE><I>
According to Fred Brook's law, adding people to a late
project makes it later. It's like adding gas to a fire.
New people need time to familiarise ... their training
takes up the time of ... [other] people ... and merely
increasing the number of people increases the complexity
and amount of project communication. Brooks points out
that the fact that a woman can have a baby in nine
months does not imply that nine women can have a baby in
one month.
<P>
Managers need to understand ... More workers working
doesn't necessarily mean more work will get done.
<BLOCKQUOTE>--Steve McConnell, "Code Complete"</BLOCKQUOTE>
</I></BLOCKQUOTE>
<P>
In his essay "The Magic Cauldron", Eric S. Raymond estimates that almost 95% of
software development is "in-house". This is the traditional meal-ticket of
programmers and software engineers. It is from this heartland that Brook's Law is
drawn.
<P>
Brook's Law - specifically - is that adding people to a late project willy-nilly
will only make it later. Brooks derived this law from his own personal experience
as a project manager on IBM's original OS/360 project. In his book, <I>The
Mythical Man-Month</I>, Brooks pointed out the fallacy of simply throwing more
"man-hours" (labour units) at the project in order to deliver it earlier.
<P>
According to McConnell, Brook's analysis of his own laws suggests an exception to
the rule. <I>" ... if a project's tasks are partitionable, you can divide them
further and assign them to ... people who are added late to the project."</I>
<P>
In short, we can summarise Brook's Law in two parts:
<OL>
<LI> If, in a (late) project, we can further subdivide tasks efficiently,
we can add extra programmers or software engineers without penalty.
<LI> Otherwise, if tasks cannot be easily or effeciently subdivided, there
will be a penalty for adding extra programmers or software engineers.
</OL>
<P>
Brooks gave several justifications for his law, outlined by McConnell above.
One of the easier to seize upon, in economic and mathematical terms, is the
complexity problems. It is argued that programming requires a large amount of
communications between workers. It can be shown mathematically that if the number
of programmers rises linearly, the number of possible communications paths
between them rises quadratically. This is illustrated by the diagram below.
<P>
<I>{DIAGRAM ILLUSTRATING QUADRATIC COMMS PATHS}</I>
<P>
But Brook's Law is not original. In fact, the first famous instances of the
principles that Brooks expounded are not found in software engineering - they
are found in a rice paddy.
<P>
In the classic example of the Law of Diminishing Returns, Adam Smith asked us
to imagine a rice paddy. We might start with one worker on this paddy, who is
barely able to care for and harvest even a fraction of the paddy. In comparison
to neighbouring paddies, this paddy is woefully inefficient compared to its
neighbours.
<P>
So another worker is added. Productivity rises sharply, as two workers can now
work the field. We can measure this rise in productivity in terms of the total
output and the change in the output - the marginal output.
<P>
We continue to add workers. At first, they replicate each other's work, all,
say, take a quadrant of the paddy and work it. Later, some will specialise. Some
will care for the rice, some will harvest it. Productivity continues to rise.
<P>
But the trend is not endless. At a certain point, adding more workers no longer
causes a rise in productivity. Perhaps these extra workers need to be trained by
other workers. Perhaps they get in one another's way, or there are workers
standing by, idle, as excess working capacity. In any case, the marginal rate of
productivity begins to fall, followed by the average total output.
<P>
It is not difficult to draw parallels with software engineering. Indeed, if
Adam Smith had been working today, he may have used a software project as his
example!
<P>
Let us take a project with one programmer. This programmer has begun to code,
but the project is large. He is unable to produce many lines of code on his
own, having to continuously stop and refer to manuals for unfamiliar areas,
even to remind himself of what part of the project he is dealing with.
<P>
Let us add another programmer. Suddenly, they can divide the work amongst
them, working on two different parts of the program at once. Then we can
add more programmers - including specialists. The program is divided and
subdivided into smaller units of specific purpose, and the specialists
can focus on these parts.
<P>
The subdivision of units and the matching of specialists with program
components means that the productivity rises.
<P>
But again, we come to a certain point where it begins to falter. Programmers
are added who need to be trained on the deep secrets of the existing work.
They need to be introduced to procedures and their tools, diverting time for
existing programmers. They add overhead to communications paths, and some
may spend time fallow, dragging down the average total output once more.
<P>
In economic terminology, Brook's Law might be re-summarised thusly:
<OL>
<LI> If, in a production process, we can further subdivide tasks efficiently,
we can add extra Labour units without penalty.
<LI> Otherwise, if tasks cannot be easily or effeciently subdivided, there
will be a penalty for Labour units.
</OL>
The implication is that there is a curve with a rise, a peak, and fall; that there
is a point of maximum productivity. Indeed, while this rewriting of Brook's Law in
economic terms does not exactly match up to the Law of Diminishing Returns, it is
really the graphs that clinch it.
<P>
<I>{LODR GRAPH}</I>
<P>
This is an example of a typical Law of Diminishing Returns graph. The RED? line shows
Average Total Output (ATO), and the BLUE? line shows Marginal Output (MO).
<P>
The real signature of diminishing returns is the marginal output line. The MO line
in mathematical terms, is the derivative of the ATO line. It shows the rate of
change of the ATO at any given "x-value", or units of labour.
<P>
The classic Law of Diminishing Returns MO is shown. It rises, peaks, and then crosses
zero at the point where ATO peaks. It becomes negative, causing the ATO curve to
nose over and dive.
<P>
Examples from where hard, actual data could be drawn are legion. In this case, it
is generally applicable that this pattern will occur where the Law is true. Indeed,
it is the recurrence of this pattern that is often used as proof of the Law's
applicability.
<P>
<I>{BROOK'S LAW GRAPH}</I>
<P>
This is a graph based on Brook's Law. In fact, it is adapted from a
graph in <I>The Mythical Man-Month</I>, the book in which Brook's Law
is introduced.
<P>
The graph is rendered with "programmes" on the x-axis, and
"statements" on the y-axis. Again, we can see the classic hill-shape
of the Law of Diminishing Returns. And, more importantly, we see the
classic Marginal Output derivative repeat its rise and fall.
<P>
So it becomes reasonable to assert that the Law of Diminishing Returns
and Brook's Law are roughly equivalent. The terminology between them
fluctuates, but the meaning and the consequent graphs are highly
similar. And, just as the Law of Diminishing Returns has been
demonstrated to appear over and over, so has Brook's Law. It is not a
case of a coincidental match of graphs.
<P>
For the rest of this essay, Brook's Law and the Law of Diminishing
Returns will be assumed to be functionally equivalent. This being so,
it becomes viable to apply certain basic economic analyses to Brook's
Law. In particular, we investigate the bold claim that Brook's Law can
be broken.
<H1>The Bazaar Introduced</H1>
In his influential paper, "The Cathedral and the Bazaar", Eric
S. Raymond proposes a theory about how Brook's Law can be overcome. He
describes a model of development which he calls the "Bazaar".
<P>
The Bazaar concept is somewhat multi-faceted. Some key elements of Bazaar projects
include:
<UL>
<LI> Internet or Internet-like ease of communications
<LI> Open-source licensing environment
<LI> Easily-accessible sourcecode
<LI> Easily contactable author/maintainer
<LI> Transparent processes and decision-making
</UL>
<P>
In essence, Raymond argues that when Bazaar conditions exist, the
opposite of Brook's Law becomes true: more programmers mean higher
productivity. In doing so, he proposes several reasons why this might
be so. These arise, almost naturally, from a combination of what
Raymond considers the 'Hacker mindset'[4], ease of communication, and the
open availability of sourcecode.
<OL>
<LI><B>Low Management Overhead:</B> Programmers are able to modify
sourcecode freely. They can add new features and fix bugs without
being directed to do so, and without the necessity of broad and
deep coordination.
<LI><B>Competition between Solutions:</B> As problems to solve
become known in an open-source project, several competing solutions
may emerge from multiple sources. The best solution can then be
chosen and integrated. If there is disagreement, parties can split
off their version of the solution by "forking the tree", in which
case, both solutions are implemented.
<LI><B>Parallel Handling of Vertical Problems:</B> A vertical
problem is one which is standalone (it stands up without support:
vertically). Because these problems can be solved in isolation (the
best example being debugging), the amount that can be solved scales
easily with additional workers. An open-source environment has no
practicable physical limits on how many programmers can
participate, which means they excel in solving vertical problems in parallel.
</OL>
<P>
Raymond asserts that, for these reasons, open-source projects can
break Brook's Law. In particular, he points to the parallel nature of
open-source development, going so far as to say <I>"Given enough
eyeballs, all bugs are shallow"</I>, dubbing this "Linus' Law", in honour
of the Linux system.
<P>
Previously, we sought to establish a link between Brook's Law and the
Law of Diminishing Returns. The outcome was that Brook's Law is
analogous to the Law of Diminishing Returns, but having been derived
as a single case from one field of human endeavour, rather than as a
general law. The two were shown to be equivalent in describing a process.
<P>
By implication, then, Raymond has asserted that the Law of Diminishing
Returns can be broken in a Bazaar environment.
<H1>The GNOME Project</H1>
In order to assess Raymond's claim, this essay uses a high-profile Bazaar-style project. The GNOME project (as with many newer Free Software projects) explicitly adopted Raymond's theory of the Bazaar as its working principles[5].
<P>
The GNOME has properties which make it an ideal source of data:
<UL>
<LI> It is a Bazaar project
<LI> There is a wide range of sub-projects, from the small to the large
<LI> Several hundred Developers and Debuggers are involved, making the
data-set large enough to be useful
<LI> Activity logs available at all times through a public access CVS[6] system
</UL>
<P>
It is also the last property - the extensive use of CVS - that renders
GNOME a useful source of data. The CVS system automatically keeps extensive
logs of all programmer activity. For the GNOME project, all of this
data is publicly available. It is these logs that form the tables on
which this essay is based.
<P>
GNOME also happens to be a quite large project. It is not the largest
Free Software project (the largest probably being the Linux operating
system), but it is one of the largest with public-access CVS logs. It
is its size which allows for the construction of smooth curves.
<H2>Endnotes</H2>
<OL>
<LI> Sourcecode is another form of software. It is from sourcecode that the
harder-to-modify "executable" is derived.<P>
Sourcecode is the 'recipe' for a task a computer might operate. It tells
a computer how the data it is using is structured, how to access it, and
what to do with it. This can be expressed in a number of artificial
"programming languages". If one has access to the sourcecode, it is
possible to intimately understand how a program works. It also becomes
possible to expand or modify its functionality; or to reuse pre-existing
sourcecode in new programs.
<LI> In more detailed terms, we might define the factors thus:
<UL>
<LI><B>Land:</B> The basic resources which are used as 'ingredients'
in the production process. In computer science, these
might be algorithms, or previously existing sourcecode.
<LI><B>Labour:</B> The people who do the majority of the human work, as
directed by the Enterprise. In Free Software, they are
called "Developers" and "Debuggers"
<LI><B>Capital:</B> Essentially, "things that help make things". Capital
typically constitutes machinery. In Free Software,
these are the Networks, Hardware and Software tools
used in the writing and support of the sourcecode.
<LI><B>Enterprise:</B> The factor which brings the other three together
to create the final product. In Free Software, the
line between Labour and Enterprise is blurry.
</UL>
<LI> Meaning "Lines Of Code". Probably <I>the</I> most-used and most popular
metric in software engineering. It refers to a single line of instructions
for the computer that is not blank or a comment intended for human use.
Because, typically, only one instruction is placed per line, LOC provides a
usable measurement of both the gross size and general complexity of a given
project.
<LI> "Hacker" is a term of considerable stigma in mainstream
society, usually taken to mean a person who maliciously
trespasses on other people's computer systems.
<P>
Raymond's use of the term is, in fact, the original
computer-world meaning. He takes a hacker to be a person who
delights in problem solving (especially in programming), and
who believes that once a problem is solved, the solution ought
to be shared.
<P>
The more common use - intruder - is delineated by traditional
hackers with the word "crackers". Hackers take great lengths
to distance themselves from crackers and their activities.
<LI> Ironically, Miguel de Icaaza, in private conversation with the essay's
author, said that he did not himself believe that it was a case of
"The Cathedral <I>versus</I> the Bazaar". He saw them as two extremes
between which there are many comprimises - two ends of the same stick.
<LI> Concurrent Versioning System. CVS is a change-management system. It provides
a central point of work for multiple programmers working on the same
sourcecode. Programmers can 'check out' code, work on it on their own
computer, and then 'merge' their changes back into the original.
<P>
Apart from its usefulness in centralising code storage and management, CVS
provides change-tracking capabilities. It keeps 'delta-files', which
list every change made to any file, at any time, by any programmer. This is
normally used as a sort of super-powered "undo" function.
</BODY>
</HTML>