Apache: The feather floats to the top

By David Walker (Google profile)

It wasn't much of a start. The terse official history of the Apache Web server records that the program began life in 1995 as a mere set of patches, collected and distributed by a programmer named Brian Behlendorf. The patches went on to the NCSA Web server, at that time the Web leader. Hence the name - not "Apache server" but "a patchy server". By mid-1995 a core group of eight programmers had evolved the patches into a program in its own right, renamed it in honour of the Apache Indian tribe, and got it all the way to a public release - Apache 0.6.2. To add to the American Indian flavour, they gave their new server an emblem: a slightly garish and completely harmless-looking pink feather.

In the months that followed, Apache showed it was anything but harmless. In August 1995, a software firm called Netcraft began systematically polling sites to check which server they were running. In Netcraft's first survey, Apache ran 3.5 per cent of all Web domains (see more), well behind leaders NCSA and CERN. Within eight months it had overtaken both of them. By late 1998, sitting astride the Unix operating system, Apache ran more than half the Web sites on the planet. The software's licence made it essentially free.

In February 1996, a new entrant appeared in Netcraft's closely-watched survey: "Microsoft Internet Information Server", initially running a mere 45 domains. Bundled with business versions of Windows, and sporting the familiar Windows interface style and massive corporate credibility, the new entrant quickly attracted Microsoft-centric firms and then hosting services. After an early surge, IIS seemed to plateau in 1998, with market share just over 20 per cent while Apache climbed to 60 per cent. Then in mid-2001 several large hosting and domain name companies started moving to IIS, taking with them a seven-digit number of mostly small sites. By early 2002, Netcraft's server survey graph featured an upward-sloping IIS plot that mirrored Apache's downward-sloping one. And shining further attention on the IS-Apache contest, the British Royal Family's Web developers also moved from Apache to IIS.

Apache's admirers fretted. Might the Microsoft server consign Apache to the same fate as Netscape, Real, Borland, Lotus and other would-be rivals? IIS groaned under a flawed architecture that demanded frequent server restarts. Its Windows underpinnings had been designed for secure corporate environments, not the exposed world of the Web. The conventional wisdom said Microsoft's server was insecure and inferior. Yet it kept gaining ground. "Maybe the admins are right and we're wrong," puzzled Ziff-Davis technical expert Larry Seltzer (see more).

But the figures masked underlying changes that have taken the Web in the opposite direction ever since.

First, the tech recession has forced IT managers to look more closely at software costs - and Apache's cost savings attract even the very largest corporations, according to Forrester Research vice-president John Rymer. When firms like IBM embrace Linux and other open-source programs, free software's credibility rises.

On top of that, IIS's security has come under increasing scutiny as a series of "worms" have targeted IIS vulnerabilities. In July 2001 the Code Red worm appeared; in September its successor, Nimda, arrived. The respected research group Gartner quickly issued a historic "advisory" (full version):

"Gartner recommends that enterprises hit by both Code Red and Nimda immediately investigate alternatives to IIS, including moving Web applications to Web server software from other vendors, such as iPlanet [the former Netscape server, now owned by Sun] and Apache. Although these Web servers have required some security patches, they have much better security records than IIS and are not under active attack by the vast number of virus and worm writers."

The September 2001 collapse of the World Trade Center towers merely reinforced the message. Security and cost-avoidance are in; risk and large budgets are out. In this environment Apache, free and with its roots in the security-conscious Unix world, has an edge Microsoft cannot easily blunt.

So since early 2002, Apache has clearly reversed its decline in the Netcraft market share figures. As it approaches its tenth year, it commands 68 per cent of the market - more than ever before. Microsoft holds just 21 per cent - its lowest figure since mid-2001.

Microsoft is now battling to redraw the picture that many systems chiefs have of IIS, as a flawed, insecure platform. In January 2002, Bill Gates famously made "trustworthy computing" his company's new mantra. IIS 6 has replaced the server's previously wide-open security settings with smarter defaults, and reworked much of the flawed architecture that once necessitated frequent IIS restarts. Microsoft's .Net looks attractive to many developers. Forrester's John Rymer says that although .Net's adoption is hard to measure, version 6.0 and .Net have given IIS "very good momentum" in large companies.

But IIS's progress wins few plaudits. A string of company-sponsored research papers, comparing the costs of Windows and Linux-based applications like Apache, has made little impact on an IT community suspicious of paid "research" findings. (Worse, one Microsoft-sponsored paper found Linux/Apache provided lower "total cost of ownership" than Windows/IIS.) Meanwhile, new IIS patch announcements flow weekly. Any server administrator finds her log files constantly filling with requests like "GET /c/winnt/system32/cmd.exe?/c+dir HTTP/1.0" - calling cards from new worms, crawling the Internet in a blind search for IIS vulnerabilities. Two years into "trustworthy computing", Microsoft's reputation has made little obvious progress. As the Gartner research group puts it, "the worms in 2003 showed there's a long way to go before Windows is secure".

Beyond security and cost, Apache attracts Web site administrators because of the way it can be configured and maintained.

Matthew Berk is a former research director at Jupiter Research who now works as an independent IT analyst. Before Jupiter, he worked as head of technology for first inc.com and then edu.com. He's thus in the perfect position to understand Apache's attractions. And the Apache feature that he talks about most enthusiastically is transparency - its ability to tell the administrator how it is set up and what it is doing. "Apache is incredibly clean and easy to understand even if you're not particularly expert at systems administration," he says. "With IIS you don't really always know what's going on inside that box. With Apache it's incredibly clear. You know what to turn on; you know what to turn off." Paradoxically, Apache gains this transparency not through a Windows-style GUI but through the retro simplicity of a heavily-commented text file called httpd.conf - an interface which explains the effect of every change you might make.

Meanwhile, other factors keep driving Apache's adoption:

• Linux is continuing to gain ground as a server platform, and Linux servers mostly run Apache.
• The Apache Software Foundation, which controls Apache's growth, has won respect. Declared Gartner in August 2003: "The ASF is a hallmark of quality in open source software" (see more).
• From its beginnings as a server, Apache has grown into an open-source project second only to Linux in its scope and ambition. It now encompasses everything from a Web content management system (Apache Lenya) to a collection of open-source Java solutions (Jakarta) to a Java-based project management tool (Maven).
• A lightweight Apache-oriented server-side scripting language, PHP, and a quick-and-dirty database, MySQL, are gaining in popularity.
• As Matthew Berk and John Rymer both point out, Apache's Java Web application platform, Tomcat, is being adopted everywhere from small Web development shops to the largest sites on the Web.
• Apache has gradually become a simple installation on Windows, the workhorse operating system that most non-experts use. You can download a bundled package of Apache, MySQL and PHP from DeveloperSide.net and be running them on Windows in ten minutes. That exposes Apache to more people every day.

Microsoft has little reason to give up the fight. For one thing, Netcraft's figures clearly understate IIS's reach by stressing the number of sites rather than their size and influence. First, small and mid-range sites naturally gain more from Apache's lack of up-front cost. Second, Apache completely dominates the world of "virtual hosting", where large server companies runs sites or simply park domains for millions of smaller businesses and individuals. In recent months, Microsoft server software specialist firm Port80 has analysed at the server software running the Web's 1000 most trafficked sites, as measured by Nielsen NetRatings. Its conclusion? Microsoft IIS holds a narrow 43 percent share among the top 1,000, with Apache at 40 per cent share and the former Netscape server third with nine per cent. And Microsoft continues to work on IIS's ability to run every type of site and provide "trustworthy computing". Says Forrester's Rymer: "I don't think you should underestimate how serious they are about this."

But for the web as a whole, Apache retains its lead - and shows no sign of losing it. For now, the feather is floating upward.