Smart Clients: A Framework For Building Scalable Services

Slow response times, network congestion, and ``hot sites of the day'' being overrun by thousands to millions of requests per day are fairly commonplace on the Web today. These problems will only worsen as the Web continues to experience rapid growth. As a result, it has become increasingly important to design and implement network services, such as http, ftp, and web indexing services, to scale gracefully with offered load. Such scalable services must, at minimum, address the following problems:

  • Incremental Scalability: If the offered load begins to exceed a service's hardware capacity, it should be a simple operation to add hardware to increase system capacity. This operation should be transparent to users (i.e., no service interruption).
  • Load Balancing: Load should be spread dynamically among server resources so that clients receive the best available quality of service.
  • Dynamic Resource Recruitment: Related to the issue of scalability, a service should be able to transparently recruit resources to handle peaks in the load. For example, while the US Geological Survey Web site is normally quite responsive, it was left completely inaccessible immediately after a recent Bay Area earthquake in May, 1996.
  • Fault Tolerance: When possible, the service should remain available in the face of host, software, or network upgrades or failures.

Unfortunately, providing these properties for network services while remaining compatible with the de facto URL naming standard has proven difficult. URLs implicitly associate a single hostname, and hence a single point of failure and congestion, with each service. A number of efforts address this limitation by hiding the physical location of a particular service behind a logical hostname. Examples of such systems include HTTP redirect, DNS Aliasing, Failsafe TCP, Active Networks, and Magic Routers. We refer to these systems as ``server-side'' because the transparency mechanism is logically co-located with the servers.

Our work is motivated by the observation that in many cases the client, rather than the server, is the right place to implement transparent access to network services. Client-side solutions offer the main advantage of increased flexibility. For example, clients aware of the relative load on a number of ftp mirror sites can connect to the least loaded mirror to deliver the highest throughput to the end user. Ideally, the selection and connection process takes place without any intervention from the end user. Note that in this example, clients must take into account available network bandwidth to each mirror site as well as the relative load of the sites. Such flexibility would be much more difficult to implement if a server attempted to implement such load balancing for geographically distributed clients.