Why virtual server?
With the explosive growth of the Internet and its increasingly important role in our lives, the traffic on the Internet is increasing dramatically, which has been growing at over 100% annual rate. The workload on the servers is increasing rapidly so that servers will be easily overloaded for a short time, especially for a popular web site. To overcome the overloading problem of the servers, there are two solutions. One is the single server solution, i.e. to upgrade the server to a higher performance server, but it will soon be overloaded when requests increases so that we have to upgrade it again, the upgrading process is complex and the cost is high. The other is the multiple server solution, i.e. to build a scalable network service system on a cluster of servers. When load increases, we can simply add a new server or more into cluster to meet the increasing requests, and commodity server is of highest performance/cost ratio. Therefore, it is more scalable and more cost-effective to build server cluster system for network services.
There are several methods to construct the cluster of servers.
- DNS based load balancing cluster
- Dispatcher based load balancing cluster
DNS load balancing is probably the simplest method to build network service cluster. It uses Domain Name System to distribute requests to different servers through resolving the domain name to different IP addresses of servers. When a DNS request comes to the DNS server to resolve the domain name, the DNS server gives out one of server IP addresses based on scheduling strategies, such as in round-robin manner, then subsequent requests from clients using the same local caching name server are sent to the same server in the specified time-to-live (TTL) of name resolving.
However, due to the caching nature of clients and hierarchical DNS system, it easily leads to dynamic load imbalance among the servers, thus it is not easy for a server to handle its peak load. The TTL value of a name mapping can't be well chosen at the DNS server, with small values DNS traffic is high and DNS server will be a bottleneck, and with high values the dynamic load imbalance will get even worse. Even the TTL value is set with zero, the scheduling granularity is per host, different users' access pattern may lead to dynamic load imbalance, because some people may pull lots of pages from the site, and others may just surf a few pages and go away. Moreover, it is not so reliable, when a server node fails, the clients who maps the name to the IP address will find the server is down, and the problem still exists even if they press "reload" or "refresh" button in their browsers.
Dispatcher, also known as load balancer, can be used to distribute load among servers in a cluster, so that the parallel services of servers can appear as a virtual service on a single IP address, and end users interact as if it were a single server without knowing all servers in clusters. Compared to DNS based load balancing, dispatcher can schedule requests at fine granularity, such as per connection, for better load balancing among servers. Failure can be masked when one server or more fail. Server management is becoming easy, and administrator can take a server or more in and out of service at any time, which won't interrupt services to end users.
Load balancing can be done in two levels, application-level and IP-level. For example, Reverse-proxy and pWEB is an application-level load balancing method to build a scalable web server. They forward the HTTP request to the different web servers in the cluster, get back the result, and then return it to the clients. Since the overhead of dealing the HTTP requests and replies in the application-level is high, I believe the application-level load balancer will be a new bottleneck when the number of server nodes increase to 5 or more, which depends on the throughput of each server.
I prefer the IP-level load balancing, because the overhead of IP load balancing is small and the maxim number of server nodes can reach 25 or up to 100. That's IP Virtual Server code designed for. How it works will be explained in detail in the next section.