How many servers are best in FCFS system?

In this section, we study the question of ``how many servers are best?'' in an M/PH/$k$/FCFS queue (single priority class) with respect to minimizing mean response time. Matrix analytic methods allow us to evaluate the mean response time in a wide rage of loads and job size variability in this setting (see Section 3.2), and this complements the prior work with respect to ``how many servers are best'' in FCFS systems.

Figure 4.1: How many servers are best in a single server (FCFS) system? (a) The optimal number of servers as a function of the load, $\rho$, and the squared coefficient of variation of the job sizes, $C^2$. (b) Mean response time as a function of the number of servers at various job size variabilities ($C^2=1,4,16,64$).

(a)

(b)

Figure 4.1(a) shows the optimal number of servers as a function of the load and variability of the job size, where the job size has a two-phase PH distribution. All of our results are expressed as a function of the variability of the job size distribution (specifically, its squared coefficient of variation, $C^2$)^4.1, and the server load, $\rho$. While other factors, e.g. the exact form of the job size distribution, might affect our results, we posit that load and variability be the most relevant factors. As is proved by Stidham [188], a single server minimizes mean response time when $C^2=1$ (exponential distribution) or when $C^2=0.5$ (Erlang-2 distribution). Observe, however, that under high job size variability and/or high load, the optimal number of servers is more than 1; we prefer multiple slow servers to a single fast server. For example, at load $\rho = 0.4$ and $C^2 = 20$, we see that three servers are best. Computations are only done for up to six servers -- the level curves shown will continue into the upper right portion of the plot if a larger number of servers is considered.

Figure 4.1(b) shows that for any particular job size variability, $C^2 > 1$, having a larger number of slower servers may reduce the mean response time up to a point, after which further increasing the number of servers increases the mean response time. To understand why, note that by increasing the number of servers (while maintaining fixed total capacity), we are allowing short jobs to avoid queueing behind long jobs -- specifically, an arriving short job is more likely to find a server free. Thus, increasing the number of servers mitigates the impact of job size variability, hence improving performance. If the number of servers is too high however, servers are more likely to be idle, under-utilizing the system resources.