Ever find yourself standing in checkout line at the grocery store, assessing all the factors determining how long you are going to wait, making calculations about whether you are in the fastest line or not? It’s pretty simple to figure out if you should switch lines. You just look at the length of the line next to you, and the size of the carts in that line, multiply them together and then compare to your line.
It’s a basic calculation but it’s the same formula that is the basis for many different kinds of analysis of web applications. I often find myself using the checkout line analogy to explain how I do performance tuning. The principle is known as Little’s Law, and it relates the queue length, the total time spent in the checkout line, and the time spent checking each customer out. It says the number of things in a system, N, is equal to the product of the rate at which things pass through the system, T, and the total time spent in the system, R. Another version of it says that the time you should expect to spend checking out (R) is the time it takes that junior checker to process a cart full of groceries (1/T) multiplied by the number of people ahead of you (N). You really didn’t need someone like Little to tell you that, did you?
I find people who spend a lot of time analyzing system statistics feel the same way. They develop an intuitive sense of how the system responds under certain conditions, like increased load (N) or maxed out throughput (T). Intuition makes a great beginning but a lousy conclusion*. Performance management requires systematic, quantitative analysis. Understanding how to use Little’s Law will allow you to reinforce intuition with evidence. It can be applied to any problem independently, and to the system at different levels. You just need to draw a box around the part of the system you are talking about and plug in the numbers.
For instance, consider a complete web application that has 1000 requests currently being processed or queued for processing. We also know it’s processing at a rate of 200 requests per second. The CPU is maxed out so we also know that it can’t increase that rate. What happens if we suddenly double the load on the server? Effectively you double N, the number of things in the system. The throughput (T) is still fixed at 200 requests per second, so that means the response time (R) has to double, according to Little’s law.
Now narrow the scope. Let’s draw the box around a connection pool. Let’s say you’ve got a DB connection pool limited to 5 connections, and that your average database operation takes 10 ms. Is this connection pool big enough? Let’s say you need to support 1000 transactions per second. The question is will you ever have more than 5 requests in the pool? Because that means somebody is waiting for a free connection. From Little’s Law, N = 0.01 seconds (R) x 1000 requests/second (T), or 10. You would need to double the size of the connection pool to accommodate that throughput comfortably (not strictly true, of course, unless the time between request arrival was constant).
The point is that you can refer to Little’s Law for just about any part of a system. It also forms the basis for many other important laws used in analyzing web applications, including the Response Time and Utilization laws. In this blog I’ll have lots more examples where we use Little’s law to analyze system performance.
Hopefully this at least gives you something to think about next time your standing in a long line at the checkout counter.