A Scientific Approach to Conducting Server Benchmark Testing

By CIOReview | Wednesday, July 5, 2017

As computing demands grow, so does the need to maintain high server performance and keep up the right scale. The task of managing servers, though, is not a layman’s task. Data center administrators must constantly keep a track of server performance and make optimal use of hardware resources. A viable way to check how different servers perform is by carrying out benchmark testing. The formula is to conduct a test that is quintessential of what a system usually performs and note the time covered to run the process. Thereafter, the same test is conducted on various other systems and the respective findings are measured.

With the evolution of server architecture, it became all the more tricky to identify different computer systems by merely analyzing their specifications. As a result, metric and benchmark testing came into existence in server environments. On the contrary, understanding server metrics is not as simple as sounds. Every machine has unique functions based on its design and requirements of the operating system or workloads. There are a lot of different variables to keep tabs on.  

Given that server performance does not typically depend on a single factor, the process of its evaluation is arguably no less than a scientific analysis. One of the best approaches to server performance testing involves the application of ‘science’—a systematic series of procedures covering every aspect from soup to nuts. Following are the steps to the scientific method:

Observation: The first priority of a systems administrator, before assessing the performance of a server, is to understand the functions of the server. Is it a virtual platform or will it be running a dedicated application? Knowing the answers to these questions will help the admin form a rough idea as to where to begin the tests and go further.

Hypothesis: After completing the initial research, the administrator can move on to the next step i.e. setting a benchmarking goal. In the absence of a definite goal or direction, the entire process of testing shall go in vain. Therefore, it is crucial that the system operator creates a hypothesis—on which the testing methodology will rest—that can be verified. Quite notably, the assumption (hypothesis) must not be based on data that cannot be verified through benchmark testing.

Prediction: Just like building an assumption, the system admin needs to make a general prediction on how the server test will turn out. For instance, if a server is dedicated to work for applications, the system admin can predict that its performance may be enhanced by assigning extra cores to the workload which in turn will also improve the performance of the application. In some cases, the admin may even predict the degree of improvement and further confirm it through benchmarking.

Setting controls: Once the admin makes a prediction, the obvious implication is that he has set a control. This can be explained with the help of an example—there may a given number of cores assigned to a server. At this point, all that the admin is required to do is change just one setting at a time till he/ she discerns a change in performance that is impactful. In case the admin opts for setting a different control, he might also have to adjust processor settings while leaving the other settings in their primary state.

Testing: Having set the controls, the admin can proceed to the most crucial step—testing. The test should be performed from a basic criterion (usually a known starting point) and the machine’s setup is fine-tuned accordingly. Each test sequence will have an outcome which needs to be documented for later referencing. In this context, a test sequence can be as a modification in the settings on the hardware. Each time a setting is changed, the test must be replayed, and the results recorded. After playing the test sequences for a sufficient number of times, the system admin must compile an organized spreadsheet of all the data to scan through to reach the conclusion.

Conclusion: The last step involves the task of confirming if the application really functions with the setup and resources assigned to it. This can be explained with the help of an example: suppose the application works optimally with the incorporation of lesser cores than it requires. At this juncture, the admin can single out a certain core setting that renders the best server performance as against all the other variables like number of applications currently running, amount of total memory required, software upgrades, and such others. While doing so, he/she must also keep in mind that any change in the variables shall entail further experimentation.