Computer-system evaluation

Computer-system evaluation

The evaluation of performance, from the perspectives of both developers and users, of complex systems of hardware and software. Modern computer-based information systems have become increasingly complex because of networking, distributed computing, distributed and heterogeneous databases, and the need to store large quantities of data. People are relying increasingly on computer systems to support daily activities. When these systems fail, significant breakdowns may ensue. See Distributed systems (computers), Local-area networks

A computer system can fail in two major ways. First, functional failure occurs when the system fails to generate the correct results for a set of inputs. For example, if an information system fails to retrieve records that match a set of keywords, or if an air-missile tracking system fails to distinguish between a friendly and enemy missile, a functional failure has occurred. Second, performance failure occurs when the system operates correctly but fails to deliver the results in a timely fashion. For example, if an information system takes a longer time than users are willing to wait for the records they requested, the system is said to fail performance-wise even though it may eventually retrieve the correct set of records. Also, if the air-missile tracking system fails to detect an enemy missile in sufficient time to launch a counterattack, the system manifests performance failure.

Therefore, in designing a computer system it is necessary to guarantee that the end product will display neither functional nor performance failure. It is then necessary to predict the performance of computer systems when they are under design and development, as well as to predict the impact of changes in configurations of existing systems. This requires the use of predictive performance models.

The input parameters of performance models include workload intensity parameters, hardware and system parameters, and resource demand parameters. The outputs generated by performance models include response times, throughputs, utilization of devices, and queue lengths. There are analytic, simulation, and hybrid performance models. Analytic models are composed of a set of equations, or computational algorithms, used to compute the outputs from the input parameters. Simulation models are based on computer programs that emulate the behavior of a system by generating arrivals of so-called customers through a probabilistic process and by simulating their flow through the system. As these simulated entities visit the various system elements, they accumulate individual and system statistics. Hybrid models combine both analytic and simulation approaches by, for example, replacing an entire subsystem in an analytic model by an equivalent device whose input–output behavior is obtained by simulating the subsystem. Analytic models can be exact or approximate. See Simulation

Approximations are needed either when there is no known mathematically tractable exact solution or when the computation of an exact solution is very complex. Modern computer systems are very complex because of ubiquitous networking, distributed processing using client–server architectures, multiprocessing, and sophisticated input–output subsystems using network-attached storage devices. For this reason, most computer system performance models are approximate models. See Client-server system, Multiprocessing

The design and development of complex software systems is a time-consuming and expensive task. Performance modeling techniques must be integrated into the software development methodology. This integrated approach is called software performance engineering. One goal is to estimate the resource consumption of software under development so that performance models can be used to influence the architecture of the software under development. Better estimates on the resource consumption are obtained as the software development process evolves. See Information systems engineering, Software engineering