The technique of operational analysis (OA) is used in the study of systems performance, mainly for estimating mean values of various measures of interest, such as, number of jobs at a device and response times. The basic principles of operational analysis allow errors in assumptions to be quantified over a time period. The assumptions which are used to derive the operational analysis relationships are studied. Using Karush-Kuhn-Tucker (KKT) conditions bounds on error measures of these OA relationships are found. Examples of these bounds are used for representative performance measures to show limits on the difference between true performance values and those estimated by operational analysis relationships. A technique for finding tolerance limits on the bounds is demonstrated with a simulation example. 1. Introduction The analysis of the performance of a network of devices is important in many areas. Computer systems and industrial manufacturing systems are two examples. The types of networks considered in this paper are operationally connected, queue and server devices. That is, each device is connected in some way with every other device in the network and each device may have a queue assigned to it. Certain information about these types of networks may be obtained using a technique known as operational analysis (OA). Relationships used to estimate performance measures (PMs) of networks may be derived in operational analysis under a few restrictive assumptions. OA is a technique which was originally defined as an aid in computer system performance analysis [1–6]. It can be an aid in the understanding of system performance in general [7] and is a complementary approach to stochastic analysis used in many networks of servers performance analyzes and in computer programs [8–14]. Other used or suggested applications for the OA approach include telecommunications [15], E-commerce [16, 17], flexible manufacturing systems [18], and Petri nets [19–21]. The performance measures derived are such things as average number of units at a device, average response time, and throughput. The behavior of a single, arbitrary device in a network will be considered. Two basic principles define the OA approach [2].(1)All assumptions that are made in analyzing the performance of a real system should be subject to direct verification.(2)All variables that appear in any equation which characterize the performance of a real system should be verifiable by direct measurement. The validity of PM equations developed using these principles can be shown for a particular set of data
References
[1]
J. P. Buzen, “Fundamental operational laws of computer system performance,” Acta Informatica, vol. 7, no. 2, pp. 167–182, 1976.
[2]
J. P. Buzen, “Operational analysis: an alternative to stochastic modeling,” in Performance of Computer Installations, pp. 175–194, North-Holland, Amsterdam, The Netherlands, 1978.
[3]
J. P. Buzen and P. J. Denning, “Operational treatment of queue distributions and mean value analysis,” Tech. Rep. CSD-TR 309, Department of Computer Science, Purdue University, 1979.
[4]
J. P. Buzen and P. J. Denning, “Measuring and calculating queue length distributions,” Computer, vol. 13, no. 4, pp. 33–44, 1980.
[5]
P. J. Denning and J. P. Buzen, “The operational analysis of queueing network models,” Computer Surveys, vol. 10, no. 3, pp. 225–261, 1978.
[6]
J. P. Buzen, “From stochastic modeling to operational analysis: the journey begins,” in Fundamental Concepts in Computer Science, E. Gelenbe and J. Kahane, Eds., pp. 141–149, Imperial College Press, London, UK, 2009.
[7]
E. D. Lazowska, J. Zahorjan, G. S. Graham, and K. C. Sevcik, Quantitative System Performance, Prentice-Hall, Englewood Cliffs, NJ, USA, 1984.
[8]
R. Suri, “Robustness of queueing network formulas,” Journal of the ACM, vol. 30, no. 3, pp. 564–594, 1983.
[9]
E. D. Lazowska, J. Zahorjan, and E. C. Sevcik, “Computer system performance evaluation using queueing network models,” Annual Reviews in Computer Science, vol. 1, pp. 107–137, 1986.
[10]
B. V. Ivanovskii, “Operational analysis of communication networks with lockups,” Automatic Control and Computer Sciences, vol. 22, no. 3, pp. 26–32, 1988.
[11]
P. J. Denning, “Queueing in networks of computers,” American Scientist, vol. 79, pp. 206–209, 1991.
[12]
P. J. Denning, “In the queue: mean values,” American Scientist, vol. 79, pp. 402–403, 1991.
[13]
Y. Dallery and X.-R. Cao, “Operational analysis of stochastic closed queueing networks,” Performance Evaluation, vol. 14, no. 1, pp. 43–61, 1992.
[14]
M. El-Taha and S. Stidham Jr., “Deterministic analysis of queueing systems with heterogeneous servers,” Theoretical Computer Science, vol. 106, no. 2, pp. 243–264, 1992.
[15]
J. P. Buzen, “An overview of performance prediction in MVS systems and SNA networks,” in Proceedings of the ACM Fall Joint Computer Conference, pp. 751–759, Dallas, Tex, USA, 1986.
[16]
D. A. Menasce and V. A. Almeida, Scaling for E-Business: Technologies, Models, Performance, and Capacity Planning, Prentice-Hall, Englewood Cliffs, NJ, USA, 2000.
[17]
D. K. Buch and V. M. Pentkovski, “Performance characterization experience of multi-tier E-business systems using queuing operational analysis,” in Proceedings of the IEEE International Symposium on Performance Analysis of Systems and Software, Tucson, Ariz, USA, 2001.
[18]
M. P. Fanti, B. Maione, G. Piscitelli, and B. Turchiano, “Two method for real-time routing selection in flexible manufacturing systems,” in Proceedings of the IEEE International Conference on Robotics and Automation, pp. 1158–1166, May 1992.
[19]
W. M. Zuberek, “Throughput analysis in timed Petri nets,” in Proceedings of the 35th Midwest Symposium on Circuits and Systems, pp. 1576–1580, Washington, DC, USA, 1992.
[20]
W. M. Zuberek, “Throughput analysis of simple closed timed Petri net models,” in Proceedings of the 36th Midwest Symposium on Circuits and Systems, pp. 930–933, Detroit, Mich, USA, August 1993.
[21]
G. Chiola, C. Anglano, J. Campos, J. M. Colom, and M. Silva, “Operational analysis of timed Petri nets and application to the computation of performance bounds,” in Quantitative Methods in Parallel Systems, F. Baccelli, A. Jean-Marie, and I. Mitrani, Eds., Esprit Basic Research Series, pp. 161–174, Springer, Berlin, Germany, 1995.
[22]
J. A. Brumfield, Operational Analysis of Queuing Phenomena [Ph.D. thesis], Department of Computer Science, Purdue University, 1982.
[23]
N. M. Bengtson, “Using operational analysis in simulation: a queuing network example,” Journal of the Operational Research Society, vol. 39, no. 12, pp. 1125–1136, 1988.
[24]
N. M. Bengtson, “Measuring errors in operational analysis assumptions,” IEEE Transactions on Software Engineering, vol. 13, no. 7, pp. 767–776, 1987.
[25]
N. M. Bengtson, “Operational analysis revisited: error measure limits of assumptions,” GSTF International Journal on Computing, vol. 3, no. 2, 2013.
[26]
K. C. Sevcik and M. Klawe, “Operational analysis versus stochastic modeling of computer systems,” in Proceedings of the Computer Science and Statistics: 12th Annual Symposium on the Interface, pp. 177–184, Waterloo, Canada, 1979.
[27]
F. S. Hillier and G. J. Lieberman, Introduction to Operations Research, McGraw-Hill, New York, NY, USA, 8th edition, 2005.
[28]
C. Eisenhart, M. W. Hastay, and W. A. Wallis, Eds., Selected Techniques of Statistical Analysis, McGraw-Hill, New York, NY, USA, 1947.
