Simulation is a useful tool for the evaluation of a Master Production/Distribution Schedule (MPS). The goal of this paper is to propose a new approach to designing a simulation model by reducing its complexity. According to the theory of constraints, a reduced model is built using bottlenecks and a neural network exclusively. This paper focuses on one step of the network model design: determining the structure of the network. This task may be performed by using the constructive or pruning approaches. The main contribution of this paper is twofold; it first proposes a new pruning algorithm based on an analysis of the variance of the sensitivity of all parameters of the network and then uses this algorithm to reduce the simulation model of a sawmill supply chain. In the first step, the proposed pruning algorithm is tested with two simulation examples and compared with three classical pruning algorithms from the literature. In the second step, these four algorithms are used to determine the optimal structure of the network used for the complexity-reduction design procedure of the simulation model of a sawmill supply chain. 1. Introduction Simulation is a useful tool for the evaluation of planning or scheduling scenarios [1]. Indeed, simulation highlights the evolution of the machine states, WIP (work in process) and queues. This information is useful for “predictive scheduling” [1] or rescheduling. Considering the theory of constraints [2], the optimization of production processes requires maximizing the utilization rate of the bottlenecks. This is the main indicator for evaluating a Master Production/Distribution Schedule (MPS). For this, a useful technique is simulating dynamic discrete events of the material flow [3]. In fact, simulation models of actual industrial cases are often very complex and modelers encounter problems of scale [4]. In addition, many works use the simplest (reduced/aggregated) models of simulation [5–8]. Neural networks can extract performing models from experimental data [9]. Consequently, the use of neural networks has been proposed in order to reduce simulation models [8–10]. To build a neural model, an important issue is determining the structure of the network. The main techniques used to control the complexity of the network are architecture selection, regularization [11, 12], early stopping [13], and training with noise [14]; the last three are closely related [14, 15]. This paper focuses on architecture selection. To determine the optimal structure of the network, two approaches can be used. The first is constructive,
References
[1]
P. Lopez and F. Roubellat, Ordonnancement de la Production, Hermès, Paris, France.
[2]
E. Goldratt and J. Cox, The Goal: A Process of Ongoing Improvement, The North River Press, Great Barrington, Mass, USA, 2nd edition, 1992.
[3]
A. Thomas and P. Charpentier, “Reducing simulation models for scheduling manufacturing facilities,” European Journal of Operational Research, vol. 161, no. 1, pp. 111–125, 2005.
[4]
E. H. Page, D. M. Nicol, O. Balci et al., “An aggregate production planning framework for the evaluation of volume flexibility,” in Proceedings of the Winter Simulation Conference (WSC '99), pp. 1509–1520, Phoenix, Ariz, USA, December 1999.
[5]
S. C. Ward, “Argument for constructively simple models,” Journal of the Operational Research Society, vol. 40, no. 2, pp. 141–153, 1989.
[6]
R. J. Brooks and A. M. Tobias, “Simplification in the simulation of manufacturing systems,” International Journal of Production Research, vol. 38, no. 5, pp. 1009–1027, 2000.
[7]
L. Chwif, R. J. Paul, and M. R. P. Barretto, “Discrete event simulation model reduction: a causal approach,” Simulation Modelling Practice and Theory, vol. 14, no. 7, pp. 930–944, 2006.
[8]
P. Thomas and A. Thomas, “Expérimentation de la reduction d'un modèle de simulation par réseau de neurones: cas d'une scierie,” in Proceedings of the 7ème Conférence Internationale de Modélisation, Optimisation et Simulation des Systèmes (MOSIM '08), pp. 2030–2038, Paris, France, March-April 2008.
[9]
P. Thomas, G. Bloch, F. Sirou, and V. Eustache, “Neural modeling of an induction furnac using robust learning criteria,” Integrated Computer-Aided Engineering, vol. 6, no. 1, pp. 15–26, 1999.
[10]
P. Thomas, D. Choffel, and A. Thomas, “Simulation reduction models approach using neural network,” in Proceedings of the 10th International Conference on Computer Modelling and Simulation (EUROSIM '08), Cambridge, UK, April 2008.
[11]
P. L. Bartlett, “For valid generalization, the size of the weights is more important than the size of the network,” Neural Information Processing Systems, vol. 9, pp. 134–140, 1997.
[12]
G. C. Cawley and N. L. C. Talbot, “Preventing over-fitting during model selection via bayesian regularisation of the hyper-parameters,” Journal of Machine Learning Research, vol. 8, pp. 841–861, 2007.
[13]
H. Drucker, “Effect of pruning and early stopping on performance of a boosting ensemble,” Computational Statistics and Data Analysis, vol. 38, no. 4, pp. 393–406, 2002.
[14]
C. M. Bishop, Neural Network for Pattern Recognition, Oxford University Press, Oxford, UK, 1995.
[15]
P. Lauret, E. Fock, and T. A. Mara, “A node pruning agorithm based on a fourier amplitude sensitivity test method,” IEEE Transactions on Neural Networks, vol. 17, no. 2, pp. 273–293, 2006.
[16]
R. Chentouf and C. Jutten, “Combining sigmoids and radial basis function in evolutive neural architecture,” in Proceedings of the 4th European Symposium on Artificial Neural Networks (ESANN '96), pp. 129–134, Bruges, Belgium, April 1996.
[17]
R. Setiono, “Feedforward neural network construction using cross validation,” Neural Computation, vol. 13, no. 12, pp. 2865–2877, 2001.
[18]
I. Rivals and L. Personnaz, “Neural-network construction and selection in nonlinear modeling,” IEEE Transactions on Neural Networks, vol. 14, no. 4, pp. 804–819, 2003.
[19]
L. Ma and K. Khorasani, “New training strategies for constructive neural networks with application to regression problems,” Neural Networks, vol. 17, no. 4, pp. 589–609, 2004.
[20]
B. Hassibi and D. G. Stork, “Second order derivatives for network pruning: optimal brain surgeon,” in Advances in Neural Information Processing Systems, S. H. Hanson, J. D. Cowan, and C. L. Gilles, Eds., pp. 164–171, Morgan Kaufmann, San Mateo, Calif, USA, 1993.
[21]
M. Cottrell, B. Girard, Y. Girard, M. Mangeas, and C. Muller, “Neural modeling for time series: a statistical stepwise method for weight elimination,” IEEE Transactions on Neural Networks, vol. 6, no. 6, pp. 1355–1364, 1995.
[22]
P. Thomas and G. Bloch, “Robust pruning for multilayer perceptrons,” in Proceedings of the 2nd IMACS/IEEE Multiconference on Computational Engineering in Systems Applications (CESA '98), pp. 17–22, Nabeul-Hammamet, Tunisia, April 1998.
[23]
J. Xu and D. W. C. Ho, “A new training and pruning algorithm based on node dependence and Jacobian rank deficiency,” Neurocomputing, vol. 70, no. 1–3, pp. 544–558, 2006.
[24]
X. Zeng and D. S. Yeung, “Hidden neuron pruning of multilayer perceptrons using a quantified sensitivity measure,” Neurocomputing, vol. 69, no. 7–9, pp. 825–837, 2006.
[25]
X. Liang, “Removal of hidden neurons in multilayer perceptrons by orthogonal projection and weight crosswise propagation,” Neural Computing and Applications, vol. 16, no. 1, pp. 57–68, 2007.
[26]
E. Romero and J. M. Sopena, “Performing feature selection with multilayer perceptrons,” IEEE Transactions on Neural Networks, vol. 19, no. 3, pp. 431–441, 2008.
[27]
I. T. Jollife, Principal Component Analysis, Springer, New York, NY, USA, 1986.
[28]
P. Demartines, Analyse de données par réseaux de neurones auto-organisés [Ph.D. dissertation], Institut National Polytechnique de Grenoble, Valence, France, 1995.
[29]
G. Dreyfus, J. M. Martinez, M. Samuelides, et al., Réseaux de Neurones: Méthodologies et Applications, Editions Eyrolles, Paris, France, 2002.
[30]
H. Stoppiglia, G. Dreyfus, R. Dubois, and Y. Oussar, “Ranking a random feature for variable and feature selection,” Journal of Machine Learning Research, vol. 3, pp. 1399–1414, 2003.
[31]
T. Cibas, F. F. Soulié, P. Gallinari, and S. Raudys, “Variable selection with neural networks,” Neurocomputing, vol. 12, no. 2-3, pp. 223–248, 1996.
[32]
P. Leray and P. Gallinari, “Feature selection with neural networks,” Behaviometrika, vol. 26, no. 1, pp. 145–166, 1999.
[33]
G. Castellano and A. M. Fanelli, “Variable selection using neural-network models,” Neurocomputing, vol. 31, no. 1–4, pp. 1–13, 2000.
[34]
S. Gadat and L. Younes, “A stochastic algorithm for feature selection in pattern recognition,” Journal of Machine Learning Research, vol. 8, pp. 509–547, 2007.
[35]
R. Setiono and W. K. Leow, “Pruned neural networks for regression,” in Proceedings of the 6th Pacific Rim International Conference on Artificial Intelligence (PRICAI '00), pp. 500–509, Melbourne, Australia, August 2000.
[36]
A. P. Engelbrecht, “A new pruning heuristic based on variance analysis of sensitivity information,” IEEE Transactions on Neural Networks, vol. 12, no. 6, pp. 1386–1389, 2001.
[37]
B. P. Zeigler, Theory of Modeling and Simulation, John Wiley & Sons, New York, NY, USA, 1976.
[38]
G. Innis and E. Rexstad, “Simulation model simplification techniques,” Simulation, vol. 41, no. 1, pp. 7–15, 1983.
[39]
R. C. Leachman, Preliminary Design and Development of a Corporate Level Production Planning System for the Semi Conductor Industry, Optimization in Industry, Chichester, UK, 1986.
[40]
Y.-F. Hung and R. C. Leachman, “Reduced simulation models of wafer fabrication facilities,” International Journal of Production Research, vol. 37, no. 12, pp. 2685–2701, 1999.
[41]
S. Hsieh, J.-S. Hwang, and H.-C. Chou, “A petri-net-based structure for AS/RS operation modelling,” International Journal of Production Research, vol. 36, no. 12, pp. 3323–3346, 1998.
[42]
G. Cybenko, “Approximation by superpositions of a sigmoidal function,” Mathematics of Control, Signals, and Systems, vol. 2, no. 4, pp. 303–314, 1989.
[43]
K.-I. Funahashi, “On the approximate realization of continuous mappings by neural networks,” Neural Networks, vol. 2, no. 3, pp. 183–192, 1989.
[44]
R. Reed, “Pruning algorithms—a survey,” IEEE Transactions on Neural Networks, vol. 4, no. 5, pp. 740–747, 1993.
[45]
C. Jutten and O. Fambon, “Pruning methods: a review,” in Proceedings of the 3rd European Symposium on Artificial Neural Networks (ESANN '95), pp. 129–140, Brussels, Belgium, April 1995.
[46]
Y. LeCun, J. S. Denker, and S. A. Solla, “Optimal brain damage,” Advances in Neural Information Processing Systems, vol. 2, pp. 598–605, 1990.
[47]
M. Norgaard, System identification and control with neural networks [Ph.D. dissertation], Institute of Automation, Technical University of Denmark, Copenhagen, Denmark, 1996.
[48]
C.-S. Leung, K.-W. Wong, P.-F. Sum, and L.-W. Chan, “A pruning method for the recursive least squared algorithm,” Neural Networks, vol. 14, no. 2, pp. 147–174, 2001.
[49]
H.-S. Tang, S. Xue, and T. Sato, “ filtering in neural network training and pruning with application to system identification,” Journal of Computing in Civil Engineering, vol. 21, no. 1, pp. 47–58, 2007.
[50]
M. E. Ricotti and E. Zio, “Neural network approach to sensitivity and uncertainty analysis,” Reliability Engineering and System Safety, vol. 64, no. 1, pp. 59–71, 1999.
[51]
D. Sabo and X.-H. Yu, “A new pruning algorithm for neural network dimension analysis,” in Proceedings of the International Joint Conference on Neural Networks (IJCNN '08), pp. 3313–3318, Hong Kong, China, June 2008.
[52]
H. Chandrasekaran, H.-H. Chen, and M. T. Manry, “Pruning of basis functions in nonlinear approximators,” Neurocomputing, vol. 34, pp. 29–53, 2000.
[53]
A. Saltelli, K. S. Chan, and E. M. Scott, Sensitivity Analysis, John Wiley & Sons, New York, NY, USA, 2000.
[54]
F. Gruau, “A learning and pruning algorithm for genetic boolean neural networks,” in Proceedings of the 1st European Symposium on Artificial Neural Networks (ESANN '93), pp. 57–63, Brussels, Belgium, April 1993.
[55]
D. W. Ruck, S. K. Rogers, and M. Kabrisky, “Feature selection using a multilayer perceptron,” Neural Network Computing, vol. 2, no. 2, pp. 40–48, 1990.
[56]
G. Tarr, Multilayered feedforward networks for image segmentation [Ph.D. dissertation], Air Force Institute of Technology, Wright-Patterson AFB, Ohio, USA, 1991.
[57]
L. Ljung, System Identification: Theory for the Users, Prentice-Hall, Englewood Cliffs, NJ, USA, 1987.
[58]
P. Thomas and G. Bloch, “Initialization of one hidden layer feed-forward neural networks for non-linear system identification,” in Proceedings of the 15th IMACS World Congress on Scientific Computation, Modelling and Applied Mathematics (WC '97), pp. 295–300, Berlin, Germany, August 1997.
[59]
H. El Haouzi, Approche méthodologique pour l'intégration des systèmes contr？lés par le produit dans un environnement de juste-à-temps: application à l'entreprise TRANE [Ph.D. dissertation], Université Henri Poincaré Nancy 1, Nancy, France, 2008.
[60]
T. Klein, Le kanban actif pour assurer l'intéropérabilité décisionnelle centralisé/distribué: application à un industriel de l'ameublement [Ph.D. dissertation], Université Henri Poincaré Nancy 1, Nancy, France, 2008.
