Int. J. Simul. Multisci. Des. Optim.
Volume 5, 2014
|Number of page(s)||13|
|Published online||04 February 2014|
Interpretability and variability of metamodel validation statistics in engineering system design optimization: a practical study
Electronics and Communications Engineering Department, College of Engineering at Al-Lith, Umm Al-Qura University, Makkah Al-Mukarramah, KSA
2 Biomedical Engineering Department, Hijjawi College of Engineering Technology, Yarmouk University, Irbid, Jordan
3 Biomedical Engineering Department, College of Engineering, Jordan University of Science and Technology, Irbid, Jordan
* e-mail: email@example.com
Accepted: 5 November 2013
Prediction accuracy of a metamodel of an engineering system in comparison to the simulation model it approximates is one fundamental criterion that is used in metamodel validation. Many statistics are used to quantify prediction accuracy of metamodels in deterministic simulations. The most frequently used ones include the root-mean-square error (RMSE) and the R-square metric derived from it, and to a lesser degree the average absolute error (AAE) and its derivates such as the relative average absolute error (RAAE). In this paper, we compare two aspects of these statistics: interpretability of results returned by these statistics and their sample-to-sample variations, putting more emphasis on the latter. We use the difference-mode to common-mode ratio (DMCMR) as a measure of sample-to-sample variations for these statistics. Preliminary results are obtained and discussed via a number of analytic and electronic engineering examples.
Key words: Simulation / Modeling / Metamodel validation
© H. Hamad et al., Published by EDP Sciences, 2014
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Current usage metrics show cumulative count of Article Views (full-text article views including HTML views, PDF and ePub downloads, according to the available data) and Abstracts Views on Vision4Press platform.
Data correspond to usage on the plateform after 2015. The current usage metrics is available 48-96 hours after online publication and is updated daily on week days.
Initial download of the metrics may take a while.