UC Santa Cruz

The general problem addressed in this work concerns the analysis of a function of multiple real variables in which the output is itself a function of a real variable as well as categorical information. We are interested in the prediction of the output and the analysis of the boundaries separating regions during classification. Efficient and accurate prediction of the output curves is important for safety analysis and validation of a complex system, like an adaptive flight control system. An understanding of the boundaries between regions, where the aircraft can maintain stable flight or will break up is essential for safety certification. As a motivating application we are using the NASA IFCS (Intelligent Flight Control System) flight simulator.

For output prediction, we developed a new statistical method for emulation of computer models with multiple outputs. An emulator is a computationally efficient statistical model that is used to approximate a computationally expensive simulation by treating the simulator as a black box and learning a mapping from inputs to outputs. Our approach for emulating the curves is to represent them in an orthogonal basis (e.g., PCA, Fourier, or Wavelet), which captures curve characteristics, and then to predict the coefficients. We allow for the possibility of output curves whose length varies with input, which may occur when a simulator fails to run to completion for some inputs and the failures occur at different output time steps. To the best of our knowledge, the variable-length output problem has not yet been addressed. We have developed a hierarchical model, which first uses classification into a few groups and then fitting distinct models for different classes of output curves.

For the analysis of boundaries, we developed a new sequential approach based upon design of computer experiments. A dictionary of suitable linear or non-linear parameterized boundary shapes, which capture underlying physical and design knowledge can be provided by the domain expert. We incorporate this knowledge into our modeling and determine the most likely shapes and its parameters. Since each iteration requires a costly run of the system simulator, we developed a candidate selection mechanism, which is specifically tailored toward boundary detection and which can take priors into account in order to reduce the number of required simulation runs. We present results of experiments with artificial and IFCS simulation data sets.