A scalable sampling method to high-dimensional uncertainties for optimal and reinforcement learning-based controls

Xie, JunfeiWan, YanMills, KevinFilliben, JamesLewis, Frank2022-03-032022-03-032017-05-26Xie, J., Wan, Y., Mills, K., Filliben, J.J. and Lewis, F.L., 2017. A scalable sampling method to high-dimensional uncertainties for optimal and reinforcement learning-based controls. IEEE control systems letters, 1(1), pp.98-103.Xie, J., Wan, Y., Mills, K., Filliben, J.J. and Lewis, F.L., 2017. A scalable sampling method to high-dimensional uncertainties for optimal and reinforcement learning-based controls. IEEE control systems letters, 1(1), pp.98-103.https://hdl.handle.net/1969.6/90232Modern dynamical systems often operate in environments of high-dimensional uncertainties that modulate system dynamics in a complicated fashion. These high-dimensional uncertainties, non-Gaussian in many realistic scenarios, complicate real-time system analysis, design, and control tasks. In this letter, we address the scalability of computation for systems of high-dimensional uncertainties by introducing new sampling methods, the multivariate probabilistic collocation method (M-PCM), and its extension called M-PCM-orthogonal fractional factorial design (OFFD) which integrates M-PCM with the OFFDs to break the curse of dimensionality. We explore the capabilities of M-PCM and M-PCM-OFFD-based optimal control and adaptive control using the reinforcement learning approach. The analyses and simulation studies illustrate the efficiency and effectiveness of these two approaches.Modern dynamical systems often operate in environments of high-dimensional uncertainties that modulate system dynamics in a complicated fashion. These high-dimensional uncertainties, non-Gaussian in many realistic scenarios, complicate real-time system analysis, design, and control tasks. In this letter, we address the scalability of computation for systems of high-dimensional uncertainties by introducing new sampling methods, the multivariate probabilistic collocation method (M-PCM), and its extension called M-PCM-orthogonal fractional factorial design (OFFD) which integrates M-PCM with the OFFDs to break the curse of dimensionality. We explore the capabilities of M-PCM and M-PCM-OFFD-based optimal control and adaptive control using the reinforcement learning approach. The analyses and simulation studies illustrate the efficiency and effectiveness of these two approaches.en-USuncertaintyoptimal controlsystem dynamicsaerospace electronicscomputational modelingsampling methodsscalabilityuncertaintyoptimal controlsystem dynamicsaerospace electronicscomputational modelingsampling methodsscalabilityA scalable sampling method to high-dimensional uncertainties for optimal and reinforcement learning-based controlsA scalable sampling method to high-dimensional uncertainties for optimal and reinforcement learning-based controlsArticlehttps://doi.org/10.1109/LCSYS.2017.2708598