Timezone: »

 
Poster
Learning low-dimensional state embeddings and metastable clusters from time series data
Yifan Sun · Yaqi Duan · Hao Gong · Mengdi Wang

Tue Dec 10 10:45 AM -- 12:45 PM (PST) @ East Exhibition Hall B + C #69
in Algorithms -- Representation Learning »

This paper studies how to find compact state embeddings from high-dimensional Markov state trajectories, where the transition kernel has a small intrinsic rank. In the spirit of diffusion map, we propose an efficient method for learning a low-dimensional state embedding and capturing the process's dynamics. This idea also leads to a kernel reshaping method for more accurate nonparametric estimation of the transition function. State embedding can be used to cluster states into metastable sets, thereby identifying the slow dynamics. Sharp statistical error bounds and misclassification rate are proved. Experiment on a simulated dynamical system shows that the state clustering method indeed reveals metastable structures. We also experiment with time series generated by layers of a Deep-Q-Network when playing an Atari game. The embedding method identifies game states to be similar if they share similar future events, even though their raw data are far different.

Keywords: Time Series Analysis · Representation Learning · Applications · Algorithms
[ Paper]  [ Poster] 

Author Information

Yifan Sun (Carnegie Mellon University)
Yaqi Duan (Princeton University)
Hao Gong (Princeton University)
Mengdi Wang (Princeton University)

Mengdi Wang is interested in data-driven stochastic optimization and applications in machine and reinforcement learning. She received her PhD in Electrical Engineering and Computer Science from Massachusetts Institute of Technology in 2013. At MIT, Mengdi was affiliated with the Laboratory for Information and Decision Systems and was advised by Dimitri P. Bertsekas. Mengdi became an assistant professor at Princeton in 2014. She received the Young Researcher Prize in Continuous Optimization of the Mathematical Optimization Society in 2016 (awarded once every three years).

More from the Same Authors