Given observations in the form of a (large dimensional) vector time series, how do we efficiently explore the underlying functional relationships across different dimensions and time lags?

Classical regression analysis such as classification and regression trees (CART) and multivariate adaptive regression splines (MARS) cannot be easily adapted to fast online implementation with provable guarantees. In this study, we propose a methodology for adaptive nonlinear sequential modeling of vector-time series data.

Data is modeled as a nonlinear function of past values corrupted by noise, and the underlying nonlinear function is assumed to be approximately expandable in a spline basis. We cast the modeling of data as finding a good fit representation in the linear span of multi-dimensional spline basis, and use a variant of l1-penalty regularization in order to reduce the dimensionality of representation. Using adaptive filtering and expectation–maximization techniques, we design a fast online algorithm with complexity near linear in time. The algorithm also automatically tunes the underlying parameters based on the minimization of the regularized sequential prediction error. We demonstrate the generality and flexibility of the proposed approach on both synthetic and real-world datasets. Moreover, we analytically investigate the performance of our algorithm by obtaining both bounds of the prediction errors, and consistency results for variable selection.