Time Series Prediction Using Support Vector Machine: A Survey By Ma Yongning

Outline Introduction to time series prediction Classic techniques Introduction to SVM Variants of SVM Hybrid techniques Application Conclusion

Time Series Prediction The objective of time series prediction is to find a function f(x) such that, the predicted value of the time series at a future point in time is unbiased and consistent. Goal of time series prediction is to estimate some future value based on current and past data samples

ANN Approach – Autoregression Filter Linear, stationary process The current term of the series can be estimated by a linear weighted sum of previous terms in the series A number of techniques exist for computing AR coefficients. The main two categories are least squares and Burg method

ANN Approach – Kalman Filter Also known as linear quadratic estimation (LQE), is an algorithm which uses a series of measurements observed over time, containing noise (random variations) and other inaccuracies, and produces estimates of unknown variables Assume linear, stationary process, and model is known

Introduction to SVM A technology used to find the optimal separating hyperplane f(x)= wx + b or f(x)= wg(x) + b Linear SVM developed by Vapnik and Chervonenkis in 1960s Non-Linear SVM developed in 1990s by applying the kernel trick Unbiased if b = 0

Computation of Separating Hyperplane Assume f(x)= wx + b (Linear Case) Task: Maximize the margin M = 2/||w|| Which means to minimize ||w|| Also, we allow some error ξ

Some Math Our problem now is Subject to And use convex optimization to solve for f(x)

The Kernel Trick In the linear case we have K(x, y) =. If the data points are not linearly separable, we construct Such that Which means K(x, y) can be expressed in dot product of terms in inner product space V (e.g. Hilbert Space) whose dimension is dim(V), therefore separable in V.

An Simple Example Common kernel functions are polynomial or Gaussian To see how they transform input vectors into higher dimension, consider quadratic kernel Which corresponds to feature The dimension of the feature is O(n^2):

Non-Linear Decision Boundary

Least-Square SVM (Among other Variants) Express the error term as in linear regression: Then problem can be solved by letting partial derivatives of Lagrangian to be zero, which eliminates w and b, and leads to a linear system of a and b, that can be directly solved. LS-SVM sacrifices a bit accuracy for efficiency

Hybrid Approaches Fuzzy SVM (blurred decision boundaries) SOM and SVM (SOM used to cluster data first) GA and SVM (used to optimize the free variables in SVM; a broad type will be adaptive SVM -ASVM, where free variables are subject to change in the training process) Specific to time series: use ARIMA to predict linear region Smooth data first by SOM, Kalman filter. But in the case of SOM, it worsened the performance

Application of SVM for Time Series in Different Industries Financial Data Time Series Prediction  Use of SVR to predict five specific financial time series sources including the S&P 500 and several foreign bond indices.  The SVR significantly outperformed the BP NN because of it’s ability to appropriately fit the data.  A hybrid system of using SOM combined with SVR yielded not only better prediction performance but also superior convergence speed.

Application of SVM for Time Series in Different Industries General Business Appications  Electricity Price Forecasting  Credit Rating Analysis  Customer “Churning”- Auto Insurance Market Prediction  Financial Failure of Dotcoms – Financial Analysis using 24 financial ratios  Production Value Prediction of the Taiwanese Machinery Industry

Application of SVM for Time Series in Different Industries Environmental Parameter Estimation Using SVR  Used to predict parameters such as rainfall estimation and detection, weather forecasting, short term air quality(environmental pollution such as nitrogen oxides) etc…

Financial Data Time Series Prediction Electric Utility Load Forecasting Applications Using SVR.  Forecasting of electric power consumption demands by consumers, which is a non-linear prediction problem.  (Factors influencing usage for example are “holiday time periods”, weather(temperature and humidity), electricity pricing etc…

Financial Data Time Series Prediction Machine Reliability Forecasting Applications  The prediction of machine reliability is a non-linear problem.  Example is to predict the “period reliability ration” for the automotive industry based on time series data containing vehicle damage incidents and the number of damages repaired.  Another example is the use of SVR to predict engine failure in both the repair and design process of mechanical engines. Data used as input was the engine age at the time of unscheduled maintenance action per maintenance period and output was the predicted engine age of the next unscheduled maintenance action.

Financial Data Time Series Prediction Control System and Signal Processing Applications.  Mobile Position Tracking  Internet Flow Control  Adaptive Inverse Disturbance Cancelling  Narrowband Interference Suppression  Antenna Beamforming  Elevator Traffic Flow Prediction  Dynamically Tuned Gyroscope Drift Modeling

Financial Data Time Series Prediction Miscellaneous Applications  Biological Neuron Application(Australian Crayfish)  Kalman Filtering Method of Switching Dynamics Associated with Unsupervised Segmentation  Natural Gas Load Forecasting(using weather related factors such as temperature, day of week and holidays etc…)  Transportation Travel Time Estimation  Use of Particle Swarm Optimization in conjuncture with SVR.

Conclusion Support Vector Machines/Support Vector Regression(SVR) are powerful learning mechanisms that have been developed and matured over the last 15 years. Useful for predicting and forecasting time series for a myriad of applications. SVR research continues to be a viable approach in the prediction of time series data in non-linear systems.