Stock Price Prediction With Big Data and Machine Learning

Abstraction: Spark MLLib decision-tree classifier for limit order book price movement prediction

Key points:

• Implements ideas from "Modeling high-frequency limit order book dynamics with support vector machines"; uses Decision Tree (multiclass) instead of SVM via Spark MLLib in Scala
• Feature vectors divided into three categories: basic (prices/volumes at up to 10 bid/ask levels), time-insensitive (spread, mid-price, price ranges), and time-sensitive (historical windows over raw order log)
• Labels each point with MeanPriceMove: Stationary, Up, or Down over a 1-second horizon
• Training data: NYSE TAQ OpenBook EQY_US_NYSE_BOOK_20130403; validation: next day's file; 36 features total
• Decision Tree achieves ~70% classification accuracy on ORCL symbol; caution noted — high stationary accuracy may mask poor movement prediction; 2 trading days is insufficient for production conclusions

Connections: Apache Spark · Nyse · Decision Tree · Feature Engineering · Financial Prediction

Source: http://eugenezhulenev.com/blog/2014/11/14/stock-price-prediction-with-big-data-and-machine-learning/