Stock Price Movement Classification Using Ensembled Model of Long Short-Term Memory (LSTM) and Random Forest (RF)

Albertus Gunawan - Bina Nusantara University, Jakarta, Indonesia
Antoni Wibowo - Bina Nusantara University, Jakarta, Indonesia


Citation Format:



DOI: http://dx.doi.org/10.62527/joiv.7.4.1640

Abstract


Stock investing is known worldwide as a passive income available for everyone. To increase the profit possibly gained, many researchers and investors brainstorm to gain a strategy with the most profit. Machine learning and deep learning are two of these approaches to predicting the stock's movement and deciding the strategy to gain as much as possible. To reach this goal, the researcher experiments with Random Forest (RF) and Long Short-Term Memory (LSTM) by trying them individually and merging them into an ensembled model. The researcher used RF to classify the results from LSTM models obtained throughout the Hyperparameter Optimization (HPO) process. This idea is implemented to lessen the time needed to train and optimize each LSTM model inside the ensembled model. Another anticipation done in this research to overcome the time needed to train the model is classifying the return for longer periods. The dataset used in this model is 45 stocks listed in LQ45 as of August 2021 This research results in showing that LSTM gives better results than RF model especially when using Bayesian Optimization as the HPO method, and that the ensembled model can return better precision in predicting stocks in comparison to the LSTM model itself. Future improvement can focus on the model structure, additional model types as the ensemble model estimator, improvement on the model efficiency, and datasets research to be used in predicting the stock movement prediction

Keywords


Machine Learning; Deep Learning; Long Short-Term Memory; Random Forest; Ensembled Model; Bayesian Optimization; Random Search; Stock Investing; Classification

Full Text:

PDF

References


Z. Jiang, "Forecast of Stock Price," 2022, doi: 10.2991/978-94-6463-052-7_169.

D. Kumar, P. K. Sarangi, and R. Verma, "A systematic review of stock market prediction using machine learning and statistical techniques," in Materials Today: Proceedings, Elsevier Ltd, 2020, pp. 3187–3191. doi: 10.1016/j.matpr.2020.11.399.

R. Gupta and M. Chen, "Sentiment Analysis for Stock Price Prediction," in Proceedings - 3rd International Conference on Multimedia Information Processing and Retrieval, MIPR 2020, Institute of Electrical and Electronics Engineers Inc., Aug. 2020, pp. 213–218. doi: 10.1109/MIPR49039.2020.00051.

R. Rosillo, D. de la Fuente, and J. A. L. Brugos, "Technical analysis and the Spanish stock exchange: testing the RSI, MACD, momentum and stochastic rules using Spanish market companies," Appl Econ, vol. 45, no. 12, pp. 1541–1550, Apr. 2013, doi: 10.1080/00036846.2011.631894.

Z. Tan, Z. Yan, and G. Zhu, "Stock selection with random forest: An exploitation of excess return in the Chinese stock market," Heliyon, vol. 5, no. 8, Aug. 2019, doi: 10.1016/j.heliyon.2019.e02310.

H. Qian, "Stock Predicting based on LSTM and ARIMA", doi: 10.2991/978-94-6463-036-7_72.

P. Ghosh, A. Neufeld, and J. K. Sahoo, "Forecasting directional movements of stock prices for intraday trading using LSTM and random forests," Financ Res Lett, 2021, doi: 10.1016/j.frl.2021.102280.

M. C. Lee, "Research on the Feasibility of Applying GRU and Attention Mechanism Combined with Technical Indicators in Stock Trading Strategies," Applied Sciences (Switzerland), vol. 12, no. 3, Feb. 2022, doi: 10.3390/app12031007.

S. Krishna Lakshminarayanan and J. McCrae, "A Comparative Study of SVM and LSTM Deep Learning Algorithms for Stock Market Prediction," 2019.

H. Maqsood et al., "A local and global event sentiment based efficient stock exchange forecasting using deep learning," Int J Inf Manage, vol. 50, pp. 432–451, Feb. 2020, doi: 10.1016/j.ijinfomgt.2019.07.011.

N. Jing, Z. Wu, and H. Wang, "A hybrid model integrating deep learning with investor sentiment analysis for stock price prediction," Expert Syst Appl, vol. 178, Sep. 2021, doi: 10.1016/j.eswa.2021.115019.

N. Naik and B. R. Mohan, "Stock price movements classification using machine and deep learning techniques-the case study of indian stock market," in Communications in Computer and Information Science, Springer Verlag, 2019, pp. 445–452. doi: 10.1007/978-3-030-20257-6_38.

L. Shi, Z. Teng, L. Wang, Y. Zhang, and A. Binder, "DeepClue: Visual interpretation of text-based deep stock prediction," IEEE Trans Knowl Data Eng, vol. 31, no. 6, pp. 1094–1108, Jun. 2019, doi: 10.1109/TKDE.2018.2854193.

Y. Song, J. W. Lee, and J. Lee, "A study on novel filtering and relationship between input-features and target-vectors in a deep learning model for stock price prediction," Applied Intelligence, vol. 49, no. 3, pp. 897–911, Mar. 2019, doi: 10.1007/s10489-018-1308-x.

E. Koo and G. Kim, "A Hybrid Prediction Model Integrating GARCH Models With a Distribution Manipulation Strategy Based on LSTM Networks for Stock Market Volatility," IEEE Access, vol. 10, pp. 34743–34754, 2022, doi: 10.1109/ACCESS.2022.3163723.

Y. Ma, R. Han, and W. Wang, "Portfolio optimization with return prediction using deep learning and machine learning," Expert Syst Appl, vol. 165, Mar. 2021, doi: 10.1016/j.eswa.2020.113973.

I. R. Parray, S. S. Khurana, M. Kumar, and A. A. Altalbe, "Time series data analysis of stock price movement using machine learning techniques," Soft comput, vol. 24, no. 21, pp. 16509–16517, Nov. 2020, doi: 10.1007/s00500-020-04957-x.

P. Patil, C. S. M. Wu, K. Potika, and M. Orang, "Stock market prediction using ensemble of graph theory, machine learning and deep learning models," in ACM International Conference Proceeding Series, Association for Computing Machinery, Jan. 2020, pp. 85–92. doi: 10.1145/3378936.3378972.

M. Nikou, G. Mansourfar, and J. Bagherzadeh, "Stock price prediction using DEEP learning algorithm and its comparison with machine learning algorithms," Intelligent Systems in Accounting, Finance and Management, vol. 26, no. 4, pp. 164–174, Oct. 2019, doi: 10.1002/isaf.1459.

M. Nabipour, P. Nayyeri, H. Jabani, A. Mosavi, E. Salwana, and S. Shahab, "Deep learning for stock market prediction," Entropy, vol. 22, no. 8, Aug. 2020, doi: 10.3390/E22080840.

D. Zhang and S. Lou, "The application research of neural network and BP algorithm in stock price pattern classification and prediction," Future Generation Computer Systems, vol. 115, pp. 872–879, Feb. 2021, doi: 10.1016/j.future.2020.10.009.

V. D. Ta, C. M. Liu, and D. A. Tadesse, "Portfolio optimization-based stock prediction using long-short term memory network in quantitative trading," Applied Sciences (Switzerland), vol. 10, no. 2, Jan. 2020, doi: 10.3390/app10020437.

I. K. Nti, A. F. Adekoya, and B. A. Weyori, "A comprehensive evaluation of ensemble learning for stock-market prediction," J Big Data, vol. 7, no. 1, Dec. 2020, doi: 10.1186/s40537-020-00299-5.

H. Shah, N. Tairan, H. Garg, and R. Ghazali, "A Quick Gbest Guided Artificial Bee Colony algorithm for stock market prices prediction," Symmetry (Basel), vol. 10, no. 7, Jul. 2018, doi: 10.3390/sym10070292.

P. Lv, Q. Wu, J. Xu, and Y. Shu, "Stock Index Prediction Based on Time Series Decomposition and Hybrid Model," Entropy, vol. 24, no. 2, Feb. 2022, doi: 10.3390/e24020146.

M. Agrawal, P. K. Shukla, R. Nair, A. Nayyar, and M. Masud, "Stock prediction based on technical indicators using deep learning model," Computers, Materials and Continua, vol. 70, no. 1, pp. 287–304, 2021, doi: 10.32604/cmc.2022.014637.

K. Pawar, R. S. Jalem, and V. Tiwari, "Stock Market Price Prediction Using LSTM RNN," in Advances in Intelligent Systems and Computing, Springer Verlag, 2019, pp. 493–503. doi: 10.1007/978-981-13-2285-3_58.

M. Vijh, D. Chandola, V. A. Tikkiwal, and A. Kumar, "Stock Closing Price Prediction using Machine Learning Techniques," in Procedia Computer Science, Elsevier B.V., 2020, pp. 599–606. doi: 10.1016/j.procs.2020.03.326.

L. Yang and A. Shami, "On hyperparameter optimization of machine learning algorithms: Theory and practice," Neurocomputing, vol. 415, pp. 295–316, Nov. 2020, doi: 10.1016/j.neucom.2020.07.061.

S. Chen and L. Ge, "Exploring the attention mechanism in LSTM-based Hong Kong stock price movement prediction," Quant Finance, vol. 19, no. 9, pp. 1507–1515, 2019, doi: 10.1080/14697688.2019.1622287.

S. Basak, S. Kar, S. Saha, L. Khaidem, and S. R. Dey, "Predicting the direction of stock market prices using tree-based classifiers," North American Journal of Economics and Finance, vol. 47, pp. 552–567, Jan. 2019, doi: 10.1016/j.najef.2018.06.013.

G. Biau and E. Scornet, "A Random Forest Guided Tour," 2015. Accessed: Aug. 15, 2022. [Online]. Available: https://arxiv.org/abs/1511.05741

S.-C. Wang, "Artificial Neural Network," in Interdisciplinary Computing in Java Programming, Boston, MA: Springer US, 2003, pp. 81–100. doi: 10.1007/978-1-4615-0377-4_5.

Y. Hua, Z. Zhao, R. Li, X. Chen, Z. Liu, and H. Zhang, "Deep Learning with Long Short-Term Memory for Time Series Prediction," IEEE Communications Magazine, vol. 57, no. 6, pp. 114–119, Jun. 2019, doi: 10.1109/MCOM.2019.1800155.

A. Graves, M. Liwicki, S. Fernández, R. Bertolami, H. Bunke, and J. Schmidhuber, "A novel connectionist system for unconstrained handwriting recognition," IEEE Trans Pattern Anal Mach Intell, vol. 31, no. 5, pp. 855–868, 2009, doi: 10.1109/TPAMI.2008.137.

S. Chakraborty, J. Banik, D. Chatterjee, and S. Addhya, "Study of Dependency on number of LSTM units for Character based Text Generation models," 2020.

K. Greff, R. K. Srivastava, J. Koutnik, B. R. Steunebrink, and J. Schmidhuber, “LSTM: A Search Space Odyssey,” IEEE Trans Neural Netw Learn Syst, vol. 28, no. 10, pp. 2222–2232, Oct. 2017, doi: 10.1109/TNNLS.2016.2582924.

S. Badillo et al., "An Introduction to Machine Learning," Clin Pharmacol Ther, vol. 107, no. 4, pp. 871–885, Apr. 2020, doi: 10.1002/cpt.1796.

J. Wu, X. Y. Chen, H. Zhang, L. D. Xiong, H. Lei, and S. H. Deng, "Hyperparameter optimization for machine learning models based on Bayesian optimization," Journal of Electronic Science and Technology, vol. 17, no. 1, pp. 26–40, Mar. 2019, doi: 10.11989/JEST.1674-862X.80904120.

J. Bergstra, J. B. Ca, and Y. B. Ca, "Random Search for Hyper-Parameter Optimization Yoshua Bengio," 2012. [Online]. Available: http://scikit-learn.sourceforge.net.