STOCK PREDICTION – AI PROJECTS

Stock price forecasting is a popular and important topic in financial and academic studies. Share Market is an untidy place for predicting since there are no significant rules to estimate or predict the price of share in stock market. Many methods like technical analysis, fundamental analysis and statistical analysis etc. are all used to attempt to predict the stock price in the share market but none of these methods are proved as a consistently acceptable prediction/forecasting tool.

In this project, we attempt to implement Time Series Analysis approach to forecast stock market prices. It aims at forecasting stock market price by using previous recorded stock prices. It proposes the Moving Average method for the prediction of stock market closing price. A popular and widely used statistical method for time series forecasting is the SARIMA model. SARIMA model is used so as to identify unknown and hidden patterns in share market data. This project is for Nepalese users as the forecasting is done on the listed companies of Nepal Stock Exchange Ltd. Our project is an attempt to train the Merolagani website‟s stock price dataset in our model which can efficiently help in the calculation of the closing index for a given trading day. The results have led to the main conclusion: SARIMA model with MSE as 8.87 and ARIMA model with MSE as 205.82 improves the efficiency of the stock forecast when compared to forecasts obtained by ARIMA model. The accuracy of our model is 99.83%. This result contributes to the development of more robust forecasting and enables the users to make the smart decision regarding the stock prices.

BACKGROUND

Stock Market forecasting and analysis is the act of trying to determine the future value of a company stock or other financial instrument traded on an exchange. A stock market is a public market for the trading of company stock and derivatives at an agreed price. Stock market is the important part of economy of the country and plays a vital role in the growth of the country. Both investors and industry are involved in stock market and wants to know whether some stock will rise or fall over a period of time. It is based on the concept of demand and supply. If the demand for a company‟s stock is higher, then the company share price increases and if the demand for company‟s stock is less then, the company share price decreases. Indeed, forecasting the trend of stock market rise or fall incident need to be focused because the obtained result can be utilized for customers in decision making finalizing whether to buy or sell the particular shares of a given stock.

The Nepal Stock Exchange Limited (NEPSE) is the only stock exchange of Nepal. It is established under the company act, operating under securities exchange act, 1983. It is located in Singha Durbar Plaza, Kathmandu Nepal. Also merolagani.com provides Nepal stock exchange quotes, News, analysis and provides portfolio management tools. The basic objective of NEPSE is to import free marketability to the government and corporate securities by facilitating transactions in its trading floor through member, market intermediates such as broker, market makers etc.

Due to involvement of many number of industries and companies, merolagani.com contain very large sets of data from which it is difficult to extract information and analyze their trend of work manually. Stock market analysis and prediction will reveal the market patterns and predict the time to purchase stock. The successful prediction of a stock’s future price could yield significant profit. This is done using large historic market data to represent varying conditions and confirming that the time series patterns have statistically significant predictive power for high probability of profitable trades and high profitable returns for the competitive business investment.

SCOPE

• Saving of time

Many people have problem analyzing the stock prices and making smart decisions regarding it. Also, recommendation from the broker might not be much effective and more time consuming. The system helps in saving lots of time.

• Relief from processor problem

Many mobile phones and limited processing power computers can‟t handle forecasting system due to its extremely large dataset. The solution opted for this can be use of web services. The proposed system uses web services, thus makes process simple.

• Work on several numbers of data

The number of choices for the stock prices on internet is very high and it is tedious to analyze the data by self while searching. The scope of this proposed system includes working with numerous data, with ease.

LITERATURE REVIEW

Many research groups are exploring stock market trend prediction using web data analytics. As said “Man seeks for better solutions”, we came to the idea of developing this project after we researched many Stock Market Predictions papers available on the Internet. Many research were conducted by using different approach like Fundamental and Macro-economical Analysis Approach, different Machine Learning Approach and many more.

The paper [1] describes a multi-agent system that uses numerical, financial and economical data in order to evaluate the company‟s position on the market, profitability, performance, future expectations in the company‟s evolution. Determining the effect of political, governmental and social decisions along with detecting the way in which the price is constructed based on technical and fundamental analysis methods and the bid/ask situation helps in determining a more precise buy/sell signals, reducing the false signals and determining some risk/gain positions on different periods of time. In order to validate the results a prototype was developed in this paper.

In the paper [2], they proposed a stock price predication model which is combinational feature from technical analysis and sentiment analysis (SA). The features of sentiment analysis are based on Point wise mutual information (PMI) which is a term expansion method from multidimensional seed word. The features of technical analysis based on expert rule from trading information. Experimental results show that the use of sentiment analysis and technical analysis achieves higher performance than that without sentiment analysis in predicting stock price.

The effectiveness [3] of long short term memory networks trained by back propagation through time for stock price prediction is explored in this paper. Arrange of different architecture LSTM networks are constructed trained and tested. LSTMs had been conventionally proven successful for time series prediction. HengjianJia found that LSTMs learn patterns effective for stock market prediction and he obtained decent RMSEs with different architectures of LSTM. This study helped us realize this problem as a time-series problem, and gave an insight to solve this problem with a sliding window approach.

Deep Neural Networks, being the most exceptional innovation in Machine Learning, have been utilized to develop a short-term prediction model. The paper discusses about two distinct sorts of Artificial Neural Networks, Feed Forward Neural Networks and Recurrent Neural Networks.

The applications of Deep Learning in different financial domains were explained by J. B. Heaton and his colleagues [5]. Their study discussed a few prediction problems in the financial domain. It also stated a few advantages deep learning predictors have over traditional predictors. A few of them being, over fitting can be easily avoided and correlation in input data can also be handled easily.

OVERVIEW OF THE SYSTEM

The system consists of mainly three main steps. Firstly, data is collected and filtered for relevancy from various sources. Secondly, analysis is carried out on the collected data by examining the current market direction, tracking the industry group and specific companies after which the data is represented and scored accordingly. At last, a time series analysis is designed and SARIMA model along with the model diagnostic yielding best accuracy is used for better visualization of stock market.

• Data Sources

This project requires historic data of stock market as the project also emphasizes on data mining techniques. So, it is necessary to have a trusted source having relevant and necessary data required for the prediction. We are using Merolagani website (http://www.merolagani.com/) as the primary source of data. The site is updated on daily basis and it is also a repository for years of stock market data for Nepal. We have performed web scraping to get all the required data from this website using Scrappy tool to scrape the data.

• Time Series Analysis

A set of regular time-ordered observations of a quantitative characteristic of an individual or collective phenomenon taken at successive periods/ points of time is known as a time series method. Although many other soft computing methods have been developed recently but moving average method is still considered as the best method by many people due to its easiness, objectiveness, reliability, and usefulness. Moving average method comes in various forms, but their underlying purpose remains the same, that is to track the trend determination of the given time series data

• AutoRegressive Integrated Moving Average (ARIMA)

One of the most common methods used in time series forecasting is known as the ARIMA model, which stands for AutoRegressive Integrated Moving Average. ARIMA is a model that can be fitted to time series data in order to better understand or predict future points in the series. Differencing, autoregressive, and moving average components make up a non-seasonal ARIMA model which can be written as a linear equation:

Yt=c+ϕ1ydt−1+ϕpydt−p+…+θ1et−1+θqet−q+et………………………………………………………….

(1)

Where yd is Y differenced d times and c is a constant.

There are three distinct integers (p, d, and q) that are used to parameterize ARIMA models. Together these three parameters account for seasonality, trend, and noise in datasets. The process of fitting an ARIMA model is sometimes referred to as the Box-Jenkins method.

1. p is the auto-regressive part of the model. It allows us to incorporate the effect of past values into our model using the Partial autocorrelation graph.
2. d is the integrated part of the model. This includes terms in the model that incorporate the amount of differencing (i.e. the number of past time points to subtract from the current value) to apply to the time series which may be 0, 1 or 2.
3. q is the moving average part of the model. This allows us to set the error of our model as a linear combination of the error values observed at previous time points in the past using autocorrelation graph.

• Seasonal AutoRegressive Integrated Moving Average (SARIMA)

The ARIMA model above assumes non-seasonal series, which needs to be de-seasonalized. In this case, the model is specified by two sets of order parameters: (p, d, q) as described above and (P,D, Q)m(P,D,Q)m parameters describing the seasonal components of m period.

• Model Diagnostics

We have also performed the model diagnostics which suggests that the model residuals are normally distributed based on the following:

• In the top right plot, we see that the red KDE line follows closely with the N (0, 1) line (where N (0, 1)) is the standard notation for a normal distribution with mean 0 and standard deviation of 1). This is a good indication that the residuals are normally distributed.
• The qq-plot on the bottom left shows that the ordered distribution of residuals (blue dots) follows the linear trend of the samples taken from a standard normal distribution with N (0, 1). Again, this is a strong indication that the residuals are normally distributed.
• The residuals over time (top left plot) don’t display any obvious seasonality and appear to be white noise. This is confirmed by the autocorrelation (i.e. correlogram) plot on the bottom right, which shows that the time series residuals have low correlation with lagged versions of it.

We need information about stock price of the respective organization such as closing price at their respective dates. The task of system is to fit the thus obtained data into SARIMA model and forecast the stock price rise or fall in graphical manner. For the purpose of fitting the (p, d, q) parameter of SARIMA model, we have used the ARIMA model. The p parameter allows to include past values (obtained from PACF graph).The d parameter allows the amount of differencing (values may be 0, 1, or 2) and the parameter q allows to set the error of our model (obtained from ACF) .The obtained results will be shown in the graph using the matplotlib library. The graph plots the stock actual closing price along with the forecasted price.

RESULT AND ANALYSIS

Evaluating the Stock market prediction has at all times been tough work for analysts. Thus, we attempt to make use of vast written data to forecast the stock market indices. We have implemented the application of time series analysis using SARIMA and ARIMA model and their salient feature. Our initial analysis show significant ACF and PACF between different input parameters.

In this project, the factors that are taken into account for change in the closing price of a particular company are: Date, Closing price, Opening price, High, Low, Volume, RSI. We performed analysis on obtained data to establish relation between our output parameters and the selected factors (Date and Closing price)

The fig shows the data of Agriculture Development Bank Limited and Nabil Bank. A total of 3587 data were used for ADBL and 4513 data were used for Nabil Bank for calculation using only their Closing price and Date.

The result obtained in both the cases was fairly accurate as from fig taken. The prediction is fairly accurate unless there is huge and sudden variation in actual data. On other hand, this also proves the hypothesis that stock market is actually unpredictable. After the phase of forecasting, the result will be displayed to the users in the form of web pages which will benefit the financial analysts, investors to take trading decision by observing market behavior.

We obtained the MSE (Mean Square Error) using the SARIMA model as 8.87 whereas the MSE using the ARIMA model is 205.82.

CONCLUSION

Financial analysts, investors can use this prediction model to take trading decision by observing market behavior. The system we have designed is quite simple and works on time series analysis. While completing the project various problems were tackled and resolved to make the system more flexible. The project has been a great learning experience for us. Importance of team work is well understood through the project. This project did provide us the opportunity to learn new language Django, Python, also the practical approach of software engineering. Thus, we can say that the project was the great opportunity for us to test all the knowledge we have gained over the years studying engineering.

REFERENCES

1. M. Tirea and V.Negru, “Intelligent Stock Market Analysis System- A Fundamantal and Macro-economical Analysis Approach”, 2014 16^th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing, Timisoara, 2014, pp.519-526
2. J. Wu, C. Su, L. Yu and P. Chang, ”Stock Price Predication using Combinational Features from Sentimental”, 2012
3. H. Jia, “Investigation Into The Effectiveness Of Long Short Term Memory Networks For Stock Price Prediction”, 2016.
4. K. Khare, O. Darekar, P. Gupta and V. Z. Attar, “Short term stock price prediction using deep learning,” 2017 2nd IEEE International Conference on Recent Trends in Electronics, Information & Communication Technology (RTEICT), Bangalore, 2017, pp. 482-486.
5. Heaton, J.B. and Polson, Nick and Witte, Jan Hendrik, “Deep Learning for Finance: Deep Portfolios”, 2016.
6. Wei Li, Jian Liao, “A Comparative Study on Trend Forecasting Approach for Stock Price Time Series”, 2017
7. Carson Kai-Sang Leung, Richard Kyle MacKinnon, Yang Wang, “A Machine Learning Approach for Stock Price Prediction”, 2014
8. Seng Hansun, “A New Approach of Moving Average Method in Time Series Analysis”, 2013 Rodolfon C. Cavalcante1, Adriano L. I. Oliveira, “An Autonomous Trader Agent Ensemble”, 2014
9. P. Khanal, S. R. Shakya, “Analysis and Prediction of Stock Prices of Nepal usingdifferent Machine Learning Algorithms”, 2016
10. Manuel R. Vargas, Beatriz S. L. P. de Lima and Alexandre G. Evsukoff, “Deep learning for stock market prediction from financial news articles”, 2017
11. Yuh-Jen Chen, Ph.D., “Enhancement of Stock Market Forecasting Using a Technical Analysis-based Approach”, 2014
12. F. B. Oriani and Guilherme P. Coelho, ”Evaluating the Impact of Technical Indicators on Stock Forecasting”, 2016
13. Seksan Sangsawad, Chun Che Fung, “Extracting Significant Features Based on Candlestick Patterns Using Unsupervised Approach”, 2017

VISUALIZATION :