Beyond Words: An Applied Study of Sentiment Analysis on Scandinavian Earnings Call Transcripts Comparing a Traditional Lexicon, Machine Learning, FinBERT, and GPT-4 Turbo

Abstract

The efficacy of four sentiment analysis methods in predicting stock price movements based on earnings call transcripts from Scandinavian companies is explored in this thesis. The four methods include the traditional "bag-of-words" using the financial lexicon by Loughran and McDonald (2011), a supervised machine learning lexicon trained through stock price movements, and two large language models, FinBERT and GPT-4 Turbo. Using a standardised ternary classification system, we perform horse race regressions to compare the predictive accuracy of these methods. Our findings suggest that GPT-4 Turbo demonstrates superior predictive power, outperforming all other methods in both individual and joint regression analyses. It achieves an impressive adjusted R2 of 5.2%, significant economic magnitude at the 1% level, with a 2.7% (-1.9%) increase (decrease) in stock prices associated with positive (negative) sentiment classification. While the supervised machine learning lexicons show no advantage over the Loughran and McDonald (2011) lexicon with an adjusted R2 of 3.3%, the latter exhibits impressive performance of 4.2%, likely due to its construction from similar textual data. On the other hand, FinBERT shows mixed results, suggesting potential biases introduced by the segmentation of longer texts.