Author information:
Zalán Kocsis https://orcid.org/0000-0002-7844-7094: Magyar Nemzeti Bank, Senior Economic Analyst. E-mail: kocsisz@mnb.hu
Mónika Mátrai-Pitz https://orcid.org/0009-0008-2576-4193: Magyar Nemzeti Bank, Lead Data Scientist. E-mail: pitzm@mnb.hu
Abstract:
Our study compares the characteristics of text analysis methods on communications text samples of the US Federal Reserve and four Central and Eastern European central banks. Based on our results, methods based on BERT-type models are the most accurate at capturing the monetary policy, real economic and inflation information contained in central bank texts, outperforming OpenAI GPT-4.1 and GPT-5 models. BERT-type methods are faster than GPT models and can be run offline without a subscription, but their disadvantages are that the models require separate training, which entails hardware and labour costs, and that modifying the method is cumbersome. Conversely, GPT models are more flexible and have proven to be more accurate on new central bank samples. The dictionary-based methods used as benchmarks are significantly less accurate, but their use may be justified in certain cases due to their speed, cost-free operation and the transparency of the method.
Cite as (APA):
Kocsis, Z., & Mátrai-Pitz, M. (2025). Which Text Method to Choose for Analysing Central Bank Communication? A Comparison of Artificial Intelligence and Previous Techniques. Financial and Economic Review, 24(4), 34–64. https://doi.org/10.33893/FER.24.4.34
Column:
Study
Journal of Economic Literature (JEL) codes:
C55, C63, E52, E58
Keywords:
central bank communications, artificial intelligence, text analysis
References:
Apel, M. – Blix Grimaldi, M. (2012): The Information Content of Central Bank Minutes. Working Paper No. 261, Sveriges Riksbank. https://doi.org/10.2139/ssrn.2092575
Araci, D. (2019): FinBERT: Financial Sentiment Analysis with Pre-Trained Language Models. arXiv preprint arXiv:1908.10063.https://doi.org/10.48550/arXiv.1908.10063
Bihari, P. (2015): Odüsszeuszi utazás – az előretekintő iránymutatás tapasztalatai (An Odyssean journey – experience with forward guidance). Közgazdasági Szemle (Economic Review), 62(4): 749–766. https://www.kszemle.hu/tartalom/cikk.php?id=1568
Bihari, P. – Sztanó, G. (2015): Sikertörténet vagy sok hűhó semmiért? (A success story or much ado about nothing?). Köz-Gazdaság, 10(2): 23–39. https://unipub.lib.uni-corvinus.hu/2033/
Blinder, A.S. – Ehrmann, M. – Fratzscher, M. – De Haan, J. – Jansen, D.J. (2008): Central Bank Communication and Monetary Policy: A Survey of Theory and Evidence. Journal of Economic Literature, 46(4): 910–945. https://doi.org/10.1257/jel.46.4.910
Brown, T.B. – Mann, B. – Ryder, N. – Subbiah, M. – Kaplan, J. – Dhariwal, P. et al. (2020): Language Models are Few-Shot Learners. Advances in Neural Information Processing Systems, 33: 1877–1901. https://doi.org/10.48550/arXiv.2005.14165
Byrne, D. – Goodhead, R. – McMahon, M. – Parle, C. (2023): The Central Bank Crystal Ball: Temporal Information in Monetary Policy Communication. Research Technical Paper Vol. 2023 No. 1, Central Bank of Ireland. https://ideas.repec.org/p/cbi/wpaper/1-rt-23.html
Correa, R. – Garud, K. – Londono, J.M. – Mislang, N. (2017): Sentiment in Central Banks’ Financial Stability Reports. International Finance Discussion Paper No. 1203. https://doi.org/10.17016/IFDP.2017.1203
Csortos, O. – Lehmann, K. – Szalai, Z. (2014): Theoretical considerations and practical experiences of forward guidance. MNB Bulletin, 2014 July: 45–55. https://www.mnb.hu/letoltes/csortos-lehmann-szalai.pdf
Devlin, J. – Chang, M.W. – Lee, K. – Toutanova, K. (2019): BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In: Burstein, J. – Doran, C. – Solorio, T. (eds.): Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1, June, pp. 4171–4186. https://doi.org/10.18653/v1/N19-1423
Dovern, J. – Fritsche, U. – Slacalek, J. (2012): Disagreement among Forecasters in G7 Countries. Review of Economics and Statistics, 94(4): 1081–1096. https://doi.org/10.1162/REST_a_00207
Ehrmann, M. – Eijffinger, S. – Fratzscher, M. (2010): The role of central bank transparency for guiding private sector forecasts, ECB Working Paper No. 1146. https://dx.doi.org/10.2139/ssrn.1532312
Eusepi, S. – Preston, B. (2010): Central Bank Communication and Expectations Stabilization. American Economic Journal: Macroeconomics, 2(3): 235–271. https://doi.org/10.1257/mac.2.3.235
Evdokimova, T. – Nagy-Mohacsi, P. – Ponomarenko, O. – Ribakova, E. (2023): Central Banks and Policy Communication: How Emerging Markets have Outperformed the Fed and ECB. Peterson Institute for International Economics Working Paper No. 23–10. https://doi.org/10.2139/ssrn.4628672
Fanta, N. – Horváth, R. (2024): Artificial intelligence and central bank communication: the case of the ECB. Applied Economics Letters, 32(18): 2600–2607. https://doi.org/10.1080/13504851.2024.2337318
Fulop, A. – Kocsis, Z. (2023): News indices on country fundamentals. Journal of Banking & Finance, 154, 106951. https://doi.org/10.1016/j.jbankfin.2023.106951
Gambacorta, L. – Kwon, B. – Park, T. – Patelli, P. – Zhu, S. (2024): CB-LMs: language models for central banking. BIS Working Paper No 1215. https://bis.org/publ/work1215.htm
Gábriel, P. – Pintér, K. (2006): The effect of the MNB’s communication on financial markets. MNB Working Papers 2006/9, Magyar Nemzeti Bank. https://www.mnb.hu/letoltes/wp2006-9.pdf
Geiger, F. – Kanelis, D. – Lieberknecht, P. – Sola, D. (2025): Monetary-Intelligent Language Agent (MILA). Deutsche Bundesbank Technical Paper No. 01/2025. https://hdl.handle.net/10419/316448
Gentzkow, M. – Kelly, B. – Taddy, M. (2019): Text as Data. Journal of Economic Literature, 57(3): 535–574. https://doi.org/10.1257/jel.20181020
Gorodnichenko, Y. – Pham, T. – Talavera, O. (2023): The Voice of Monetary Policy. American Economic Review, 113(2): 548–584. https://doi.org/10.1257/aer.20220129
Gössi, S. – Chen, Z. – Kim, W. – Bermeitinger, B. – Handschuh, S. (2023): FinBERT-FOMC: Fine-Tuned FinBERT Model with Sentiment Focus Method for Enhancing Sentiment Analysis of FOMC Minutes. ICAIF’23: Proceedings of the Fourth ACM International Conference on AI in Finance, pp. 357–364. https://doi.org/10.1145/3604237.3626843
Grandini, M. – Bagli, E. – Visani, G. (2020): Metrics for Multi-Class Classification: An Overview. arXiv preprint arXiv:2008.05756. https://doi.org/10.48550/arXiv.2008.05756
Gürkaynak, R.S. – Sack, B. – Swanson, E. (2005): The Sensitivity of Long-Term Interest Rates to Economic News: Evidence and Implications for Macroeconomic Models. American Economic Review, 95(1): 425–436. https://doi.org/10.1257/0002828053828446
Hansen, A.L. – Kazinnik, S. (2024): Can ChatGPT Decipher Fedspeak?. SSRN, April 11. https://doi.org/10.2139/ssrn.4399406
Hansen, S. – McMahon, M. (2016): Shocking language: Understanding the macroeconomic effects of central bank communication. Journal of International Economics, 99(S1): S114–S133. https://doi.org/10.1016/j.jinteco.2015.12.008
Hansen, S. – McMahon, M. – Prat, A. (2018): Transparency and Deliberation Within the FOMC: A Computational Linguistics Approach. The Quarterly Journal of Economics, 133(2): 801–870. https://doi.org/10.1093/qje/qjx045
Horváth, D. – Kálmán, P. – Kocsis, Z. – Ligeti, I. (2014): What factors influence the yield curve?. MNB Bulletin, 2014 March: 28–39. https://www.mnb.hu/letoltes/horvath-kalman-kocsis-ligeti.pdf
Huang, A.H. – Wang, H. – Yang, Y. (2022): FinBERT: A Large Language Model for Extracting Information from Financial Text. Contemporary Accounting Research, 40(2): 806–841. https://doi.org/10.1111/1911-3846.12832
Kim, W. – Spörer, J. – Lee, C.L. – Handschuh, S. (2024): Is Small Really Beautiful for Central Bank Communication? Evaluating Language Models for Finance: Llama-3-70B, GPT-4, FinBERT-FOMC, FinBERT, and VADER. In: Proceedings of the 5th ACM International Conference on AI in Finance, pp. 626–633. https://doi.org/10.1145/3677052.3698675
Lewis, P. – Perez, E. – Piktus, A. – Petroni, F. – Karpukhin, V. – Goyal, N. et al. (2020): Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. Advances in Neural Information Processing Systems, 33: 9459–9474. https://proceedings.neurips.cc/paper/2020/file/6b493230205f780e1bc26945df7481e5-Paper.pdf
Liu, Y. – Ott, M. – Goyal, N. – Du, J. – Joshi, M. – Chen, D. et al. (2019): RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv preprint arXiv:1907.11692. https://doi.org/10.48550/arXiv.1907.11692
Liu, A. – Feng, B. – Xue, B. – Wang, B. – Wu, B. – Lu, C. et al. (2024): DeepSeek-V3 Technical Report. arXiv:2412.19437. https://doi.org/10.48550/arXiv.2412.19437
Mátrai-Pitz, M. – Siket, B. (2023): Reálgazdasági és inflációs tonalitásindexek (Real economic and inflation tonality indices). Python-kód és dokumentáció. Mimeo.
Mikolov, T. – Chen, K. – Corrado, G. – Dean, J. (2013): Efficient Estimation of Word Representations in Vector Space. arXiv preprint arXiv:1301.3781. https://doi.org/10.48550/arXiv.1301.3781
Nagy Mohácsi, P. – Evdokimova, T. – Ponomarenko, O. – Ribakova, E. (2024): Emerging Market Central Banking and Communication: The Great Catchup. Financial and Economic Review, 23(1): 29–49. https://doi.org/10.33893/FER.23.1.29
Naszódi, A. – Csávás, C. – Erhart, S. – Felcser, D. (2016): Which Aspects of Central Bank Transparency Matter? A Comprehensive Analysis of the Effect of Transparency of Survey Forecasts. International Journal of Central Banking, 46(4): 147–192. https://www.ijcb.org/journal/ijcb16q4a4.pdf
Niţoi, M. – Pochea, M.M. – Radu, Ş.C. (2023): Unveiling the sentiment behind central bank narratives: A novel deep learning index. Journal of Behavioral and Experimental Finance, 38, 100809. https://doi.org/10.1016/j.jbef.2023.100809
Pennington, J. – Socher, R. – Manning, C.D. (2014): Glove: Global Vectors for Word Representation. In: Moschitti, A. – Pang, B. – Daelemans, W. (eds.): Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), October, pp. 1532–1543. https://doi.org/10.3115/v1/D14-1162
Peskoff, D. – Visokay, A. – Schulhoff, S. – Wachspress, B. – Blinder, A. – Stewart, B.M. (2024): GPT Deciphering Fedspeak: Quantifying Dissent Among Hawks and Doves. arXiv preprint arXiv:2407.19110. https://doi.org/10.48550/arXiv.2407.19110
Pfeifer, M. – Marohl, V.P (2023): CentralBankRoBERTa: A Fine-Tuned Large Language Model for Central Bank Communications. Journal of Finance and Data Science, 9, 100114. https://doi.org/10.1016/j.jfds.2023.100114
Picault, M. – Renault, T. (2017): Words are not all created equal: A new measure of ECB communication. Journal of International Money and Finance, 79: 136–156. https://doi.org/10.1016/j.jimonfin.2017.09.005
Poole, W. – Rasche, R.H. – Thornton, D.L. (2002): Market Anticipations of Monetary Policy Actions. Review-Federal Reserve Bank of Saint Louis, 84(4): 65–94. http://doi.org/10.20955/r.84.65-94
Romer, C.D. – Romer, D.H. (2000): Federal Reserve Information and the Behavior of Interest Rates. American Economic Review, 90(3): 429–457. http://doi.org/10.1257/aer.90.3.429
Seelajaroen, R. – Budsaratragoon, P. – Jitmaneeroj, B. (2020): Do monetary policy transparency and central bank communication reduce interest rate disagreement?. Journal of Forecasting, 39(3): 368–393. http://doi.org/10.1002/for.2631
Takahashi, K. – Yamamoto, K. – Kuchiba, A. – Koyama, T. (2022): Confidence interval for micro-averaged F1 and macro-averaged F1 scores. Applied Intelligence, 52(5): 4961–4972. https://doi.org/10.1007/s10489-021-02635-5
Thorsrud, L.A. (2018): Words are the New Numbers: A Newsy Coincident Index of the Business Cycle. Journal of Business & Economic Statistics, 38(2): 393–409. https://doi.org/10.1080/07350015.2018.1506344
Tobback, E. – Nardelli, S. – Martens, D. (2017): Between hawks and doves: measuring central bank communication. ECB Working Paper No. 2085. https://doi.org/10.2139/ssrn.2997481
Touvron, H. – Lavril, T. – Izacard, G. – Martinet, X. – Lachaux, M-A. – Lacroix, T. et al. (2023): Llama: Open and Efficient Foundation Language Models. arXiv preprint arXiv:2302.13971. https://doi.org/10.48550/arXiv.2302.13971
Vaswani, A. – Shazeer, N. – Parmar, N. – Uszkoreit, J. – Jones, L. – Gomez, A.N. et al. (2017): Attention Is All You Need. Advances in Neural Information Processing Systems, 30. https://doi.org/10.48550/arXiv.1706.03762
Wei, J. – Wang, X. – Schuurmans, D. – Bosma, M. – Ichter, B. – Xia, F. et al. (2022): Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. Advances in Neural Information Processing Systems, 35, 24824–24837. https://doi.org/10.48550/arXiv.2201.11903
Yao, R. – Chai, Q. – Yao, J. – Li, S. – Chen, J. – Zhang, Q. et al. (2025): Interpreting Fedspeak with Confidence: A LLM-Based Uncertainty-Aware Framework Guided by Monetary Policy Transmission Paths. arXiv preprint arXiv:2508.08001. https://doi.org/10.48550/arXiv.2508.08001
Yao, S. – Zhao, J. – Yu, D. – Du, N. – Shafran, I. – Narasimhan, K. et al. (2023): ReAct: Synergizing Reasoning and Acting in Language Models. In: International Conference on Learning Representations (ICLR), January. https://doi.org/10.48550/arXiv.2210.03629