Research
Policymaker sentiment signals a slower ECB cutting cycle
Policymakers of the European Central Bank (ECB) expressed increased confidence as they closed in on the first rate cut. However, an analysis of the language they have used since this cut indicates that this confidence does not extend to future ECB meetings, and that subsequent rate cuts are unlikely to follow in quick succession.
Summary
No more forward guidance
For years, the ECB’s policy statements provided clear guidance about the central bank’s future plans. This allowed policymakers to steer expectations and market rates at a time when policy rates were near the lower bound. But this forward guidance strategy was abandoned when inflation caught the ECB off guard and forced policymakers to embark on a rushed hiking cycle.
Although formal guidance has largely been discontinued, we find that the speeches and interviews by individual rate setters still provide some information about the ECB’s upcoming moves. Members of the Governing Council regularly express their opinion about the outlook for the economy, inflation, and monetary policy. Incorporating the sentiments of the individual policymakers into a model of macroeconomic variables indicates that their words provide additional insight to forecasts of upcoming policy changes.
Figure 1 illustrates the feature importance of various factors over time, including macroeconomic variables such as the Taylor rule and the sentiments expressed by policymakers. Our analysis finds that the (relative) importance of the different variables in predicting upcoming policy decisions has shifted over time. The relatively high predictive power of previous press statements around 2014-2018 coincides with the peak era of forward guidance. However, the importance of press statements has declined since 2015. Over this period, the sentiments expressed by different groups of policymakers has become more influential.[1]
Although a model based on the words of policymakers may not necessarily be the strongest predictor of upcoming changes in monetary policy, we find that it may provide a supplemental analysis and a cross-check.
[1] The total feature importance in the 2018-2022 period is relatively low. This may indicate that other factors have become more important, or that historical relationships between the explanatory variables and monetary policy have changed, e.g., due to the impact of the pandemic on the economy.
Figure 1: The ECB’s communication contains information for predicting policy changes
A three-stage approach to turn speeches into a prediction
To transform the speeches and interviews of ECB policymakers into a probability of a rate hike or cut, we implemented a three-stage modelling approach. The first step is topic modelling, the second stage is involves sentiment analysis, and finally we run a classification model. Additionally, we use Shapley Additive Explanations (SHAP) to analyze and explain the predictions of the model.
Topic modelling
We use Latent Dirichlet Allocation (LDA) to infer the topics from a given text. LDA is a generative probabilistic model that uncovers hidden (“latent”) topics and themes from a collection of documents. It models each document as being generated by a process of repeatedly sampling words and topics from statistical distributions. By applying mathematical algorithms, LDA is able to recover the most likely distributions that were used in this generative process. These distributions tell us something about which topics exist and how they are distributed among each document.
Our analysis successfully identified six topics with relevant keywords: economic outlook, financial stability, inflation outlook, monetary development, monetary policy, and a category for other topics. This helps us track what concerns the policymakers over time.
Sentiment analysis
To measure the sentiment of a text, historically a dictionary approach is often used. This approach compares the text against a list of words or phrases that are labelled as indicating a positive or negative sentiment. This is known to cause potential issues in finance, where “liability” isn’t necessarily a negative phrase, for example. Likewise, a static dictionary of words doesn’t reliably capture the sentiment of policymakers, who often express themselves carefully.
We therefore employ a novel approach that involves using a Generative AI model to analyze and quantify the tone and concerns about the inflation outlook.
ChatGPT-4 was utilized to analyze the sentiment in policymakers' speeches. The sentiment score is derived using the following rules:
To improve accuracy, we employed a few-shot learning technique. We used various evaluation methods to ensure the model’s performance, such as checking consistency and benchmarking the model output.
The sentiment scores are then grouped into three categories, matching the historical bias of the policymakers: doves, neutrals, and hawks. For each group we compute a (weighted) average sentiment index based on the topic distribution from the LDA model, where we attach higher weights to the texts that have a heavier emphasis on the topics that are relevant to the implementation of monetary policy.
Classification model
A tree-boosting machine learning technique, LightGBM, was applied to predict the chance of a rate cut, using a set of macroeconomic variables (e.g., GDP, PMI, HICP) and policymaker sentiments.
To train and estimate our model’s error, we used time series cross-validation with a sliding window of seven years. During each fold, the LightGBM model is trained and validated on the upcoming policy change. Since our time series start from 2000, with a sliding window of seven years, this meant the cross-validation began with the first fold, where the training data ranges from 2000-2006 and the validation data is the first meeting in 2007.
To estimate the model’s error, we calculated the confusion matrix to assess the precision, recall, and F1 score for each residual for each fold.
Shapley Additive Explanations
Shapley Additive Explanations (SHAP) is a common method used to explain the predictions of complex machine learning models. SHAP first determines a baseline, which represents the model’s average prediction if it had no specific input features. Then it starts adding features one by one to see how the prediction changes. By averaging the SHAP values across all predictions, we can estimate the importance of each feature.Not all policymakers are seen as equally important
Central bankers are human and, like all of us, they have their own biases when it comes to their view of the economic outlook and risks. Historically, policymakers that are more risk-averse with respect to high inflation are considered hawks, and officials that are more concerned about the risk of low growth or even a recession are considered dovish.
Furthermore, not all policymakers are seen as equally important. Naturally, the President and Vice-President of the ECB have a lot of influence. So do the other four members of the Executive Board, especially the Chief Economist and the person responsible for market operations. Through their contributions to policy discussions, these positions are seen as particularly influential, which is also reflected in their public appearances. These four Executive Board members tend to focus on the economic outlook and monetary policy, whereas the remaining two members of the Executive Board are more likely to emphasize topics that are not immediately related to the ECB’s monetary policy decisions, such as the development of a digital euro, or climate change (see figure 2).
Figure 2: What’s on policymakers’ minds during public appearances?
The remaining members of the Governing Council are the heads of the national central banks of the countries that make up the euro area. The perceived influence of these policymakers often correlates to the size of the country they represent, though some particularly vocal governors of smaller central banks can punch above their weight.
When tracking policymakers’ opinions, analysts historically take note of the person’s bias and influence in order to gauge the overall balance of opinions in the Governing Council (see figure 3). However, there are some potential flaws in this analysis. For example, the more vocal members do not necessarily represent the majority in the Governing Council. Especially when views are divided, the minority may actually become more vocal to try to steer the opinion.
Figure 3: The perceived bias and influence of the current members of the Governing Council
By contrast, our analysis indicates that the opinions of the neutral Council members generally hold the most information about the future direction of monetary policy. Intuitively, this makes sense. Even though some policymakers may have a bit more traction over the agenda of a policy meeting, when the decision comes to a vote, each member of the Governing Council carries the same weight. Hawks and doves tend to be more or less entrenched in their opinions. So in a split vote, the median voter is likely to be a neutral policymaker. In other words, especially when the outcome of an ECB meeting is a close call, neutral policymakers will cast the deciding vote.
Figure 4 illustrates this. Here, we plot the impact of macroeconomic variables and the sentiment of policymakers on the model prediction for the upcoming rate decision. The horizontal axis represents the importance of the variable. This can change over time, as reflected by the areas of each variable. The higher up in the list, the greater the (absolute) impact that specific variable has on the predicted outcome of policy meetings.
Figure 4: The sentiment of policymakers without a clear bias tends to be the most valuable
Intertemporal confusion points to an uncertain outlook
[2] The time lags are expressed as the number of meetings between the model’s prediction and the actual rate change.
Figure 5: Predictive power is strongest in the near term
Figure 6: Confusion matrix
Interestingly, when central bankers do provide insight into their views beyond the next meeting, our sentiment model tends to focus on this longer-term view. This makes sense to some extent. Arguably, it better reflects a Governing Council member’s overall sentiment than their comments about the near term.
However, when a policymaker gives somewhat mixed messages about the near-term and the longer-term outlook, it may lead to intertemporal confusion. In these situations, the model may incorrectly incorporate the longer-term sentiment into the probability of a policy change in the near term. For example, Vujcic recently said that “the ECB will likely loosen monetary policy,” but he added that rates will “stay in restrictive enough territory to make sure that inflation is brought down to the 2% target.” The first part of this statement was a clear hint at a rate cut, but the second part kept the sentiment score closer to the hawkish side of the balance.
The confusion matrix of our classification model confirms that this sometimes leads to incorrect predictions. This confusion matrix compares the times that the ECB actually changed rates with the model’s predictions. Our model occasionally predicted the odds of a change in policy rates to be low when this in fact did happen, missing 16 out of the 88 times that the ECB hiked or cut interest rates. Likewise, there were 14 instances when the model forecast the ECB to change rates when this did not materialize (see figure 6).
As a result, our model predicted a lower probability of a rate cut at the June meeting than we would have expected (69%).
In the week since that ECB meeting, many policymakers have already spoken to the media. We re-ran the analysis and, based on their interviews and speeches so far, our model sees a likelihood of 51% that the ECB will cut rates again in July. We believe this to be on the high side. A manual check of the documents suggests that there may be two reasons for this relatively high score. First, policymakers have talked to the media to justify the cut that they have just implemented. Secondly, most policymakers are keeping the door open for future adjustments. These factors may have added to the model’s prediction of a rate cut.
That said, the sharp decline in the estimated probability between the previous meeting and the upcoming one is worth noting. We believe this underscores that the ECB will take a cautious approach to this cutting cycle. Presumably, the likelihood will continue to decline as the time since the previous policy decision progresses and policymakers increasingly focus on the July meeting in their interviews.
Figure 7: Ex-ante probability of a June rate cut
Figure 8: Modelled probability of a rate cut in July
Note: This is based on speeches and interviews between 8 and 15 July.