Do Irish politicians simplify speech when given access to a wider audience?
A case study on Irish TD speech patterns after the introduction of cameras into the Dáil
Authors: Jonathan Chew, Samantha Mincher, and Ryan Westwood
Date: 18 July 2022
Video recording of parliamentary proceedings has drastically changed the way in which political practice is conducted, and some have questioned whether cameras incentivize politicians to engage in performative antics rather than passing meaningful legislation. Those who oppose televising the political process believe politicians will alter their speech to appeal to a wider audience – essentially turning the legislature into a stage for electioneering. Using a dataset of transcribed Irish Dáil Éireann plenary sessions, we used quantitative text analysis to investigate whether the 2011 implementation of broadcast cameras in the Dáil led to an overall increase in the complexity of TD parliamentary speech. Our results show that the introduction of broadcasting technologies did not produce more than a marginally negative effect on TD speech complexity.
In the last 50 years, there has been a rise in the role of televised media in the political arena and this broadcasting of parliamentary debate has affected how politicians behave. Most liberal democracies broadcast their legislative debates as a means to increase transparency between elected officials and the citizens they represent. Yet those who oppose cameras in the legislative process believe recording elected officials hinders the policy-making process, causing a distraction and allowing politicians to continue to campaign rather than meaningfully debate issues that advance policy agendas. The “personalization of politics” theory establishes that politics have begun to favor individual leaders over parties and institutions, and cameras may contribute to this imbalance.
Ireland introduced cameras into their legislature fairly recently in comparison to other liberal democracies, having begun broadcasting in November 2011. With their legislative debates televised, members of the Irish Parliament (TDs, shortened from the Irish Teachtaí Dála) were able to increase their audience beyond that of their colleagues and to have their policy stances witnessed by the electorate. This, in turn, allowed them to take credit for – or to distance themselves from – legislative propositions.
The world of policy making can be saturated in jargon and typically involves specialists whose advanced speech would be difficult for the general public to understand. If a politician uses too much jargon-saturated or ‘complex’ speech, it could result in them alienating much of the public. Thus, our research project studies whether TDs reduced speech complexity after the introduction of televised legislative debates as a means to appeal to a wider group.
Our analysis focuses primarily on how Oireachtas TV broadcasting affected the Flesch-Kincaid (FK) complexity of speeches given by TDs in the Assembly of Ireland (Dáil Éireann). Designed in 1975 by the United States navy, the Flesch-Kincaid readability test uses the total number of syllables, words, and sentences in a text to assess its approximate grade level. It has been described as a better metric for understanding text characteristics than other traditional readability formulas. It has also been shown to correlate highly with Benoit et. al.’s model of political speech sophistication. In our study, we assigned a Flesch-Kincaid score to each entry in a data set of over 30,000 speeches given shortly before and after cameras were introduced in the Dáil.
Our initial analysis using Ordinary Least Squares (OLS) regression yielded a statistically significant (yet small) drop in readability, providing some support for our hypothesis that the introduction of cameras into the Dáil would decrease complexity scores. Yet after running and validating a topic model on our speech corpus using semisupervised Latent Dirichlet allocation (k = 15), then controlling for both party and topic, our regressions concluded no statistically significant relationship.
Figure 1: Average Daily Readability of TD Speeches by Party
We also break down the data set by party. As shown in the figure above, most parties do not experience a clear change in average daily readability after the date when cameras were introduced into the Dáil (represented by the red dashed vertical line). In particular, average readability for Fianna Fáil, Fine Gael, Labour, and Sinn Féin, the four parties with the most daily speeches in our dataset, remains relatively static throughout the six-month time period. This reflects the results from our regression using party identity as a control, which showed that while there were statistically significant differences in readability between parties, the presence or absence of cameras did not have any statistically significant effect.
Figure 2: Average Daily Readability of Speeches on Selected Topics
In our analysis which used topic modeling to categorize the content of different speeches, we found that cameras played a statistically significant role in altering readability (p<0.05), but that the effect was very small, a 0.30 drop in the expected Flesch Kincaid score for speeches given after November 15 compared to before. For purposes of clarity, the figure above only shows five of the fifteen topics in our model – specifically, the daily readability averages for all speeches on the given topics.
We additionally conducted sensitivity tests as a means to examine the relationship between cameras and the complexity of speech on an immediate-term (7-day period), a short-term (14-day period), and a long-term (8-month period) level. In the immediate 7-day sensitivity test, we found a statistically significant relationship, with a positive 3.73 FK point effect of cameras on speech complexity, but in the 14-day short-term observation, we did not find any significant effect of cameras on speech complexity and a slightly negative 0.18-FK point effect in the long term. Our hypothesis that technical topics would have the most positive effects receives mixed feedback. On the one hand, “budgeting”, which may seem relatively more technical given the framing that it is the simultaneous consideration of all other topics, appears to have the most significantly negative effect. On the other hand, “general vocabulary” has the expected relatively negative effect of about one grade level, and expectedly technical topics like “legal issues” and” “general/government procedure” have the most positive effects with complexity increasing by more than four FK points.
We offer a few potential explanations as to why we did not observe a significant change in the complexity of speech after the introduction of cameras into the Dáil. Firstly, we believe the role of increased media consumption due to the over-saturation of political content made available through social media and 24 hours news can not be underplayed. By 2011, politicians were well used to their images and campaign speeches being broadcasted, photographed, and quoted on cable news networks, on social media, and the internet in general. With politicians being accustomed to the over-saturation of their image they would be less likely to change how they participate in the Dail solely because of the introduction of cameras. Secondly, politicians may not have recognized any benefit to appeal to a wider audience by reducing their complexity of speech during debates as Irish law prohibits parties from using Dàil footage for campaign purposes. Our research focuses primarily on complexity of speech scores as a means to assess change post the introduction of cameras, but further research could go as far to see if any changes in political communication affected the amount of legislation that was put forth and made law.
Although our analysis concluded with a null result, we believe the lack of an observable change in the Flesch Kincaid complexity score offers researchers and the general public a rosier view of politics and of democracy. There is a widely held view that although TDs are tasked with the responsibility of governing in the interest of their constituents, they can be swept up in the desire to be reelected and will play to the camera rather than engage in meaningful policymaking. Our results refute those who believe that broadcasting parliamentary debates negatively affects the democratic process by showing that Irish TDs typically do not simplify their speech complexity even when the cameras are turned on.
About the authors: Jonathan Chew, Samantha Mincher, and Ryan Westwood are current MSc Politics and Data Science students at University College Dublin. The blog post builds on a collaborative research paper submitted for the module ”Connected_Politics”.