Ofcom’s Behavioural Insights Hub ran its first in-house online trial this year, and we’re looking to do more.
We found that upgrading a typical broadband contract summary with behavioural insights led to greater comprehension of key contract information, as well as making it quicker to read. Our online randomised control trial worked effectively and marks the start of how we plan to use this agile methodology right across our regulatory regimes.
The purpose: building our own expertise
We’ve recently carried out several online trials in collaboration with external agencies, focusing on a variety of online safety features. These collaborative trials have generated valuable insights, but they can take significant time to set-up, design and run. We recognised that by developing our own in-house capabilities we could complement our work with external partners with a more agile approach, while extending our experience of running trials to Ofcom’s other regulated sectors. This led us to embark on a new journey, one where we take a more hands-on approach, starting with the telecoms sector.
We identified broadband contract summaries as the ideal starting point.
Contract summaries are designed to distil complex, lengthy, and often quite technical agreements into a more digestible format. A consistent template also aims to help consumers to more easily understand key terms, compare different offers and make informed decisions. However, we wondered: could we improve these summaries?
The trial: upgrading a contract summary
To test this, we created two versions of a contract summary; one mimicked a typical summary that we see in the real world, while the enhanced summary was boosted using behavioural insights.
The enhanced summary was designed to be clearer, more concise, and easier to navigate.
 |
 |
 |
| Clear summary box | Purposeful headings | Helpful bullet points |
We conducted an online trial with 1,200 participants, randomly allocating them to review either the typical or enhanced summary. Our aim was to improve consumers’ understanding of key contract information – measured by a set of 10 simple questions.
The results: three key findings
3. And it was quicker to read.
Not only did the enhanced summary boost comprehension, but it also allowed participants to process the information more quickly. The typical summary took 85 seconds to read on average, compared to 67 seconds for the enhanced summary – a 21% reduction.
Interestingly, both summaries were rated similarly in terms of readability. Although the improvements were effective, this suggests that participants didn’t consciously notice the differences between the two summaries.
As with all online trials, there are limitations to our study – some of which we hope to improve upon as we run more in-house trials. For example, the experimental set-up may have led to higher levels of attention and engagement than we might expect in a real-world setting, where distractions are more common. Although we are confident that the enhanced summary would also be more effective in the real world, the actual levels of comprehension from either design could be lower.
Due to our experimental design, we’re also unable to pinpoint exactly which behavioural elements were responsible for the improved comprehension. Existing research would suggest that each of the changes would have contributed positively, but it’s likely that a combination of changes is needed to see substantial changes.
Looking ahead: future applications
This trial is just the beginning of what we hope will be a growing programme of in-house trials. The success of this experiment has shown that online trials and behavioural insights can generate policy insights across our regulated sectors, in addition to our programme of online trials in online safety.
We plan to make more use of our in-house capability to test new initiatives, evaluate the effectiveness of existing policies, and refine our approaches across all of our regulated sectors. We look forward to sharing our progress as we explore new ways to enhance our work and deliver on our commitment to evidence-based policymaking.
Notes
This blog was written by Alex Jenkins based on research by Alex Jenkins, Ceren Cibik and Jonathan Porter. For more detail about the methodology and analysis, please contact behavioural.insights@ofcom.org.uk.
Disclaimer: The analyses, opinions and findings in this blog should not be interpreted as an official position of Ofcom. Ofcom’s Behavioural Insight blogs are written as points of interest and are the personal views of the author(s). They are not intended to be an official statement of Ofcom's policy or thinking.