Project Description

Overview

The Sun currently use customer surveys in their digital editorial platform designed to collect data and insights from their audience. Previous strategies were often leading to low engagement and the brief for the UXD team was to investigate how to help increase engagement in order for the business to collect a greater volume of customer and audience insights. The business had a longer term strategy to eventually use these insights to help drive more relevant content, personalised experiences and promotions in order to add further value within the platform.

Objectives

  • Deepen audience engagement with interactivity

  • Collect valuable survey data and audience insights

Strategy & Requirements

During initial discussions with stakeholders, an idea had emerged to potentially combine these customer survey questions with more engaging poll questions that could be contextual placed in articles and designed to match closely to specific editorial content. This polls could be used throughout the platform specifically in area of website that we knew would gain larger amounts of traffic. 

Our basic hypothesis was, if user were tempted to interact to a poll question that is relevant and contextual to what they are reading, they are more likely to further engage in a more commercially driven survey question if that was presented to them in that same moment.

This strategy would mean us building a new tool for the editorial team and journalists to be able to quickly create poll questions on the fly to be able to insert them into articles when published.

Love Island launch & design requirements

We knew from previous years that articles written around the Love Island topic drove particularly strong traffic on The Sun platform. Love Island 2019 was soon to be launching and The Sun were planning a lot of editorial and commercial activity. We felt this provided us with a good opportunity to launch and trial a proof of concept to test our hypothesis. Placing contextual Love Island themed polls within articles that, once a user has interacted, would then also present a commercial survey question to them.

We had 4 weeks to design and build the new tool in order launch the test during the Love Island campaign. The tool had to be flexible enough for the editorial team to quickly create polling question that contain 2, 3 or 4 answers which could also include images for the associated answer fields. This would then create a UI widget that would be placed into the editorial content by the editorial team before publishing the article. The front end widget needed to be designed to be responsive and display across mobile and desktop sites.  

The design phase

The new poll widget was designed to display inline within the article editorial content. Once a user interacts with the poll the latest poll results would transition in and be displayed for 3 seconds before the customer survey question is displayed. After interaction with the survey question a  message screen is displayed thanking users for taking part. We considered the opportunity to deliver further cross promotional messaging in the final “Thank You” screen, such as promotions for newsletter sign ups and links to article commenting features.

For the concept trial, we designed the tool to load the second survey question randomly from a pool of survey questions set up through the Content Management System (CMS). Individual user IDs would also be tracked in order that a single user would not see or have to answer the same survey question twice. Next steps would be to enable the business to have more management options available to control the display of survey question through a more comprehensive campaign builder in the CMS tool.

Targets & Expectations

We launched the test on 27th June 2019 and it ran over 7 weeks throughout the Love Island campaign. The editorial team typically launched 6-8 polls each weekday that were placed into topical Love Island articles. Each poll was editorially designed to relate specifically to the editorial content.

Based on previous data insights our initial expectations were that: 

  • 5% of traffic would interact with the main editorial poll

  • 10% of that traffic would go on to answer the second commercial survey question

  • Giving us a target of 0.5% of traffic answering both questions.

Results

Soon after launch we started to gather some results and we were thrilled with the numbers we were seeing. We collected more than 2,000 answers to commercial survey questions on the first day! During the 7 week test we recorded as many as 14,911 responses in a single day, off the back of just SIX editorial polls. The previous record delivered 5,385 customer survey data points in a week.

During the 7 week trial: 

  • More than 1.1million poll questions were answered

  • Actual interaction rate for main/editorial poll was avg 10.5%.

  • More than 150,000 survey questions were answered

  • Interaction rate for the second, commercial question was avg 17.4%

  • So avg 1.5% of users answered both questions

  • The test performed 3x better than expected

Overall evaluation of the trial

The proof of concept and 7 week trial proved to be a big success performing 3 times better than targeted with more than 150,000 individual customer survey questions answered over the 7 week campaign. The business were delighted with the success.

Our original hypothesis of combining contextual poll questions together with commercially driven survey questions and placing them into high traffic areas of the site had proved to be correct and this method of data capture was far more effective than the campaign building that had previous been used for survey questions.

Next Steps

Following on from the success of the Love Island trial, the business are now looking to roll the polling tool out across the entire newsroom and all section of the website, thus increasing the amount of polls and customer survey question displayed across the platform. Further polling campaigns are also planned for other big sporting and showbiz events where the The Sun will potentially see larger traffic and customer engagement.

Improvements are also planned for the Content Management System (CMS) in order to make it even easier for any journalist to create and insert a poll into their articles. The CMS will also include more comprehensive controls for admin users to create different groups of survey questions in order for the business to further increase the value of audience data now being collected.