Caroline is a Senior User Experience Designer by day and an inventor, movie buff, hockey player & cat fan by night.

Earlier this year The Royal Edinburgh Military Tattoo appointed Signal, and our sister agency Leith, to create a new smartphone app that would complement their world famous event.

We built a beta app in record time so it was ready for the Tattoo’s preview night on 3rd August - the perfect opportunity to test how it worked and make sure everything was in place.

Testing the beta app ‘in the wild’ was very important to help us refine the app’s UX and was also an important part of the QA process.

220,000 people experience the live show in person every August, under the dramatic backdrop of Edinburgh Castle - with a further 100 million watching the coverage worldwide on TV. No pressure then!

The full story is below - you can also watch the behind-the-scenes video by Signal's Art Director Matt Higgins for a flavour of how testing unfolded on opening night.

Tattoo App Testing

To make sure the app provides real value for attendees in the long-term, we proposed a realistic and sustainable development plan. It began with building a beta app with some foundational features and putting it in the hands of users during the 2017 event.

This would allow us to assess how well it worked for them in real-time, and focus on those elements that proved most useful and engaging. It would also give the company a firm foundation to build on for years to come.

Creating a fully featured app without testing it at an early stage would have been a big risk for The Royal Edinburgh Military Tattoo. It could result in them releasing an app with expensive, flashy features that few attendees actually use.

For example, Augmented Reality functionality was unlikely to be a good fit for the Tattoo’s audience who are predominantly 55+. We decided that the most important thing was an app that was genuinely useful before, during and after the event, but doesn't distract from the live performance.

The app’s purpose before the event was twofold - to help build excitement for the show, and to provide useful information to help attendees know what to expect on the night.

The beta app was available for anyone to download on the Android and iTunes app stores, with useful content available for all users - including information about the Tattoo’s history, a preview of the 2017 programme and information about the performers.

However some features could only be unlocked by attendees entering their ticket information - including a map of key locations (queue starts, ticket office, stadium utilities) and a seating plan. A schedule of key timings and additional information about security, weather conditions and wheelchair accessibility also enabled attendees to plan their evening in advance.

On preview night, we met with our test group of 20 people, which was evenly split between Android and iOS users (we even persuaded Game of Thrones actor Ian McEllhinney to test it for us, who was in town with his own show for the Edinburgh Fringe).

We then observed our test group’s pre-show behaviour so we could see which of these features worked best and what kind of questions they might have at this stage.

Enhancing the live show was one of our key objectives, so we designed the app to update as the performance changes. Live programme cards were triggered by a member of the Tattoo events team in the control booth as events unfolded onstage. These cards included interesting facts about what was happening at that moment and was unique content not featured in the print programme.

Our test users were seated around the stadium, to give us a full understanding the environment. For example, we discovered that the unique setting resulted in some uneven wi-fi coverage in the arena, and many attendees are tourists who may not have mobile data. As a result we reduced the app’s reliance on high download speeds.

During the preview performance we were able to see this content come to life and judge how it complemented the live show. We then refined the content and live card order/layout to provide the best possible experience throughout the rest of August, and we saw increased engagement as a result.

Finally, we observed how our test group used the app when the event ended. We wanted to enable attendees to continue to get value from the app after the event ended, including revisiting and reflecting on moments from the show and sharing their experience with others.

We also identified potential future features such as app notifications that are timed for the end of the performance, to encourage people about to take a look at merchandise, and sign up for next year’s event.

The research and insight gathering we did during the 2017 Tattoo has helped us validate the app concept, understand the operational environment and identify features and content that will offer genuine value to the user, when Version 1 of the app goes live in time for 2018’s Tattoo. We also identified future possibilities such as digital ticketing and personalised accessibility features such as audio translation.

Video and header image: Matt Higgins, Art Director 

App testing photos: John Harfield, Senior Designer

Taking the Edinburgh Tattoo’s smartphone app for a test run

Testing The Royal Edinburgh Military Tattoo's beta app 'in the wild' during the 2017 live event.

Caroline Smith

Senior UX Designer

Taking the Edinburgh Tattoo’s smartphone app for a test run

Cheltenham

Edinburgh

London