How we brought user testing from 30,000 feet to the virtual world.

That's not a metaphor, we mean physically 30,000 feet in the air.

Anyone that’s followed us for the past 18 months knows the role our voice of customer program has played in bringing our Travel Commerce Platform to market. Clickable prototypes, continuous user testing and validation are just part of our M.O. 

Our team had been seen doing gorilla UX testing while hanging out in airports and coffee shops, running out to take in a tour of the ROM or signing up for and experiencing a private beer tasting (yeah, tough gig).  Perhaps most memorable was our “fly-alongs” where we conduct user testing with passengers and crew by jumping on some of our customers’ flights - basically you find yourself 30,000 feet in the air going 600 mph while working alongside passengers and crew as they use our apps!

We believe that this is the only way to develop and deliver great software that people want and love to use. 

So what do you do when suddenly we are all sheltering in place, planes are grounded and few people are travelling? How do you test and validate the very things airlines and travellers are going to need as planes first start to fly again? 

In our case, it just happened.  No whiteboard. No brainstorming.  Our design and development team just worked with the expectation that testing and validation remained part of our process.  It happens to help that they are very creative and not shy to jump at the opportunity to put themselves out there and ham it up a bit.  Here’s how they did it...

A Virtual Fly-Along

Last week marked our first virtual fly-along. We recreated that flying at 30,000 feet, travelling 600 mph feeling with a virtual plane full of passengers and flight crew to test out our app. 

Meet the Flight Crew, Chinmay and Hank.

Chinmay is normally a product designer at Guestlogix but during this experience, he held the title of Chief Flight Attendant on “Touchless Airlines” which came easy to him because of his 15+ years of experience in the airline industry. “Good afternoon Ladies and gentlemen, my name is Chinmay and I am the Chief Flight Attendant for this flight. On behalf of Captain Goddard and the entire crew, I welcome you onboard Touchless Airlines flight 262, with non-stop service from Dallas to Chicago Midway - Chinmay Buch, Product Designer/C.F.A.

Hank is a Development Manager on our Delivery team where he has helped build software for our platform from its conception. During this experience, Hank took on the title Captain of Flight 262. “Good afternoon Ladies and Gentlemen my name is Captain Goddard and I welcome you onboard this Touchless flight 262 to Chicago Midway.” - Hank Goddard, Development Manager/Captain

Chinmay and Hank along with four other staff in the role of crew members led our virtual passengers through an entire experience starting with boarding, take-off, inflight, descent and landing. After running through our safety demonstration and take-off, we were able to test our app during the inflight portion of the experience. Passengers placed orders online and our crew members fulfilled and delivered the orders to virtual seats. While the inflight part of the flight was the key portion we were testing, it also gave us the opportunity to solicit feedback on other areas we could solve pain points. From boarding through to our descent and landing this simulation was complete with all the sounds and ambient noise we are familiar with and was able to mimic the authenticity of the full flight experience.

If you closed your eyes, the experience felt pretty damn close to the real thing. With some additional tools and technology, a MacGiver-like approach to innovation, and an all-in team that didn’t break character, we were able to simulate an experience and collect incredibly valuable data- all from our own homes during a pandemic, while air travel is restricted globally.

How did we do it?

There was no technical magic to putting this together.  We combined some off the off-the-shelf tools that you’re probably already familiar with.  Here’s the rundown…

Zoom

We used Zoom to bring the entire test group together online. Our virtual passenger participants were asked to keep their video and audio off for the duration of the experience. Our flight crew, however, had their video on and utilized Zoom backgrounds to look as though they were in the cockpit, galley and cabin of a plane. Using Zoom audio, our flight crew was able to deliver realistic announcements and play sounds that are familiar to anyone that’s flown.

Miro

We used a Miro board to create a visual of the plane and onboard seating. Passengers used avatars to represent themselves on the plane and placed their avatar on the seat they selected. Flight attendants used image icons of products - food and beverage items in this case - to drag and drop on the appropriate seats to fulfill passenger orders.

Guestlogix App

Each virtual passenger and crew members had access to the appropriate Guestlogix app.  Passengers used their app to place their orders - primarily to purchase food and drinks  - while crew members used their app to fulfill those orders in a safe, clean, “covid aware” way. No credit cards to exchange, no POS device to handle. Entirely touchless.

Blerp

Blerp was employed to play and mimic sounds you would hear while on an airplane. This really brought the experience to the next level and fully immersed people into how it sounds when travelling onboard an aircraft. Missing that onboard feeling? Listen to the sounds we used here and soothe at least a little bit of that wanderlust, without the struggle of the middle seat.

Some gCrew Flare

Truth be told the experience would not have felt as real without some flare.  Having our team lean into their roles and show off their personalities was key to keeping passengers engaged in this virtual flight.

What were the results?

We set out to test the overall usability of our apps from the interface to performance and bandwidth requirements. The team didn’t face any barriers to prevent them from creating a realistic experience to meet these objectives. While it is too early yet to know if the virtual sessions carry any biases, this first “flight” was an overwhelming success.  After the simulation, the team did exit interviews, sent out two feedback surveys and collected valuable insights from all participants at a 70% completion rate which is about 20% higher than what we had seen in our real world fly-alongs.  

Since the pandemic hit we’ve accelerated our efforts to innovate not just our technology but how we work overall with things like Couchcon2020 and Virtual Fly-Alongs. These are just two examples and we are continually looking to add, adjust and improve.  If you would like to learn more about what’s working here we are happy to share and conversely, if you are up to sharing what’s working for your teams, we’d love to chat.

With significant inventiveness, additional contemporary tools and a dedicated team, we are shooting to operate at a higher level than we did in person.   

“On behalf of Captain Goddard and the entire crew, we’d like to Thank you for choosing Touchless Airlines for your flight today.”