Improving the experience of live stream clipping by keeping users in stream

Project Overview

Date: November 10th - December 1st

Total Project Time: ~20 hours

Responsibilities:
• Survey design
• Participant recruitment
• Defining goal & success metrics
• Rapid Prototyping & Testing
• Writing a project impact report

Twitch Overview

Twitch.tv is currently the worlds leading social video service with a focus on hosting live video entertainment. Because of this, Twitch currently allows 15 million daily visitors to interact about video games, music, creative arts, their lives, and more with over 2 million unique creative streamers.

My Goals

1. To win UX Design Contest #3. TL;DR I tied for first 🏆
2. To design and improve a Twitch Experience of my choosing.
3. To test and improve my skills as a Designer.

This project was initially discovered because of UX Design Contest #3, run and organized by a Twitch UX Designer himself, Adam Ruf (@ux_adam on Twitter). From the excitement in his live streams, the idea of redesigning an experience in a service I use daily quickly captured my attention.

The project revolved around practicing and demonstrating our best skills in:

• Research
• Prototyping
• User Testing
• Writing

and to ultimately improve a current Twitch experience.

There were no boundaries around which experience we had to choose, no rules around how big or small we could think. So my initial problem was filtering through all the fascinating ideas in my head down to one.

Initial Problem

I had no idea what to fix.

Besides personal ideas and hunches, I had no real data on how others thought about the platform. Therefore, my first problem needed to be solved with some good old feedback from the users.

The first step — Asking the right people, the right questions.

1. Find your target users and discover their personas.
2. Find what they like about the service/product.
3. Find what they dislike about the service/product.

When I started this project I had ideas for both the streaming and the viewing side of Twitch, but with limited time I could only solve one. With millions of daily active users, finding a 10+ responses to a quick survey would be easier than expected, so this was my first step in finding who I’d serve.

The survey needed to be fast to complete and open-ended so the user could brain dump a little. All I needed to do was find out what the users liked, and what they didn’t, simple enough.

1. A quick screener question helped group who was answering my questions, so for this I used how often they use the platform each week (from less than one day, to all 7).

2. I kicked the user’s brain into gear by first asking about the experiences they loved about the service and why they used it.

3. Finally, I got to the essential question of the survey, asking what experiences they didn’t like.

4. I also included a fun “If you had a magic wand what would you do…” at the end to get their brains spilling their wildest dreams.

From this inquiry, a flurry of feedback and ideas were received, targeting every corner of the platform which was great to see, but not as narrowed down as I had hoped. (It's much easier when everyone points at the same thing to fix 😋)

The second step — Study the data.

1. Find if there are any trends in the data.
2. Understand if there was any bias.
3. Understand the meaning behind their answers.

Out of 21 responses captured in my survey, I quickly found a few visible trends, and there were also some ideas aligning with my own preconceptions:

• Only 10% of my responses came from the view of the content creator, which meant I would gain data from my fellow viewers a lot easier.

90% said they used the platform more than 4 days a week, which made sense for the high number since most of the distribution of this survey took place within live streams.

43% of the respondents said they use Twitch to discover new content, but 29% also said that discovering content leads to a negative experience.

• Out of all the ideas captured in my final “magic wand” question, 24% aligned with better functionality around creating their own content or finding new content.

At this point all I really had was a target persona to help, the Twitch power viewer who uses the platform more than four days a week. I also narrowed down an area in Twitch I desired to improve, which was content creation and possibly content discovery.

But creation and discovery is still a pretty big area of functionality, and I needed to identify a single experience that could benefit the viewers of Twitch.

I was able to get 9 of the 21 respondents in one-on-one discussions via Discord to see if they had any additional information. When I brought up the area of content creation and discovery, I was initally biased to think that the Twitch viewer persona only focused on the discovery portion. But while observing them use Twitch and view a stream, something new jumped out at me.

There was a function in Twitch that allows viewers to be creators, help out the streamer, and possibly get some fame. Clipping!

Target Aquired - understanding the problem.

Clipping is a way for viewers to capture a short ~60 second snippit from a live stream of something fascinating, funny, or unfortunate that just happened.

A clip can be shared via a shareable link, and they are always attached to both the streamer and the viewer who captured the clip. There is a speck of fame when you see one of your clips gaining over 1000 views, and many streamers reward viewers who can capture great clips. So always being on the lookout for that next awesome clip is something many active viewers are aware of.

3rd party browser extensions used to own this functionality, and I’m assuming due to their popularity and the ability to contain more content, Twitch felt it made sense to build their own version. The ability to relive an experience is fantastic, and being able to capture an event with the click of a button is perfect, but there was one thing that bugged me (and other users too).

The current clipping experience removes the user from the live stream and drops them into a new editing window!

The page itself is pretty basic but has a couple of frustrations:

1. Some people found it odd that it doesn't remember your theme settings (and it made them feel that this was disconnected from Twitch itself.)

2. It didn't feel fast to create something that should be "in the moment". The video slider tool can take a few seconds to get used too and people felt rushed to get back to the stream.

3. It's a full page takeover that doesn't do a great job at explaining what is happening, what the user is supposed to do, and why the user is being navigated to a new tab away after clicking a small icon or how to get back.

4. Should viewers really need to be relocated for this task? Is this worth breaking the current experience of watching something live?

The issue with removing a viewer from a live stream so they can create a clip is.. well… they’re no longer experiencing the live stream. You've officially paused/stopped/broken their experience.

The clip icon generated immediate FOMO (Fear of missing out) at the click of a button.

My goal for this project became discovering if there was a way to redesign the clipping experience to keep usability at the forefront and the viewer in the live stream so they have minimal anxiety of missing anything.

Planning Success - How can we measure UX?

Being able to measure product usage is a reasonably simple task. Usage of a page, time using a feature, number of objects created, click through rates. All of these are simple numbers to track.

But measuring the feelings of a user when they interact with a product is where things get tricky.

A user now spending 20 minutes on your page vs. 10 minutes could mean either they are loving it, or are having a hard time.

Google understood this and helped out by creating a framework of metrics to measure experiences which they call the HEART framework.

For my goal of improving the clipping experience, I could undoubtedly use the entire HEART framework to track success. But if there was a release of my proposed solution, there are three success metrics I would focus on primarily;

Happiness — This is as simple as giving the user a little 0-10 or thumbs up / thumbs down survey asking how much they enjoyed using the functionality.

Retention  — This focuses on how many users continue to come back and reuse the functionality within a set time frame.

Task Success  — Basically, how many users successfully complete the task? How many stumbled along the way? How many completely disbanded the task?

Finally, some basic heuristics evaluations can help identify if the system is working along with the user to keep them informed on what is happening. They should have a clear understanding of what is happening once they click the clip icon, and they should have a simple way of canceling the task if it was done on error or by mistake so that they can continue with their live stream experience.

Creativity Time — Pen to Paper, Mouse to Canvas.

My initial idea was to remove the separated page altogether and bring the same elements straight into the live stream so there was no user relocation. This would overlay the live stream with the clipped video and add the video controls below. This way the user still feels like they haven’t left the area.

The Good:

• This keeps the user in the live stream so they feel like they are not missing anything.

The Bad:

• They still cannot see the live stream.



Although this fixed some of the relocation issue, I realized there was still anxiety when a user is unable to see the stream. The new goal was to keep the stream in view at all times to reduce the majority of the users FOMO.

Looking at the layout of a Twitch stream I understood that free real estate was minimal, so something would have to be partially or fully hidden. It came down to the user’s priorities.

After some doodling and wireframing, I ended up with two layouts that I thought reduced FOMO, kept users in the stream, and allowed them to have all the same functionality that they have today.

Design One

Included a clip editing modal window that would open in the lower right corner of the live stream. Which is the same area where the clip button lives which allowed the users’s mouse to barely move, and deeming this area as a sort of tool/clip area.

Design Two

Had a clip editing modal much similar to design one, which would appear at the top of the users chat. This would allow the user to keep a full view of the live stream, though lose some of their older chat messages towards the top.

Let them Battle — Test and test and test.

1. Which design do people intuitively understand?
2. Is the design usable?
3. Is the design preferred over the current implementation?

Both of these designs had seemingly had equal pros and cons so I reached back out to the wild web of the west and asked for its user’s opinions in the form of a UsabilityHub preference test.

The results poured in quick and leaned pretty heavily towards design two with a 69.2% victory over design one.

Along with their choices, people left details on why they chose one option over the other. While design one was given kudos for looking clean and embedded into the stream, design two allowed no disruption to the stream at all which users cared about more than some old messages.

At this point, a top-of-chat clip creation object was winning, and I was happy with the proposed new clip creation experience. All I needed now was a bit more confirmation before I started writing out some long informative case study explaining how I redesigned.. ah.. sorry...

I reached back out in the form of another UsabilityHub test (I really love this tool btw). This test was simply a preference test, where users would chose which design they found more pleasing to the eye between the current version and my own. This survey was set to auto arrange the options to minimize any chances of biases leaning towards seeing one design before the other.

The new visual design came back as an overwhelming winner.

Finally, I set my Figma prototype to work to learn just how usable and understandable my design was. I was able to get 11 people to test the current implementation followed by them testing my prototype to collect their thoughts. The tests were performed via video calls using appear.in so I could watch their screens, their actions, and their frustrations.

From the tests, both versions were deemed simple enough to figure out and no one failed the task of creating a clip. But, 64% mentioned that they liked having the view of stream in the new version, and 27% said that the new design felt more "like Twitch".

So, What did I learn? Any Next steps?

Well, I’m still a big lover of Twitch, and I learned a lot more about them by spending just 20 hours digging through their users, some basic data, and a single function in their large platform.

Twitch is a large scale company, I have no idea how their product development process works, and I have almost no user data or business objectives that would deem this design to be useful for them. However, from what I had (nothing) I was pretty pleased with my outcomes.

The content and community of Twitch is what makes it so valuable. The smallest interruption in either can break the experience and make the user start over, or leave.

The new design is a success from a limited design standpoint. People like the visual design and functionality for the web but this functionality would also need to be available on mobile devices and possibly across multiple platforms such as smart TV's, gaming consoles, and digital media players. If I had more time, I would love to design how this functionality would work and look on all devices and platforms.

There are still some questions I have unanswered, such as if this change would actually show an improvement in success metrics? Does this change have any actual business benefits for Twitch or is this purely an enjoyment and UX improvement? Also, would a smaller clip editing window cause eye strain or difficulty creating a clip vs. the larger full page tool we get to use today?

Overall I believe this proposed new experience of clipping is a step forward to some improvements to the current clipping experience, or at least this research can shine some light on an area in Twitch that could use a bit of polishing.

And finally, if you’ve made it this far, I share my sincerest gratitude for reading!