{"id":6344,"date":"2021-03-15T09:39:00","date_gmt":"2021-03-15T17:39:00","guid":{"rendered":"https:\/\/www.emergeinteractive.com\/?p=6344"},"modified":"2022-04-01T13:56:57","modified_gmt":"2022-04-01T21:56:57","slug":"moderated-unmoderated-user-testing","status":"publish","type":"post","link":"https:\/\/www.emergeagency.com\/insights\/detail\/moderated-unmoderated-user-testing\/","title":{"rendered":"Moderated vs Unmoderated User Testing: Which is Best for My Project?"},"content":{"rendered":"\n

Can your customers use your digital product? And once you know if they can, more importantly will they <\/em>want<\/em><\/strong> to use it<\/em>? Whether your digital product is still conceptual, newly released to the market, or has been out in the wild for some time, if you don\u2019t know the answers to these questions currently, it\u2019s time to ask them directly through user testing.<\/p>\n\n\n\n

There are many methods for conducting user testing available today: guerrilla testing, lab testing, remote testing, card sorting, and more. With so many options, it can be a significant challenge to know what is right for your own project. As a product owner or manager, you may also be asking yourself: \u201cWhat\u2019s the best value for my company\u2019s investment?\u201d<\/em><\/p>\n\n\n\n

Let’s explore two methods that have created exceptional results for our clients\u2019 digital products and their respective customers: moderated vs. unmoderated usability testing. These insights will assist you in making a more informed decision to ensure an optimal customer experience<\/a>. We will also touch on the two primary bottlenecks that challenge many companies: whether or not to employ user testing at all; and time and money.<\/p>\n\n\n\n

When it comes to the overall success of your business, these common barriers to entry can be overcome by adopting this simple yet effective mantra: \u201cWithout customers, you have no business.\u201d <\/em>It\u2019s a mantra worth repeating and remembering, because it\u2019s 100% true.<\/p>\n\n\n\n


Decide what to test<\/h4>\n\n\n\n

In order to establish a better basis for collecting insights to process from user testing, you\u2019ll need to understand exactly what to test. If you\u2019re not sure where to begin, we suggest initiating an empathy mapping<\/a> workshop with key project stakeholders. This will promote a refined perspective on the pain points your audiences are encountering. These pain points are linked to key tasks performed, as well as their ultimate goal. Pain points can help shed light on friction points as well as opportunities for improvement that can be validated with customer insights.<\/p>\n\n\n\n

Taking this process a step further, you could also use analytics to visualize patterns where customers are running into challenges. Strictly using analytics however would be challenging, because the numbers reveal only one side of the story. To truly understand what a problem is, you must gain perspective directly from your customers by empathizing with them.<\/p>\n\n\n\n


Create a user testing script<\/h4>\n\n\n\n

Next, you\u2019ll use these insights to generate a user testing script. Whether it\u2019s using the current digital product or a conceptual one, think chronologically about the tasks you want testers to perform, as well as the steps involved. For each task, identify the user story, i.e., \u201cAs a [user], I would [perform an action]<\/em> to [achieve a specific outcome].\u201d<\/em> To substantiate if the user story is attainable, create a corresponding task question for your user to perform. For example, you might ask testers \u201cIf you were going to check for new messages, where might you go to perform this task?\u201d<\/em>

To assist with this script in testing, you\u2019ll want to ensure there is something to test against, i.e. a visual aid for your customers to engage with. If your digital product is still conceptual you could use sketches, wireframes, concept art, or simple prototypes. If your digital product is currently live or you have a minimal viable product<\/a>, you additionally have the ability to choose whether you want to gather customer insights from the live digital product, or a conceptual solution based on the design system<\/a>.

With a test script ready and a visual aid to assist with the testing process, you\u2019re ready to make some decisions on what type of user testing is right for your project.<\/p>\n\n\n\n

With either testing scenario, your comprehensive goal is to understand from a quantitative analysis whether or not the tasks are viable and qualitatively, if testers liked or disliked the experience of performing the task. <\/p>\n\n\n\n


Moderated User Testing <\/h4>\n\n\n\n
Less expensive, but time intensive<\/h5>\n\n\n\n

When it comes to being prudent with finances, but getting direct feedback on your digital product from your customers, moderated testing is hard to beat. What you save in financial costs however will be offset by the timeline involved to conduct this kind of testing. This method requires substantial  planning and logistical coordination between both your staff and your customers.

Firstly, you are responsible for coordinating these meetings with your customers. You will need to contact them directly and ask them to participate in a user test. In most circumstances, you will also need to provide an incentive. 

Once meeting times have been scheduled, which will vary roughly from 30 minutes to an hour based on the length of your user testing script, you\u2019ll need someone to run customers through each task. The goal is to be as unbiased as possible during this process. You don\u2019t want to unintentionally lead them to the answer. You want customers to try completing each task on their own, only interjecting if they hit a stumbling block and ask for additional clarification.<\/p>\n\n\n\n

If this process seems daunting, it\u2019s possible to partner with a digital product agency<\/a> such as Emerge to conduct these interviews. If in-person testing is not feasible, the testing can also be conducted remotely by utilizing video chat platforms with screen sharing capabilities. There may be a challenge with remote sessions if additional time orienting testers to this environment is required. Ideally, you want their camera on so you can see their facial expressions and share their screen if they are clicking on a prototype link.

Another aspect with this type of testing is that if you conduct it yourself, it is incumbent on you to parse the data from each session. You’ll need to prepare documentation to present back to your stakeholder team so they can understand how testing went. What was the breakdown of pass or fail from a quantitative standpoint? At what point did they fail, and qualitatively how did they rank their experience and what did they have to say about it? All of these factors are critical in being able to understand what constitutes a problem and what to do about it.<\/p>\n\n\n\n


Unmoderated User Testing<\/h4>\n\n\n\n
More expensive, but less time intensive<\/h5>\n\n\n\n

There are many platforms now available to perform remote user testing on your behalf, such as Usertesting.com<\/a>, Userlytics.com<\/a>, and Trymyui.com<\/a>. Depending on your specific needs, it\u2019s best to go to each site and research the costs involved before you go further because those costs can add up quickly. But these platforms are great for automating the process of collecting insights.

How this is achieved is that these platforms have a subscriber base that is paid for each test that they take. You can create screeners to narrow down the types of test takers that may elect to take a test of yours, but it is good to keep in mind that these users may or may not be current customers of yours.

Some platforms allow you to specify test participants by email address, so it\u2019s possible to use your current customers. But the participants will need to download and run user testing platform software to take the test, which can be a challenge for individuals who are less technically savvy. On desktop, this is not as big of a challenge as the software is relatively easy to install and use. For mobile users however, this quickly becomes cumbersome as users will need to go through a series of steps that include privacy prompts and jumping through specific installation hoops. By the time your users arrive at the test they may already be negatively biased based on how difficult this process was for them.

Within these platforms, you\u2019ll take your testing script and use the platform\u2019s specific test building tools to generate your test and submit it for testing. In the platforms mentioned above, this process is relatively easy to perform and get going. The only variable that changes depending on the platform used is cost. Some may charge per user, and some may change a flat monthly testing fee.

As for the time required to run testing and process results, once tests are submitted you can usually expect a turnaround time of 24 hours or less if using the platform’s paid subscribers. And, you\u2019ll get a report generated for each test that illustrates pass and fail metrics as well as any likes or dislikes around each task performed along the way. The only downside is manually reviewing each user test to ensure the data was captured correctly and that test-takers didn\u2019t run into any challenges. If you\u2019re going this route, we suggest performing an initial \u201cdry-run\u201d with a single test taker before proceeding further.<\/p>\n\n\n\n


Which usability test is right for my project?<\/h4>\n\n\n\n

While both moderated and unmoderated testing are equally viable in many project circumstances, it\u2019s best to know what your project budget and timeline are. With some thoughtfulness ahead of execution, timelines can be padded to run moderated testing and gather significant insights that lead to successful outcomes, while an increase in project budgets to run unmoderated testing can be more challenging but worthwhile if timing is an issue.

With either case, think again about the mantra from above. \u201cWithout customers, you have no business.\u201d <\/em>Completing a project without user testing is always possible, but at what cost to your business? If the experience that is built doesn\u2019t service your customers\u2019 needs, they may venture elsewhere to achieve their goals. It\u2019s a big risk to take, so it\u2019s important to consider this carefully.<\/p>\n\n\n\n


What if I need further help with testing?<\/strong><\/h4>\n\n\n\n

Moderated and unmoderated user testing are both great ways to get invaluable user and customer feedback, but either can still be a daunting challenge for those who have never run such a process or are still uncertain which is best for their unique situation.<\/p>\n\n\n\n

Concerning either circumstance, Emerge is here to help. We have an experienced team of design practitioners who specialize in both areas of testing focus that have led many successful user testing scenarios for our clients. Our approach is strategic and collaborative \u2014 aimed at rectifying the digital product challenges that simultaneously affect your business, your brand, and your customers. We\u2019re always happy to consult, so if you want to talk with us about your unique situation we\u2019d be happy to give you an unbiased understanding of what we can do for you. Schedule a call<\/a> with us today to find out more.<\/p>\n","protected":false},"excerpt":{"rendered":"

Can your customers use your digital product? And once you know if they can, more importantly will they want to use it? Whether your digital product is still conceptual, newly released to the market, or has been out in the wild for some time, if you don\u2019t know the answers to these questions currently, it\u2019s time to ask […]<\/p>\n","protected":false},"author":15,"featured_media":6385,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[5,20,23],"tags":[],"message_area_of_focus":[36],"buy_cycle":[37],"target_audience":[40],"acf":[],"yoast_head":"\nUser Testing: Is Moderated or Unmoderated Better?<\/title>\n<meta name=\"description\" content=\"Moderated and unmoderated user testing each have different strengths. Learn how to decide which is best for your digital product.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.emergeagency.com\/insights\/detail\/moderated-unmoderated-user-testing\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"User Testing: Is Moderated or Unmoderated Better?\" \/>\n<meta property=\"og:description\" content=\"Moderated and unmoderated user testing each have different strengths. Learn how to decide which is best for your digital product.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.emergeagency.com\/insights\/detail\/moderated-unmoderated-user-testing\/\" \/>\n<meta property=\"og:site_name\" content=\"EMERGE | UX Agency\" \/>\n<meta property=\"article:published_time\" content=\"2021-03-15T17:39:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2022-04-01T21:56:57+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.emergeagency.com\/wp-content\/uploads\/2020\/08\/Unmoderated_vs_moderated_testing_DETAIL-1.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1440\" \/>\n\t<meta property=\"og:image:height\" content=\"550\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Jonathon Hensley\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@EmergeInteract\" \/>\n<meta name=\"twitter:site\" content=\"@EmergeInteract\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Jonathon Hensley\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"9 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.emergeagency.com\/insights\/detail\/moderated-unmoderated-user-testing\/\",\"url\":\"https:\/\/www.emergeagency.com\/insights\/detail\/moderated-unmoderated-user-testing\/\",\"name\":\"User Testing: Is Moderated or Unmoderated Better?\",\"isPartOf\":{\"@id\":\"https:\/\/www.emergeagency.com\/#website\"},\"datePublished\":\"2021-03-15T17:39:00+00:00\",\"dateModified\":\"2022-04-01T21:56:57+00:00\",\"author\":{\"@id\":\"https:\/\/www.emergeagency.com\/#\/schema\/person\/f7b2a2674b7ff8beb49410a06d7b3fa1\"},\"description\":\"Moderated and unmoderated user testing each have different strengths. Learn how to decide which is best for your digital product.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.emergeagency.com\/insights\/detail\/moderated-unmoderated-user-testing\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.emergeagency.com\/insights\/detail\/moderated-unmoderated-user-testing\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.emergeagency.com\/insights\/detail\/moderated-unmoderated-user-testing\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.emergeagency.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Moderated vs Unmoderated User Testing: Which is Best for My Project?\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.emergeagency.com\/#website\",\"url\":\"https:\/\/www.emergeagency.com\/\",\"name\":\"EMERGE | UX Agency\",\"description\":\"EMERGE is a UX design agency in Portland, OR. We help product and marketing leaders in healthcare, manufacturing, and tech drive revenue and improve outcomes through digital products that increase customer engagement and retention. Our specialties are in UX\/UI Design, product strategy, and full-stack development.\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.emergeagency.com\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.emergeagency.com\/#\/schema\/person\/f7b2a2674b7ff8beb49410a06d7b3fa1\",\"name\":\"Jonathon Hensley\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.emergeagency.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/www.emergeagency.com\/wp-content\/uploads\/2019\/01\/jon-96x96.jpg\",\"contentUrl\":\"https:\/\/www.emergeagency.com\/wp-content\/uploads\/2019\/01\/jon-96x96.jpg\",\"caption\":\"Jonathon Hensley\"},\"url\":\"https:\/\/www.emergeagency.com\/author\/jon-henseley\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"User Testing: Is Moderated or Unmoderated Better?","description":"Moderated and unmoderated user testing each have different strengths. Learn how to decide which is best for your digital product.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.emergeagency.com\/insights\/detail\/moderated-unmoderated-user-testing\/","og_locale":"en_US","og_type":"article","og_title":"User Testing: Is Moderated or Unmoderated Better?","og_description":"Moderated and unmoderated user testing each have different strengths. Learn how to decide which is best for your digital product.","og_url":"https:\/\/www.emergeagency.com\/insights\/detail\/moderated-unmoderated-user-testing\/","og_site_name":"EMERGE | UX Agency","article_published_time":"2021-03-15T17:39:00+00:00","article_modified_time":"2022-04-01T21:56:57+00:00","og_image":[{"width":1440,"height":550,"url":"https:\/\/www.emergeagency.com\/wp-content\/uploads\/2020\/08\/Unmoderated_vs_moderated_testing_DETAIL-1.jpg","type":"image\/jpeg"}],"author":"Jonathon Hensley","twitter_card":"summary_large_image","twitter_creator":"@EmergeInteract","twitter_site":"@EmergeInteract","twitter_misc":{"Written by":"Jonathon Hensley","Est. reading time":"9 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.emergeagency.com\/insights\/detail\/moderated-unmoderated-user-testing\/","url":"https:\/\/www.emergeagency.com\/insights\/detail\/moderated-unmoderated-user-testing\/","name":"User Testing: Is Moderated or Unmoderated Better?","isPartOf":{"@id":"https:\/\/www.emergeagency.com\/#website"},"datePublished":"2021-03-15T17:39:00+00:00","dateModified":"2022-04-01T21:56:57+00:00","author":{"@id":"https:\/\/www.emergeagency.com\/#\/schema\/person\/f7b2a2674b7ff8beb49410a06d7b3fa1"},"description":"Moderated and unmoderated user testing each have different strengths. Learn how to decide which is best for your digital product.","breadcrumb":{"@id":"https:\/\/www.emergeagency.com\/insights\/detail\/moderated-unmoderated-user-testing\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.emergeagency.com\/insights\/detail\/moderated-unmoderated-user-testing\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.emergeagency.com\/insights\/detail\/moderated-unmoderated-user-testing\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.emergeagency.com\/"},{"@type":"ListItem","position":2,"name":"Moderated vs Unmoderated User Testing: Which is Best for My Project?"}]},{"@type":"WebSite","@id":"https:\/\/www.emergeagency.com\/#website","url":"https:\/\/www.emergeagency.com\/","name":"EMERGE | UX Agency","description":"EMERGE is a UX design agency in Portland, OR. We help product and marketing leaders in healthcare, manufacturing, and tech drive revenue and improve outcomes through digital products that increase customer engagement and retention. Our specialties are in UX\/UI Design, product strategy, and full-stack development.","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.emergeagency.com\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.emergeagency.com\/#\/schema\/person\/f7b2a2674b7ff8beb49410a06d7b3fa1","name":"Jonathon Hensley","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.emergeagency.com\/#\/schema\/person\/image\/","url":"https:\/\/www.emergeagency.com\/wp-content\/uploads\/2019\/01\/jon-96x96.jpg","contentUrl":"https:\/\/www.emergeagency.com\/wp-content\/uploads\/2019\/01\/jon-96x96.jpg","caption":"Jonathon Hensley"},"url":"https:\/\/www.emergeagency.com\/author\/jon-henseley\/"}]}},"_links":{"self":[{"href":"https:\/\/www.emergeagency.com\/wp-json\/wp\/v2\/posts\/6344"}],"collection":[{"href":"https:\/\/www.emergeagency.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.emergeagency.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.emergeagency.com\/wp-json\/wp\/v2\/users\/15"}],"replies":[{"embeddable":true,"href":"https:\/\/www.emergeagency.com\/wp-json\/wp\/v2\/comments?post=6344"}],"version-history":[{"count":19,"href":"https:\/\/www.emergeagency.com\/wp-json\/wp\/v2\/posts\/6344\/revisions"}],"predecessor-version":[{"id":7662,"href":"https:\/\/www.emergeagency.com\/wp-json\/wp\/v2\/posts\/6344\/revisions\/7662"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.emergeagency.com\/wp-json\/wp\/v2\/media\/6385"}],"wp:attachment":[{"href":"https:\/\/www.emergeagency.com\/wp-json\/wp\/v2\/media?parent=6344"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.emergeagency.com\/wp-json\/wp\/v2\/categories?post=6344"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.emergeagency.com\/wp-json\/wp\/v2\/tags?post=6344"},{"taxonomy":"message_area_of_focus","embeddable":true,"href":"https:\/\/www.emergeagency.com\/wp-json\/wp\/v2\/message_area_of_focus?post=6344"},{"taxonomy":"buy_cycle","embeddable":true,"href":"https:\/\/www.emergeagency.com\/wp-json\/wp\/v2\/buy_cycle?post=6344"},{"taxonomy":"target_audience","embeddable":true,"href":"https:\/\/www.emergeagency.com\/wp-json\/wp\/v2\/target_audience?post=6344"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}