{"id":17779,"date":"2020-04-01T09:18:50","date_gmt":"2020-04-01T07:18:50","guid":{"rendered":"https:\/\/www.testingtime.com\/?p=17779"},"modified":"2021-12-21T11:46:33","modified_gmt":"2021-12-21T09:46:33","slug":"great-quality-and-reliability-are-no-coincidence","status":"publish","type":"post","link":"https:\/\/www.testingtime.com\/en\/blog\/great-quality-and-reliability-are-no-coincidence\/","title":{"rendered":"Great quality and reliability are no coincidence"},"content":{"rendered":"

Table of contents<\/span><\/h2>\n

1. Phone number verification through SMS
\n<\/span><\/a>2. State of the art screening<\/a>
\n3. Profile prediction algorithm<\/a>
\n4. Quality prediction algorithm<\/a>
\n5. Reconfirmation of profile criteria and availability<\/a>
\n6. Email and SMS reminder<\/a><\/p>\n

<\/a>1. Phone number verification through SMS<\/span><\/h2>\n

Initially, we only supported email verification during a test user signup. We collected the smartphone phone number, but didn\u2019t verify whether the number was actually reachable. As a consequence, reminder SMS didn\u2019t go through or even worse, a customer reached an invalid number when trying to reach out. To avoid such poor customer experience, we took the verification process to the next level and introduced a phone number activation using an SMS code. Without this activation, a test user will not be placed as a candidate. It also allows us to screen out fake users, double profiles or bots.<\/span><\/p>\n

<\/a>2. State of the art screening<\/span><\/h2>\n

Every test user recruitment comes with a screening, where we check if a test user fits the profile requested or not. A screening is composed of a collection of screening questions with built-in logic to screen in\/out candidates. Writing good screener questions is a science. It must be as short as possible (to ensure a high conversion rate) but open enough to not convey what profile we are looking for. At TestingTime, we write the screener based on the requested profile criteria. We have built a best-practice screener question catalogue, with questions we know work well. Each answered screener question comes with an expiration date to ensure a candidate doesn\u2019t need to reply to the same question every time, but still frequently enough to guarantee up-to-date profile information. TestingTime made screening its expertise <\/span>\u2013<\/span> our whole Customer Success Team is trained and focused on this.\u00a0\u00a0<\/span><\/p>\n

<\/a>3. Profile prediction algorithm<\/span><\/h2>\n

In an ideal world, we would already know who matches the criteria for a specific study. A lot of our competitors ask an endless catalogue of questions when test users sign up. At TestingTime we decided to only ask as few questions as possible during the signup process and build their profile by asking them additional questions only when we have a potential study for the test user. Our goal is to send as few screening invites as possible to generate the necessary number of candidates.\u00a0<\/span><\/p>\n

Too many invites to a screening with no tests tend to frustrate test users after a while. That\u2019s why some potential test users don\u2019t reply to all questions anymore. Therefore we have to make a wild guess or predict candidates’ answers. In order to solve this problem, we have built an algorithm to predict whether the person may fit the wanted criteria or not. Based on that we can prioritize who to ask first. This results in fewer invites and increases the willingness of test users to participate in screeners.\u00a0<\/span><\/p>\n

\"test<\/p>\n

<\/a>4. Quality prediction algorithm<\/span><\/h2>\n

Based on data our quality algorithm predicts whether a candidate will return a good or bad rating from the customer. This check is based on another machine learning algorithm which gets better with every rating we receive. That\u2019s why it is very important for all customers to rate their participants at the end of a study. Once enough candidates for a study are generated, our algorithm picks the right mix and confirms their participation. At the moment, the algorithm is only used to support the selection. The final call is made by the responsible customer success manager. In the near future, we believe that the entire selection process can be automated.\u00a0<\/span><\/p>\n

<\/a>5. Reconfirmation of profile criteria and availability<\/span><\/h2>\n

The profile and the quality prediction algorithm screen out a lot of potential misfits and no-shows. However, we have learnt that human misunderstandings are still hard to detect. This can happen when a screener question is misunderstood or by mistake a wrong option was selected. Therefore we have built-in an extra layer of reconfirmation into the system. When the system asks a final candidate to reconfirm their availability, the person is also asked to reconfirm the required criteria. This last line of defence filters out potential misfits.<\/span><\/p>\n

<\/a>6. Email and SMS reminder\u00a0<\/span><\/h2>\n

Another thing which can go wrong is that a scheduled test user simply forgets about the test or drops sick. We send an email and SMS reminder to every test user. In both reminders, we make it super easy to cancel. We would rather know that the test user can\u2019t make it and find a replacement than not knowing about it.<\/span><\/p>\n

With all these checks in place, we managed to get our no-show and misfit rate to a stable low number and to ensure having no study junkies at our customers\u2019 test. However, we don\u2019t stop our efforts in improving TestingTime\u2019s test users quality even further. For example, we plan to look into ID verification. With many online banks coming up, there are now great solutions to verify an ID online. We believe it could bring even more trust into the quality of a test user. <\/span><\/p>\n","protected":false},"excerpt":{"rendered":"

Table of contents 1. Phone number verification through SMS 2. State of the art screening 3. Profile prediction algorithm 4. Quality prediction algorithm 5. Reconfirmation of profile criteria and availability 6. Email and SMS reminder 1. Phone number verification through SMS Initially, we only supported email verification during a test user signup. We collected the […]<\/p>\n","protected":false},"author":5,"featured_media":17523,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"om_disable_all_campaigns":false,"footnotes":""},"categories":[8987,8989],"tags":[],"class_list":["post-17779","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-business-leadership","category-research-strategy"],"acf":[],"yoast_head":"\nGreat quality and reliability are no coincidence - TestingTime<\/title>\n<meta name=\"description\" content=\"Every month, thousands of test users are screened, scheduled, delivered and paid through the TestingTime system. Learn more about our screening process.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.testingtime.com\/en\/blog\/great-quality-and-reliability-are-no-coincidence\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Great quality and reliability are no coincidence\" \/>\n<meta property=\"og:description\" content=\"Every month, thousands of test users are screened, scheduled, delivered and paid through the TestingTime system. Learn more about our screening process.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.testingtime.com\/en\/blog\/great-quality-and-reliability-are-no-coincidence\/\" \/>\n<meta property=\"og:site_name\" content=\"TestingTime\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/testingtime\" \/>\n<meta property=\"article:published_time\" content=\"2020-04-01T07:18:50+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2021-12-21T09:46:33+00:00\" \/>\n<meta name=\"author\" content=\"Reto Laemmler\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:image\" content=\"https:\/\/www.testingtime.com\/app\/uploads\/2020\/03\/Private-Pool-All-Illustrations_Zeichenfla\u0308che-1-Kopie-7-copy-4.svg\" \/>\n<meta name=\"twitter:creator\" content=\"@testingtime\" \/>\n<meta name=\"twitter:site\" content=\"@testingtime\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Reto Laemmler\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.testingtime.com\/en\/blog\/great-quality-and-reliability-are-no-coincidence\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.testingtime.com\/en\/blog\/great-quality-and-reliability-are-no-coincidence\/\"},\"author\":{\"name\":\"Reto Laemmler\",\"@id\":\"https:\/\/www.testingtime.com\/en\/#\/schema\/person\/79095def7dada61a18ac99861db51e9c\"},\"headline\":\"Great quality and reliability are no coincidence\",\"datePublished\":\"2020-04-01T07:18:50+00:00\",\"dateModified\":\"2021-12-21T09:46:33+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.testingtime.com\/en\/blog\/great-quality-and-reliability-are-no-coincidence\/\"},\"wordCount\":835,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/www.testingtime.com\/en\/#organization\"},\"articleSection\":[\"Business & Leadership\",\"Research strategy\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.testingtime.com\/en\/blog\/great-quality-and-reliability-are-no-coincidence\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.testingtime.com\/en\/blog\/great-quality-and-reliability-are-no-coincidence\/\",\"url\":\"https:\/\/www.testingtime.com\/en\/blog\/great-quality-and-reliability-are-no-coincidence\/\",\"name\":\"Great quality and reliability are no coincidence - TestingTime\",\"isPartOf\":{\"@id\":\"https:\/\/www.testingtime.com\/en\/#website\"},\"datePublished\":\"2020-04-01T07:18:50+00:00\",\"dateModified\":\"2021-12-21T09:46:33+00:00\",\"description\":\"Every month, thousands of test users are screened, scheduled, delivered and paid through the TestingTime system. Learn more about our screening process.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.testingtime.com\/en\/blog\/great-quality-and-reliability-are-no-coincidence\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.testingtime.com\/en\/blog\/great-quality-and-reliability-are-no-coincidence\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.testingtime.com\/en\/blog\/great-quality-and-reliability-are-no-coincidence\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"TestingTime\",\"item\":\"https:\/\/www.testingtime.com\/en\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Great quality and reliability are no coincidence\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.testingtime.com\/en\/#website\",\"url\":\"https:\/\/www.testingtime.com\/en\/\",\"name\":\"TestingTime\",\"description\":\"Wir rekrutieren Testpersonen\",\"publisher\":{\"@id\":\"https:\/\/www.testingtime.com\/en\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.testingtime.com\/en\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.testingtime.com\/en\/#organization\",\"name\":\"TestingTime\",\"url\":\"https:\/\/www.testingtime.com\/en\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.testingtime.com\/en\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.testingtime.com\/app\/uploads\/2017\/04\/logo.svg\",\"contentUrl\":\"https:\/\/www.testingtime.com\/app\/uploads\/2017\/04\/logo.svg\",\"width\":1,\"height\":1,\"caption\":\"TestingTime\"},\"image\":{\"@id\":\"https:\/\/www.testingtime.com\/en\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/testingtime\",\"https:\/\/twitter.com\/testingtime\",\"https:\/\/www.instagram.com\/testingtime\/\",\"https:\/\/www.linkedin.com\/company-beta\/9231506\/\",\"https:\/\/www.youtube.com\/channel\/UCpnMUgCz5FiiCUXU-U8ub1w\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.testingtime.com\/en\/#\/schema\/person\/79095def7dada61a18ac99861db51e9c\",\"name\":\"Reto Laemmler\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.testingtime.com\/en\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/www.testingtime.com\/app\/uploads\/2024\/03\/Image-2-130x130.png\",\"contentUrl\":\"https:\/\/www.testingtime.com\/app\/uploads\/2024\/03\/Image-2-130x130.png\",\"caption\":\"Reto Laemmler\"},\"description\":\"Reto ist Founder und CEO von TestingTime. Zuvor hat er als erster Mitarbeiter und VP Product Management bei Doodle, sein UX und Testing Handwerk gelernt. Test early, test often ist sein Mantra!\",\"url\":\"https:\/\/www.testingtime.com\/en\/blog\/author\/reto-laemmler\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Great quality and reliability are no coincidence - TestingTime","description":"Every month, thousands of test users are screened, scheduled, delivered and paid through the TestingTime system. Learn more about our screening process.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.testingtime.com\/en\/blog\/great-quality-and-reliability-are-no-coincidence\/","og_locale":"en_US","og_type":"article","og_title":"Great quality and reliability are no coincidence","og_description":"Every month, thousands of test users are screened, scheduled, delivered and paid through the TestingTime system. Learn more about our screening process.","og_url":"https:\/\/www.testingtime.com\/en\/blog\/great-quality-and-reliability-are-no-coincidence\/","og_site_name":"TestingTime","article_publisher":"https:\/\/www.facebook.com\/testingtime","article_published_time":"2020-04-01T07:18:50+00:00","article_modified_time":"2021-12-21T09:46:33+00:00","author":"Reto Laemmler","twitter_card":"summary_large_image","twitter_image":"https:\/\/www.testingtime.com\/app\/uploads\/2020\/03\/Private-Pool-All-Illustrations_Zeichenfla\u0308che-1-Kopie-7-copy-4.svg","twitter_creator":"@testingtime","twitter_site":"@testingtime","twitter_misc":{"Written by":"Reto Laemmler","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.testingtime.com\/en\/blog\/great-quality-and-reliability-are-no-coincidence\/#article","isPartOf":{"@id":"https:\/\/www.testingtime.com\/en\/blog\/great-quality-and-reliability-are-no-coincidence\/"},"author":{"name":"Reto Laemmler","@id":"https:\/\/www.testingtime.com\/en\/#\/schema\/person\/79095def7dada61a18ac99861db51e9c"},"headline":"Great quality and reliability are no coincidence","datePublished":"2020-04-01T07:18:50+00:00","dateModified":"2021-12-21T09:46:33+00:00","mainEntityOfPage":{"@id":"https:\/\/www.testingtime.com\/en\/blog\/great-quality-and-reliability-are-no-coincidence\/"},"wordCount":835,"commentCount":0,"publisher":{"@id":"https:\/\/www.testingtime.com\/en\/#organization"},"articleSection":["Business & Leadership","Research strategy"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.testingtime.com\/en\/blog\/great-quality-and-reliability-are-no-coincidence\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.testingtime.com\/en\/blog\/great-quality-and-reliability-are-no-coincidence\/","url":"https:\/\/www.testingtime.com\/en\/blog\/great-quality-and-reliability-are-no-coincidence\/","name":"Great quality and reliability are no coincidence - TestingTime","isPartOf":{"@id":"https:\/\/www.testingtime.com\/en\/#website"},"datePublished":"2020-04-01T07:18:50+00:00","dateModified":"2021-12-21T09:46:33+00:00","description":"Every month, thousands of test users are screened, scheduled, delivered and paid through the TestingTime system. Learn more about our screening process.","breadcrumb":{"@id":"https:\/\/www.testingtime.com\/en\/blog\/great-quality-and-reliability-are-no-coincidence\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.testingtime.com\/en\/blog\/great-quality-and-reliability-are-no-coincidence\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.testingtime.com\/en\/blog\/great-quality-and-reliability-are-no-coincidence\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"TestingTime","item":"https:\/\/www.testingtime.com\/en\/"},{"@type":"ListItem","position":2,"name":"Great quality and reliability are no coincidence"}]},{"@type":"WebSite","@id":"https:\/\/www.testingtime.com\/en\/#website","url":"https:\/\/www.testingtime.com\/en\/","name":"TestingTime","description":"Wir rekrutieren Testpersonen","publisher":{"@id":"https:\/\/www.testingtime.com\/en\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.testingtime.com\/en\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.testingtime.com\/en\/#organization","name":"TestingTime","url":"https:\/\/www.testingtime.com\/en\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.testingtime.com\/en\/#\/schema\/logo\/image\/","url":"https:\/\/www.testingtime.com\/app\/uploads\/2017\/04\/logo.svg","contentUrl":"https:\/\/www.testingtime.com\/app\/uploads\/2017\/04\/logo.svg","width":1,"height":1,"caption":"TestingTime"},"image":{"@id":"https:\/\/www.testingtime.com\/en\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/testingtime","https:\/\/twitter.com\/testingtime","https:\/\/www.instagram.com\/testingtime\/","https:\/\/www.linkedin.com\/company-beta\/9231506\/","https:\/\/www.youtube.com\/channel\/UCpnMUgCz5FiiCUXU-U8ub1w"]},{"@type":"Person","@id":"https:\/\/www.testingtime.com\/en\/#\/schema\/person\/79095def7dada61a18ac99861db51e9c","name":"Reto Laemmler","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.testingtime.com\/en\/#\/schema\/person\/image\/","url":"https:\/\/www.testingtime.com\/app\/uploads\/2024\/03\/Image-2-130x130.png","contentUrl":"https:\/\/www.testingtime.com\/app\/uploads\/2024\/03\/Image-2-130x130.png","caption":"Reto Laemmler"},"description":"Reto ist Founder und CEO von TestingTime. Zuvor hat er als erster Mitarbeiter und VP Product Management bei Doodle, sein UX und Testing Handwerk gelernt. Test early, test often ist sein Mantra!","url":"https:\/\/www.testingtime.com\/en\/blog\/author\/reto-laemmler\/"}]}},"_links":{"self":[{"href":"https:\/\/www.testingtime.com\/en\/wp-json\/wp\/v2\/posts\/17779","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.testingtime.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.testingtime.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.testingtime.com\/en\/wp-json\/wp\/v2\/users\/5"}],"replies":[{"embeddable":true,"href":"https:\/\/www.testingtime.com\/en\/wp-json\/wp\/v2\/comments?post=17779"}],"version-history":[{"count":8,"href":"https:\/\/www.testingtime.com\/en\/wp-json\/wp\/v2\/posts\/17779\/revisions"}],"predecessor-version":[{"id":19163,"href":"https:\/\/www.testingtime.com\/en\/wp-json\/wp\/v2\/posts\/17779\/revisions\/19163"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.testingtime.com\/en\/wp-json\/wp\/v2\/media\/17523"}],"wp:attachment":[{"href":"https:\/\/www.testingtime.com\/en\/wp-json\/wp\/v2\/media?parent=17779"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.testingtime.com\/en\/wp-json\/wp\/v2\/categories?post=17779"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.testingtime.com\/en\/wp-json\/wp\/v2\/tags?post=17779"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}