Quick start

Imagine you have hundreds of pictures of cats, and you need to know whether the cat in each picture is happy or sad. It's rather difficult to automate this task with a computer, but you can get internet users to do it for you by creating a task in Toloka.

First you need to decide what the task will look like and how you will set it up in Toloka:

  • The task will show an image and a choice of responses: “happy”, “sad”, and “loading error” (if the image didn't load).

  • Since the task is in English, you need to select English-speaking users to complete the task.

  • To prevent users from guessing, add a restriction: no more than two “hasty responses”. A response is considered "too fast" (hasty) if it was chosen in 2 seconds or less.

Next you need to create tasks and check whether the settings are working correctly (make sure the task is displayed correctly and the filters and quality control settings are working the way you intended). Try performing some tasks and make sure that you can get the responses.

When your tasks are ready, publish them in the production version of Toloka. You can get the responses as users complete the tasks.

Create tasks

  1. Register in the sandbox and get an OAuth token in the requester interface: https://sandbox.toloka.yandex.com/requester/profile.

  2. Create a project. The project should include:

    • Input data parameters for the tasks (the input_spec key). In our example, the input data is the URL for accessing the cat images.

    • Response parameters (the output_spec key). The user's response is returned as a string.

    • Description of the task interface. The task contains an image sized 100x400 pixels (the {{img}} component) and response buttons (the {{field type=\"radio\"}} component). Next you can configure how Yandex.Toloka interface elements are displayed (the timer and task buttons). For more information about configuring the task interface, see Yandex.Toloka Requester's guide.

    In response to the request, you will receive a JSON object for the created project. This object contains the project ID in the id key.

    Request
    POST /api/v1/projects
    Host: https://sandbox.toloka.yandex.ru
    Authorization: OAuth <OAuth token>
    Content-Type: application/JSON
    
    {
       "public_name":"Is the cat happy or sad?",
       "public_description":"Look at the picture and decide which mood the cat is in.",
       "public_instructions":"Choose the answer that fits best",
       "task_spec":{
          "input_spec":{
             "image":{
                "type":"url",
                "required":true
             }
          },
          "output_spec":{
             "result":{
                "type":"string",
                "required":true
             }
          },
          "view_spec":{
             "assets":{
                "script_urls":[
                   "$TOLOKA_ASSETS/js/toloka-handlebars-templates.js"
                ]
             },
             "markup":"{{img src=image width=\"100%\" height=\"400px\"}}\n\n{{field type=\"radio\" name=\"result\" value=\"OK\" label=\"Happy\" hotkey=\"1\"}}\n{{field type=\"radio\" name=\"result\" value=\"BAD\" label=\"Sad\" hotkey=\"2\"}}\n{{field type=\"radio\" name=\"result\" value=\"404\" label=\"Loading error\" hotkey=\"3\"}}\n\n",
             "script":"exports.Task = extend(TolokaHandlebarsTask, function (options) {\n  TolokaHandlebarsTask.call(this, options);\n}, {\n  onRender: function() {\n    // Created the DOM element for the task (accessible via #getDOMElement()) \n  },\n  onDestroy: function() {\n    // The task is completed, so global resources can be released (if they were used)\n  }\n});\n\nfunction extend(ParentClass, constructorFunction, prototypeHash) {\n  constructorFunction = constructorFunction || function () {};\n  prototypeHash = prototypeHash || {};\n  if (ParentClass) {\n    constructorFunction.prototype = Object.create(ParentClass.prototype);\n  }\n  for (var i in prototypeHash) {\n    constructorFunction.prototype[i] = prototypeHash[i];\n  }\n  return constructorFunction;\n}\n",
             "styles":"",
             "settings":{
                "showSkip":true,
                "showTimer":true,
                "showTitle":true,
                "showFinish":true,
                "showSubmit":true,
                "showMessage":true,
                "showFullscreen":true,
                "showInstructions":true
             }
          }
       },
       "assignments_issuing_type":"AUTOMATED"
       
    }
  3. Create a pool with the task settings. In response to the request, you will receive a JSON object for the created pool. This object contains the pool ID in the id key.

    Request
    POST /api/v1/pools
    Host: https://sandbox.toloka.yandex.ru
    Authorization: OAuth <OAuth token>
    Content-Type: application/JSON
    
    {
       "project_id":"2405",
       "private_name":"Cat pool 1",
       "may_contain_adult_content":true,
       "will_expire":"2017-11-25T12:37:39",
       "reward_per_assignment":0.02,
       "assignment_max_duration_seconds":60,
       "mixer_config":{
          "real_tasks_count":5,
          "golden_tasks_count":0,
          "training_tasks_count":0
       },
       "defaults":{
          "default_overlap_for_new_task_suites":5
       }
    }
  4. Add filters to the pool for selecting users by language.

    Request
    PUT /api/v1/pools/14025
    Host: https://sandbox.toloka.yandex.ru
    Authorization: OAuth <OAuth token>
    Content-Type: application/JSON
    
    {
       "project_id":"2405",
       "private_name":"Cat pool 1",
       "may_contain_adult_content":true,
       "will_expire":"2017-11-25T12:37:39",
       "reward_per_assignment":0.02,
       "assignment_max_duration_seconds":60,
       "filter":{
          "and":[
            {
              "category":"profile",
              "key":"languages",
              "operator":"IN",
              "value":"RU"
            }
          ]
       },
       "mixer_config":{
          "real_tasks_count":5,
          "golden_tasks_count":0,
          "training_tasks_count":0
       },
       "defaults":{
          "default_overlap_for_new_task_suites":5
       }
    }
  5. Add a quality control rule (the quality control key). Choose quality control rules that are appropriate for the specific type of task. Monitoring hasty responses is useful for almost any type of task. The settings shown below will block access to the pool if the user responds too quickly on two tasks in a row.

    Request
    PUT /api/v1/pools/14025
    Host: https://sandbox.toloka.yandex.ru
    Authorization: OAuth <OAuth token>
    Content-Type: application/JSON
    
    {
       "project_id":"2405",
       "private_name":"Cat pool 1",
       "may_contain_adult_content":true,
       "will_expire":"2017-11-25T12:37:39",
       "reward_per_assignment":0.02,
       "assignment_max_duration_seconds":60,
       "filter":{
         "and":[
           {
             "category":"profile",
             "key":"languages",
             "operator":"IN",
             "value":"RU"
           }
          ]
       }, 
       "quality_control":{
          "configs":[
           {
             "collector_config":{
                "type":"ASSIGNMENT_SUBMIT_TIME",
                "parameters":{
                   "history_size":10,
                   "fast_submit_threshold_seconds":3
                }
             },
             "rules":[
                {
                   "conditions":[
                      {
                         "key":"total_submitted_count",
                         "operator":"EQ",
                         "value":2
                      },
                      {
                         "key":"fast_submitted_count",
                         "operator":"GTE",
                         "value":2
                      }
                   ],
                   "action":{
                      "type":"RESTRICTION",
                      "parameters":{
                         "scope":"PROJECT",
                         "duration_days":1,
                         "private_comment":"More than two fast responses"
                      }
                   }
                }
             ]
          }
          ]
       },
       "mixer_config":{
          "real_tasks_count":5,
          "golden_tasks_count":0,
          "training_tasks_count":0
       },
       "defaults":{
          "default_overlap_for_new_task_suites":5
       }
    }
  6. Upload the input data for tasks (image URLs).

    Request
    POST /api/v1/tasks
    Host: https://sandbox.toloka.yandex.ru
    Authorization: OAuth <OAuth token>
    Content-Type: application/JSON
    
    [
          {
             "input_values":{
                "image":"https://toloka.yandex.ru/api/proxy/photos/9.jpg"
             },
             "pool_id":"14025",
             "overlap":5
          },
          {
             "input_values":{
                "image":"https://toloka.yandex.ru/api/proxy/photos/8.jpg"
             },
             "pool_id":"14025",
             "overlap":5
          },
          {
             "input_values":{
                "image":"https://toloka.yandex.ru/api/proxy/photos/7.jpg"
             },
             "pool_id":"14025",
             "overlap":5
          },
          {
             "input_values":{
                "image":"https://toloka.yandex.ru/api/proxy/photos/6.jpg"
             },
             "pool_id":"14025",
             "overlap":5
          },
          {
             "input_values":{
                "image":"https://toloka.yandex.ru/api/proxy/photos/5.jpg"
             },
             "pool_id":"14025",
             "overlap":5
          }
      ]

    Now you can view the task on the pool page in sandbox. Check whether it is displayed correctly and start the pool.

    Request
    POST /api/v1/pools/14025/open
    Host: https://sandbox.toloka.yandex.ru
    Authorization: OAuth <OAuth token>

Check the tasks

After starting the pool, make sure that all your settings are working and the tasks are displayed correctly:

  1. Register as a user in the Yandex.Toloka sandbox as a user.

  2. Log in to the sandbox with the requester's username and follow the Add trusted users link on the Users page.

  3. Click the Add user button and enter the username of the created account.

  4. Perform the tasks in the sandbox under the username of a trusted user.

Get responses

When you have completed at least one task, you can get the responses:

Request
GET /api/v1/assignments?pool_id=14025
Host: https://sandbox.toloka.yandex.ru
Authorization: OAuth <OAuth token>

Responses are returned in the format:

{
      "id": "566ec2b0ff0deeaae5f9d4fe",
      "task_suite_id": "566ec2b0ff0deeaae5f9d4ff",
      "pool_id": "14025",
      "mixed": true,
      "user_id": "566ec2b0ff0deeaae5f9d500",
      "status": "ACCEPTED",
      "created": "2015-12-15T14:52:00",
      "submitted": "2015-12-15T15:10:00"
      "tasks": [
        {"id": "57a333ea-2827-4c1c-ab1f-8ab1bfe4ee7e",
         "input_values": {
            "image": "https://toloka.yandex.ru/api/proxy/photos/9.jpg"
          }
        }
      ],
      "solutions": [
        {
          "output_values": {
            "result": "OK"
          }
        }

When the project is set up to your satisfaction (tasks are displayed correctly and user filters and quality control rules are working correctly), you can move it to the production version of Toloka. You can do this in the requester's interface (see Yandex.Toloka requester's guide) or via the Yandex.Toloka API for the production version (send requests to https://toloka.yandex.ru):