Basic concepts

Task

A part of a web page that can contain various objects: images, text, input fields, and others. The Toloka user reads the instructions for the task, completes it and enters responses.

Project

Settings for creating tasks:

  • Parameters for objects that are shown on the task page (images, text, and so on).;

  • Parameters for input fields.

  • The visual appearance of tasks.

Pool

Sets of tasks that share the same properties (such as user selection settings and payment for a task suite) and are sent out for completion at the same time.

Overlap

The number of users performing the same task (overlap is used to make sure the results are reliable).

Task suite
Tasks placed on the same page. A suite may consist of a single task.
Training tasks

Tasks for training users.

Training tasks contain correct responses and hints. If a user responds incorrectly, a hint is displayed on the screen. The user must enter the correct response before continuing to the next task.

Control tasks

Tasks with correct responses. They are used for controlling the quality of responses.

Majority vote

A quality control approach that considers a response correct if it is chosen by the majority of users. Other responses are considered incorrect.

Skill

An assessment of the user's performance (a number from 0 to 100) that can change as tasks are completed. Skills are used for user selection in a pool.

Checking tasks

Manually checking the responses received by the requester. The requester accepts or rejects the responses. Rejected responses aren't paid for.