Control tasks

Control tasks contain correct responses. They are used for calculating the user's percentage of correct responses. Depending on the result, you can either change the user's skill value, or block access to tasks.

Add tasks with correct responses to the pool. Set key values in the quality_control.configs array in the pool settings.

The example below shows these settings:

  • Calculating the skill value equal to the percentage of correct responses in control tasks. You can use the skill value for filtering users.

  • Blocking a user from accessing the project if less than 75% of the responses are correct.

The first skill calculation is made after 7 control tasks have been completed. The calculation uses the last 10 responses from the project's golden set.

{
   "configs":[
      {
         "collector_config":{
            "type":"GOLDEN_SET",
            "parameters":{
               "history_size":10
            }
         },
            "rules":[
               {
                  "conditions":[
                     {
                        "key":"total_answers_count",
                        "operator":"GT",
                        "value":7
                     }
                  ],
                  "action":{
                     "type":"SET_SKILL_FROM_OUTPUT_FIELD",
                     "parameters":{
                        "skill_id":"42",
                        "from_field":"correct_answers_rate"
                     }
                  }
               },
               {
                  "conditions":[
                     {
                        "key":"total_answers_count",
                        "operator":"GT",
                        "value":7
                     },
                     {
                        "key":"correct_answers_rate",
                        "operator":"LT",
                        "value":75.0
                     }
                  ],
                  "action":{
                     "type":"RESTRICTION",
                     "parameters":{
                        "scope":"PROJECT",
                        "duration_days":1,
                        "private_comment":"Did not pass golden set"
                     }
                  }
               }
            ]
         }
      ]
   }

Key

Value

Data type

Required during creation

collector_сonfig.parameters.history_size

The maximum number of the user's recent responses in the project to use for calculating the percentage of correct responses.

If this field is omitted, the calculation is based on all the user's responses in the pool.

Integer

No

rules.conditions.key

Parameters that are checked in the condition:

  • total_answers_count — The number of completed control tasks.

  • correct_answers_rate — The percentage of correct responses in training and control tasks (from 0 to 100).

  • incorrect_answers_rate — The percentage of incorrect responses in training and control tasks (from 0 to 100).

  • golden_set_answers_count — The number of completed control tasks.
  • golden_set_correct_answers_rate — The percentage of correct responses in training and control tasks (from 0 to 100).
  • golden_set_incorrect_answers_rate — The percentage of incorrect responses in training and control tasks (from 0 to 100).

String

Yes