deduplicate

Performs preprocessing on an array of keywords.

  1. Request
  2. Response
  3. Example

This method performs the following actions with keywords:

Unifying duplicate keywords

Joins together multiple keywords that differ only in the negative keywords, word form, or presence of stop words. The unified keyword contains the negative keywords from all the original keywords. For example, the keywords custom windows -cheap and custom window -blinds would be replaced with the single keyword phrase custom window -cheap -blinds. In addition, duplicate keywords are deleted when joining keywords.

Eliminate overlap

Adds a negative keyword if the keyword is contained in another keyword phrase that differs by just one word. An example is the keywords coat and fur coat. A negative keyword will be added to the first keyword, resulting in coat -fur. This means that a search query for fur coats will match only one of the keywords, not both.

The method accepts an input array that can contain:

  • New keywords without an ID.
  • Existing keywords that have an ID in Yandex.Direct or an external database.

As the result of unifying duplicates and differentiating overlapping keywords, the method returns instructions about which keywords to add, change, or delete. The response format is as close as possible to the request format for the add, update and delete methods in the Keywords service.

Request

Request structure in JSON format:

{
  "method": "deduplicate",
  "params": { /* params */
    "Keywords": [ {  /* DeduplicateRequestItem */
      "Id": (long),
      "Keyword": (string), /* required */
      "Weight": (long)
    }, ... ], /* required */
    "Operation": [( MERGE_DUPLICATES | ELIMINATE_OVERLAPPING ), ... ]
  } 
}

Parameters are described below.

Parameter Type Description Required
Params structure (for JSON) / DeduplicateRequest structure (for SOAP)
Keywords array of DeduplicateRequestItem Keywords to pre-preprocess. Maximum 400,000 items per array. Yes
Operation array of DeduplicateOperationEnum Operations to perform on an array of keywords:
  • MERGE_DUPLICATES — Unifying duplicate keywords.
  • ELIMINATE_OVERLAPPING — Differentiating overlapping keywords.

If this parameter is omitted, both operations are performed.

No
DeduplicateRequestItem structure
Id long The keyword ID in Yandex.Direct or in an external database. No
Keyword string

Keywords. Can contain negative keywords and operators.

The maximum length of a keyword is 4096 characters. The “!” operator before a negative keyword is not counted in the keyword length (the sequence “-!” counts as a single character).

Maximum of 7 words per keyword, not counting stop words and negative keywords. Each word can have up to 35 characters, not counting the minus sign before a negative keyword.

Yes
Weight long

Weight of the keyword. Positive integer.

If multiple keywords are joined together, the keyword with the highest weight is kept or is modified, and the others are deleted.

No
Parameter Type Description Required
Params structure (for JSON) / DeduplicateRequest structure (for SOAP)
Keywords array of DeduplicateRequestItem Keywords to pre-preprocess. Maximum 400,000 items per array. Yes
Operation array of DeduplicateOperationEnum Operations to perform on an array of keywords:
  • MERGE_DUPLICATES — Unifying duplicate keywords.
  • ELIMINATE_OVERLAPPING — Differentiating overlapping keywords.

If this parameter is omitted, both operations are performed.

No
DeduplicateRequestItem structure
Id long The keyword ID in Yandex.Direct or in an external database. No
Keyword string

Keywords. Can contain negative keywords and operators.

The maximum length of a keyword is 4096 characters. The “!” operator before a negative keyword is not counted in the keyword length (the sequence “-!” counts as a single character).

Maximum of 7 words per keyword, not counting stop words and negative keywords. Each word can have up to 35 characters, not counting the minus sign before a negative keyword.

Yes
Weight long

Weight of the keyword. Positive integer.

If multiple keywords are joined together, the keyword with the highest weight is kept or is modified, and the others are deleted.

No

Response

Response structure in JSON format:

{
  "result": { /* result */
    "Add": [{ /* DeduplicateResponseAddItem */
      "Keyword": (string) /* required */
    }, ... ],
    "Update": [{ /* DeduplicateResponseUpdateItem */
      "Id": (long), /* required */
      "Keyword": (string) /* required */
    }, ... ],
    "Delete": { /* IdsCriteria */
      "Ids": [(long), ... ] /* required */
    },
    "Failure": [{ /* DeduplicateErrorItem */
      "Position": (long), /* required */
      "Warnings": [{  /* ExceptionNotification */
        "Code": (int), /* required */
        "Message": (string), /* required */
        "Details": (string)
      }, ... ],
      "Errors": [{  /* ExceptionNotification */
        "Code": (int), /* required */
        "Message": (string), /* required */
        "Details": (string)
      }, ... ] 
    }, ... ]
  }
}

Parameters are described below.

Parameter Type Description
result structure (for JSON) / DeduplicateResponse structure (for SOAP)
Add array of DeduplicateResponseAddItem Keywords that can be input to the Keywords.add method.
Update array of DeduplicateResponseUpdateItem Keywords that can be input to the Keywords.update method.
Delete IdsCriteria Keywords that can be input to the Keywords.delete method.
Failure array of DeduplicateErrorItem Keywords that weren't processed due to an error.
DeduplicateResponseAddItem structure
Keyword string A keyword to add.
DeduplicateResponseUpdateItem structure
Id long Keyword ID from the request.
Keyword string Changed keyword.
IdsCriteria structure
Ids array of long IDs of keywords to delete.
DeduplicateErrorItem structure
Position long The sequential number of the keyword in the input array (starting from 1).
Warnings array of ExceptionNotification

Warnings that were issued when processing the keyword.

Errors array of ExceptionNotification

Errors that occurred when processing the keyword.

Parameter Type Description
result structure (for JSON) / DeduplicateResponse structure (for SOAP)
Add array of DeduplicateResponseAddItem Keywords that can be input to the Keywords.add method.
Update array of DeduplicateResponseUpdateItem Keywords that can be input to the Keywords.update method.
Delete IdsCriteria Keywords that can be input to the Keywords.delete method.
Failure array of DeduplicateErrorItem Keywords that weren't processed due to an error.
DeduplicateResponseAddItem structure
Keyword string A keyword to add.
DeduplicateResponseUpdateItem structure
Id long Keyword ID from the request.
Keyword string Changed keyword.
IdsCriteria structure
Ids array of long IDs of keywords to delete.
DeduplicateErrorItem structure
Position long The sequential number of the keyword in the input array (starting from 1).
Warnings array of ExceptionNotification

Warnings that were issued when processing the keyword.

Errors array of ExceptionNotification

Errors that occurred when processing the keyword.

Example

Request
{
  "method": "deduplicate",
  "params": {
    "Keywords": [{
      "Keyword": "A B -C"
    },
    {
      "Id": 1000,
      "Keyword": "A B -D"
    },
    {
      "Id": 1001,
      "Keyword": "A B"
    },
    {
      "Keyword": "E F"
    }]
  }
}
Response

The keywords A B -C, A B -D and A B are joined together into the combined keyword A B -C -D:

  • The keyword A B -C is new. It doesn't need to be added because the same one already exists in the database. It is skipped in the response.
  • The keyword A B -D exists in the database, so it needs to be replaced with the combined keyword. The keyword ID and new value are returned in the Update structure.
  • The A B keyword exists in the database and it needs to be deleted. The keyword ID is returned in the Delete structure.

The E F keyword is new and needs to be added. It is returned in the Add structure.

{
  "result": {
    "Add": [{
      "Keyword": "E F"
    }],
    "Update": [{
      "Id": 1000,
      "Keyword": "A B -C -D"
    }],
    "Delete": {
      "Ids": [ 1001 ]
    }
  }
}