Elasticsearch Bulk (insert/create/update/delete) API with Python client

Introduction – Elasticsearch Bulk API

The elasticsearch bulk API makes it possible to perform many insert/update/index/delete operations in a single API call. This can greatly increase the indexing speed.

Clients supports for bulk API requests
Some of the officially supported clients provide helpers to assist with bulk API requests. Python client is the most widely leading client to deals with elasticsearch operations. In our previous article, we discussed about python elasticsearch client and its installation. Now, In this article we will see how to use Bulk API helpers of that python client.

Elasticsearch Bulk API Endpoint
The REST API endpoint is /_bulk, and it expects the newline delimited JSON (NDJSON). The supported possible actions are are index, create, delete and update. For more detail explanation and information, please visit elasticsearch official document.

 

Elasticsearch Python Client  – Bulk Helpers API

Helpers is the collections of simple helpers function that abstract some specifics or the raw API.There are several helpers for the bulk API in python. All bulk helpers accept an instance of Elasticsearch class and an iterable actions The items in the action iterable should be the documents we wish to index in several formats.

The bulk() api accepts index, create, delete, and update actions. Use the _op_type field to specify an action (_op_type defaults to index):

{
    '_op_type': 'index',
    '_index': 'index-name',
    '_type': 'document',
    '_id': 111,
    '_source': {'text': 'elasticsearch bulk api with python'} # All Document json
}

 

Example Usage:

We have implemented a quick example that shows how to use elasticsearch bulk indexing with python client for importing Ansible Tower logs API data into elasticsearch. Please visit GitHub for full code.