ES Writer

An Elasticsearch writer component is designed to write the data stored in an Elasticsearch index. Elasticsearch writers typically authenticate with Elasticsearch using username and password credentials, which grant access to the Elasticsearch cluster and its indexes.

All component configurations are classified broadly into the following sections:

• ​​Basic Information​​

• Meta Information

• ​Resource Configuration​​

• ​Connection Validation​

Follow the given steps in the demonstration to configure the ES Writer component.

Configuring the ES Writer component​

Please follow the below mentioned steps to configure the meta information of ES writer:

• Host IP Address: Enter the host IP Address for Elastic Search.

• Port: Enter the port to connect with Elastic Search.

• Index ID: Enter the Index ID to read a document in elastic search. In Elasticsearch, an index is a collection of documents that share similar characteristics, and each document within an index has a unique identifier known as the index ID. The index ID is a unique string that is automatically generated by Elasticsearch and is used to identify and retrieve a specific document from the index.

• Mapping ID: Provide the Mapping ID. In Elasticsearch, a mapping ID is a unique identifier for a mapping definition that defines the schema of the documents in an index. It is used to differentiate between different types of data within an index and to control how Elasticsearch indexes and searches data.

• Resource Type: Provide the resource type. In Elasticsearch, a resource type is a way to group related documents together within an index. Resource types are defined at the time of index creation, and they provide a way to logically separate different types of documents that may be stored within the same index.

• Username: Enter the username for elastic search.

• Password: Enter the password for elastic search.

• Schema File Name: Upload spark schema file in JSON format.

• Save Mode: Select the Save mode from the drop down.

  • Append

• Selected columns: The user can select the specific column, provide some alias name and select the desired data type of that column.