Category: ElasticSearch

ElasticSearch: High Level Client Search Scrolling

This entry is part 4 of 4 in the series ElasticSearch High Level Rest Client

In this tutorial I will show you how to perform a search scroll using the high level client. If you have not already done so please follow the search tutorial.

The reason you following the search tutorial first is that sets up the search. So you just have to do a few more steps.

Imports:

  1. import org.elasticsearch.action.search.SearchScrollRequest;
  2. import org.elasticsearch.common.unit.TimeValue;

Modify the “SearchRequest”. A recommended timeout is 60000 or 1m.

  1. request.scroll(new TimeValue(60000));

Once you perform the initial search now you will get a “scrollId”. Use that to generate your new “SearchScrollRequest” using that scrollId. One thing to note is the “scrollRequest” timeout value. Set this or it may not work.

  1. final SearchScrollRequest searchScrollRequest = new SearchScrollRequest(scrollId);
  2. searchScrollRequest.scroll(new TimeValue(60000));

Now the searchResponse that we used initially we can repurpose to continue scrolling the results.

  1. searchResponse = client.searchScroll(searchScrollRequest);

We know that their are no more results when the scrollId is null or when getHits length is 0.

  1. searchResponse.getHits().getHits().length > 0

ElasticSearch: High Level Client Search

This entry is part 3 of 4 in the series ElasticSearch High Level Rest Client

In this tutorial I will show you how to perform a search using the high level client. If you have not already done so please connect to ElasticSearch.

Imports

  1. import org.elasticsearch.action.search.SearchRequest;
  2. import org.elasticsearch.search.builder.SearchSourceBuilder;
  3. import org.elasticsearch.action.search.SearchResponse;
  4. import org.elasticsearch.search.SearchHits;
  5. import org.elasticsearch.search.SearchHit;
  6. import org.elasticsearch.action.search.SearchType;

Now we can perform the search.

  1. final SearchRequest request = new SearchRequest();
  2. request.searchType(SearchType.QUERY_THEN_FETCH);
  3.  
  4. final String[] types = { "doc" };
  5. final String[] indexes = { "index" };
  6.  
  7. //Specify the types that your search applies to.
  8. //Note that this is not needed. If ommitted it will search all.
  9. request.types(types);
  10.  
  11. //Specify the indexes that your search applies to.
  12. //Note that this is not needed. If ommitted it will search all.
  13. request.indices(indexes);
  14.  
  15. final SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
  16. //You can add any type of query into this query. Adjust to what you need.
  17. searchSourceBuilder.query(MyQuery);
  18. request.source(searchSourceBuilder);
  19.  
  20. final SearchResponse searchResponse = client.search(request);
  21.  
  22. //This will let us know if the search was terminated early.
  23. final Boolean terminatedEarly = searchResponse.isTerminatedEarly();
  24. //This will let us know if it timed out.
  25. final boolean timedOut = searchResponse.isTimedOut();
  26.  
  27. //Now to loop through our hits to do what we need to
  28. final SearchHits searchHits = searchResponse.getHits();
  29. for (final SearchHit hit : searchHits) {
  30.   //Do work
  31. }

 

 

 

ElasticSearch: High Level Client Post

This entry is part 2 of 4 in the series ElasticSearch High Level Rest Client

In this tutorial I will show you how to perform a POST request. If you have not connected first please do so before continuing.

Imports

  1. import org.apache.http.HttpEntity;
  2. import org.apache.http.nio.entity.NStringEntity;
  3. import org.apache.http.entity.ContentType;
  4. import org.apache.http.util.EntityUtils;
  5. import org.elasticsearch.client.Response;

Now we can perform the POST to ElasticSearch.

  1. final Integer id = 1;
  2. final String document = "{\"key\": 1 }";
  3. final HttpEntity httpEntity = new NStringEntity(document, ContentType.APPLICATION_JSON);
  4.  
  5. final Response response = restHighLevelClient.getLowLevelClient().performRequest("POST", "/indexName/indexType/" + id, Collections.<String, String>emptyMap(), httpEntity);
  6.  
  7. //Now you can print the response
  8. System.out.println(EntityUtils.toString(response.getEntity()));

ElasticSearch: Low Level Client Get

This entry is part 3 of 3 in the series ElasticSearch Low Level Rest Client

In this tutorial I will show you how to put a json document into ElasticSearch. If you have not first connected to ElasticSearch please do so before continuing.

POM.xml

  1. <dependency>
  2.     <groupId>com.fasterxml.jackson.core</groupId>
  3.     <artifactId>jackson-databind</artifactId>
  4.     <version>2.9.5</version>
  5. </dependency>

Imports

  1. import org.apache.http.HttpEntity;
  2. import org.apache.http.nio.entity.NStringEntity;
  3. import org.apache.http.entity.ContentType;
  4. import org.elasticsearch.client.Response;
  5. import org.apache.http.util.EntityUtils;

Now perform the GET request using the low level client.

  1. ObjectMapper objectMapper = new ObjectMapper();
  2. final String document = "{\"key\": 1 }";
  3. final JsonNode document = objectMapper.readTree("{" +
  4.    " \"query\": {" +
  5.    " \"match\" : {" +
  6.    " \"key\" : 1 }}}");
  7. final HttpEntity httpEntity = new NStringEntity(document.toString(), ContentType.APPLICATION_JSON);
  8. final Response response = restClient.performRequest("GET", "/indexName/indexType/_search", Collections.<String, String>emptyMap(), httpEntity);
  9.  
  10. //Now you can print the response
  11. System.out.println(EntityUtils.toString(response.getEntity()));
  12.  
  13. //OR get the content
  14. final JsonNode content = objectMapper.readTree(response.getEntity().getContent());
  15. System.out.println(content);

ElasticSearch: Low Level Client Put

This entry is part 2 of 3 in the series ElasticSearch Low Level Rest Client

In this tutorial I will show you how to put a json document into ElasticSearch. If you have not first connected to ElasticSearch please do so before continuing.

Imports

  1. import org.apache.http.HttpEntity;
  2. import org.apache.http.nio.entity.NStringEntity;
  3. import org.elasticsearch.client.Response;
  4. import org.apache.http.entity.ContentType;
  5. import org.apache.http.util.EntityUtils;

Now perform the PUT request using the low level client.

  1. final String document = "{\"key\": 1 }";
  2. final HttpEntity httpEntity = new NStringEntity(document, ContentType.APPLICATION_JSON);
  3. final Integer id = 1;
  4. final Response response = restClient.performRequest("PUT", "/indexName/indexType/" + id, Collections.<String, String>emptyMap(), httpEntity);
  5.  
  6. //Now you can print the response
  7. System.out.println(EntityUtils.toString(response.getEntity()));

ElasticSearch: High Level Rest Client Connection

This entry is part 1 of 4 in the series ElasticSearch High Level Rest Client

In this tutorial I will show you how to use the ElasticSearch high level rest client.

First you will need to add the low level rest to the pom.

  1. <properties>
  2. 	<elasticSearch.version>6.2.4</elasticSearch.version>
  3. </properties>
  4.  
  5. <dependency>
  6.     <groupId>org.elasticsearch.client</groupId>
  7.     <artifactId>elasticsearch-rest-high-level-client</artifactId>
  8.     <version>${elasticSearch.version}</version>
  9. </dependency>

Next you will need to specify the imports.

  1. import java.util.List;
  2. import java.util.ArrayList;
  3. import java.util.Arrays;
  4. import org.apache.http.HttpHost;
  5. import org.elasticsearch.client.RestClient;
  6. import org.elasticsearch.client.RestClientBuilder;
  7. import org.elasticsearch.client.RestHighLevelClient;

Now you can connect to ElasticSearch.

  1. final List hosts = new ArrayList<>(Arrays.asList("localhost"));
  2. final Integer port = 9200;
  3. final String scheme = "http";
  4. 		
  5. final HttpHost[] httpHosts = hosts.stream().map(host -> new HttpHost(host, port, scheme)).toArray(HttpHost[]::new);
  6.  
  7. final RestClientBuilder restClientBuilder = RestClient.builder(httpHosts);
  8. final RestHighLevelClient restHighLevelClient = new RestHighLevelClient(restClientBuilder);

Now you can do whatever you need to!

ElasticSearch: Low Level Rest Client Connection

This entry is part 1 of 3 in the series ElasticSearch Low Level Rest Client

In this tutorial I will show you how to use the ElasticSearch low level rest client.

First you will need to add the low level rest to the pom.

  1. <properties>
  2. 	<elasticSearch.version>6.2.4</elasticSearch.version>
  3. </properties>
  4.  
  5. <dependency>
  6.     <groupId>org.elasticsearch.client</groupId>
  7.     <artifactId>elasticsearch-rest-client</artifactId>
  8.     <version>${elasticSearch.version}</version>
  9. </dependency>

Next you will need to specify the imports.

  1. import org.apache.http.HttpHost;
  2. import org.elasticsearch.client.Response;
  3. import org.elasticsearch.client.RestClient;
  4. import org.elasticsearch.client.RestClientBuilder;

Now you can connect to ElasticSearch.

  1. final RestClientBuilder restClientBuilder = RestClient.builder(new HttpHost("localhost", 9200, "http"));
  2.  
  3. final RestClient restClient = builder.build();

Now you can do whatever you need to!

ElasticSearch Installation

To install ElasticSearch is really straight forward. I will be using Ubuntu 16.04 for this installation.

Java 8

  1. java -version
  2. #if not installed run the following
  3. sudo apt-get install openjdk-8-jdk

Download

  1. wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.2.3.rpm

Directories

It is recommended to change the log and data directory from default implementations.

  1. #create log and data directory
  2. sudo mkdir /my/dir/log/elasticsearch
  3. sudo mkdir /my/dir/elasticsearch
  4.  
  5. # Change owner
  6. sudo chown -R elasticsearch /my/dir/log/elasticsearch
  7. sudo chown -R elasticsearch /my/dir/elasticsearch

Install

  1. sudo rpm -ivh elasticsearch-6.2.3.rpm

Change Settings

  1. sudo vi /etc/elasticsearch/elasticsearch.yml
  2.  
  3. #Change the following settings
  4. #----------SETTINGS-----------------
  5. cluster.name: logsearch
  6. node.name: ##THE_HOST_NAME##
  7. node.master: true #The node is master eligable
  8. node.data: true #Hold data and perform data related operations
  9. path.data: /my/dir/elasticsearch
  10. path.logs: /my/dir/log/elasticsearch
  11. network.host: ##THE_HOST_NAME##
  12. http.port: 9200
  13. discovery.zen.ping.unicast.hosts: ["##THE_HOST_NAME##"]
  14. #----------SETTINGS-----------------

Start/Stop/Status ElasticSearch

  1. sudo service elasticsearch start
  2. sudo service elasticsearch stop
  3. sudo service elasticsearch status

Rest API

http://localhost:9200