Elastic Search Actual Warfare Series III: Elastic Search's JAVA API Use Tutorial

Preface

It was introduced in the last article. Elastic Search Actual Warfare Series II: Elastic Search DSL Statement Use Tutorial - Illustration This article explains the use of Elastic Search 6.x's official Java API.

ElasticSearch JAVA API

At present, there are several common Elastic Search Java API packages on the market, such as JestClient, SpringBook integrated Spring Data, Spring integrated Elastic Search Template, Elastic Search Bboss and other open source packages. These third-party integrated packages have basically supported daily use, except that the ES version will be lower.

This article introduces the use of the official Java High Level REST Client of Elastic Search. The Java High Level REST Client is currently recommended by Elastic Search. It is suitable for versions over 6.x. It requires JDK to be compatible with large versions, and the package itself contains methods in Java Low Level REST Client, which can deal with some special needs. It encapsulates Restful style for some commonly used methods. It can be used directly for operation name calls and supports synchronous and asynchronous (Async) calls.

Here, our use can also directly correspond to the DSL statement used in the previous article, so that it can be very convenient for comparison and learning to use.

To do the following, let's first look at Elastic search Java High Level REST Client's initial connection writing.

    RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(new HttpHost(elasticIp, elasticPort)));

Is it very simple? Closing is also very simple. client does not close directly for nothing.

I. Additional Data

Elastic Search can add data directly, as long as you specify index (index library name) and type (type). When adding, you can specify the primary key ID by yourself, or you can generate it by Elastic Search itself without specifying it. Elastic search Java High Level REST Client adds data in three ways, so let's take a look at these three ways.

New data code example 1, created through jsonString:

    String index = "test1";
    String type = "_doc";
    // Unique number
    String id = "1";
    IndexRequest request = new IndexRequest(index, type, id);

    String jsonString = "{" + "\"uid\":\"1234\","+ "\"phone\":\"12345678909\","+ "\"msgcode\":\"1\"," + "\"sendtime\":\"2019-03-14 01:57:04\","
            + "\"message\":\"xuwujing study Elasticsearch\"" + "}";
    request.source(jsonString, XContentType.JSON);
    IndexResponse indexResponse = client.index(request, RequestOptions.DEFAULT);

New data code example 2, created by map, will automatically convert to json data:

    String index = "test1";
    String type = "_doc";
    // Unique number
    String id = "1";
    IndexRequest request = new IndexRequest(index, type, id);
    Map<String, Object> jsonMap = new HashMap<>();
    jsonMap.put("uid", 1234);
    jsonMap.put("phone", 12345678909L);
    jsonMap.put("msgcode", 1);
    jsonMap.put("sendtime", "2019-03-14 01:57:04");
    jsonMap.put("message", "xuwujing study Elasticsearch");
    request.source(jsonMap);
    IndexResponse indexResponse = client.index(request, RequestOptions.DEFAULT);

New Data Code Sample 3, created by XContentBuilder object:

   String index = "test1";
    String type = "_doc";
    // Unique number
    String id = "1";
    IndexRequest request = new IndexRequest(index, type, id);
    XContentBuilder builder = XContentFactory.jsonBuilder();
    builder.startObject();
    {
        builder.field("uid", 1234);
        builder.field("phone", 12345678909L);
        builder.field("msgcode", 1);
        builder.timeField("sendtime", "2019-03-14 01:57:04");
        builder.field("message", "xuwujing study Elasticsearch");
    }
    builder.endObject();
    request.source(builder);
    IndexResponse indexResponse = client.index(request, RequestOptions.DEFAULT);

Among the three methods mentioned above, the second one is recommended by individuals, which is easy to understand and use.

II. Creating Index Library

In the above example, we created the index library by creating data directly, but it is not friendly to generate the index library by ES itself without creating the index library, because it will use default configuration, field structure is text(text data will participle, and extra space will be occupied when storing), fragmentation and index copy adopt default values, default is 5 and 1, ES. The number of fragments cannot be modified after creation unless reindex, so here we also specify the data template for creation.
There are three ways to create index libraries using JAVA API, just like the new data in the above, but here's just one.

Code examples of new index libraries:

private static void createIndex() throws IOException {
    String type = "_doc";
    String index = "test1";
    // setting value
    Map<String, Object> setmapping = new HashMap<>();
    // Partition number, replica number, cache refresh time
    setmapping.put("number_of_shards", 10);
    setmapping.put("number_of_replicas", 1);
    setmapping.put("refresh_interval", "5s");
    Map<String, Object> keyword = new HashMap<>();
    //Setting type
    keyword.put("type", "keyword");
    Map<String, Object> lon = new HashMap<>();
    //Setting type
    lon.put("type", "long");
    Map<String, Object> date = new HashMap<>();
    //Setting type
    date.put("type", "date");
    date.put("format", "yyyy-MM-dd HH:mm:ss");

    Map<String, Object> jsonMap2 = new HashMap<>();
    Map<String, Object> properties = new HashMap<>();
    //Setting field message information
    properties.put("uid", lon);
    properties.put("phone", lon);
    properties.put("msgcode", lon);
    properties.put("message", keyword);
    properties.put("sendtime", date);
    Map<String, Object> mapping = new HashMap<>();
    mapping.put("properties", properties);
    jsonMap2.put(type, mapping);

    GetIndexRequest getRequest = new GetIndexRequest();
    getRequest.indices(index);
    getRequest.local(false);
    getRequest.humanReadable(true);
    boolean exists2 = client.indices().exists(getRequest, RequestOptions.DEFAULT);
    //If it exists, it will not be created.
    if(exists2) {
        System.out.println(index+"Index libraries already exist!");
        return;
    }
    // Start creating Libraries
    CreateIndexRequest request = new CreateIndexRequest(index);
    try {
        // Loading data type
        request.settings(setmapping);
        //Setting mapping parameters
        request.mapping(type, jsonMap2);
        //Setting aliases
        request.alias(new Alias("pancm_alias"));
        CreateIndexResponse createIndexResponse = client.indices().create(request, RequestOptions.DEFAULT);
        boolean falg = createIndexResponse.isAcknowledged();
        if(falg){
            System.out.println("Creating Index Library:"+index+"Success!" );
        }
    } catch (IOException e) {
        e.printStackTrace();
    }

}

Note: When creating index libraries, we must first determine whether the index libraries exist!!!
Alias are also designated when creating index libraries here. This alias is a good thing. Proper use can improve query performance. We'll leave it here for the next time.

III. Modification of data

When ES provides API modification, there are two ways: one is to modify directly, but if the data does not exist, an exception will be thrown, and the other is to update and insert if it does not exist. Compared with the first one, the second one is more useful, but it is not as fast as the first one.

Sample code for ES modification:

private static void update() throws IOException {
    String type = "_doc";
    String index = "test1";
    // Unique number
    String id = "1";
    UpdateRequest upateRequest = new UpdateRequest();
    upateRequest.id(id);
    upateRequest.index(index);
    upateRequest.type(type);

    // You can still use a collection like Map as an update condition
    Map<String, Object> jsonMap = new HashMap<>();
    jsonMap.put("uid", 12345);
    jsonMap.put("phone", 123456789019L);
    jsonMap.put("msgcode", 2);
    jsonMap.put("sendtime", "2019-03-14 01:57:04");
    jsonMap.put("message", "xuwujing study Elasticsearch");
    upateRequest.doc(jsonMap);
    // The upsert method means that if the data does not exist, a new one is added.
    upateRequest.docAsUpsert(true);
    client.update(upateRequest, RequestOptions.DEFAULT);
    System.out.println("Update success!");

}

Note: The upsert method means that if the data does not exist, a new one is added, and the default is false.

IV. Deleting Data

According to the above several operations, I don't think it's necessary to say that we already know the DELETE method, so let's start directly.

ES deletes code examples based on ID:

private static void delete() throws IOException {

    String type = "_doc";
    String index = "test1";
    // Unique number
    String id = "1";
    DeleteRequest deleteRequest = new DeleteRequest();
    deleteRequest.id(id);
    deleteRequest.index(index);
    deleteRequest.type(type);
    // Setting timeout time
    deleteRequest.timeout(TimeValue.timeValueMinutes(2));
    // Set refresh policy "wait_for"
    // Keep the request open until refresh makes the content of the request searchable. This refresh strategy is compatible with high index and search throughput, but it causes requests to wait for a response until refresh occurs.
    deleteRequest.setRefreshPolicy(WriteRequest.RefreshPolicy.WAIT_UNTIL);
    // Synchronized deletion
    DeleteResponse deleteResponse = client.delete(deleteRequest, RequestOptions.DEFAULT);
}       

ES deletes according to conditions:

   private static void deleteByQuery() throws IOException {
    String type = "_doc";
    String index = "test1";
    DeleteByQueryRequest request = new DeleteByQueryRequest(index,type);
    // Setting Query Conditions
    request.setQuery(QueryBuilders.termsQuery("uid",1234));
    // Synchronized execution
    BulkByScrollResponse bulkResponse = client.deleteByQuery(request, RequestOptions.DEFAULT);
}

test result

Example diagram:

Query statement

Several commonly used query API s are briefly introduced here, and then all the query statement codes are given directly.

Query API

  • Equivalent (term query: QueryBuilders.termQuery(name,value);
  • Query Builders. termsQuery (name, value, value2, value3...);
  • Range query: QueryBuilders. RanQuery (name). GTE (value). LTE (value);
  • Existence query: Query Builders. existsQuery (name);
  • Wildcard query: QueryBuilders. wildcard Query (name, +value+);
  • bool query: BoolQuery Builder boolQuery Builder = new BoolQuery Builder ();

Query all code examples

 private static void allSearch() throws IOException {
    SearchRequest searchRequestAll = new SearchRequest();
    SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
    searchSourceBuilder.query(QueryBuilders.matchAllQuery());
    searchRequestAll.source(searchSourceBuilder);
    // Synchronized query
    SearchResponse searchResponseAll = client.search(searchRequestAll, RequestOptions.DEFAULT);
    System.out.println("Total number of queries:" + searchResponseAll.getHits().getTotalHits());
}

General Query Code Example

In fact, it is equivalent query, but it adds paging, sorting, timeout, routing and other settings, and adds some processing to the query results.

   private static void genSearch() throws IOException {
    String type = "_doc";
    String index = "test1";
    // Query the specified index library
    SearchRequest searchRequest = new SearchRequest(index);
    searchRequest.types(type);
    SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
    // Setting Query Conditions
    sourceBuilder.query(QueryBuilders.termQuery("uid", "1234"));
    // Setting Start and End
    sourceBuilder.from(0);
    sourceBuilder.size(5);
    sourceBuilder.timeout(new TimeValue(60, TimeUnit.SECONDS));
    // Setting up routing
//      searchRequest.routing("routing");
    // Setting Index Library Expressions
    searchRequest.indicesOptions(IndicesOptions.lenientExpandOpen());
    // Queries select local fragmentation, which defaults to cluster fragmentation
    searchRequest.preference("_local");

    // sort
    // Sort in descending order according to default values
//  sourceBuilder.sort(new ScoreSortBuilder().order(SortOrder.DESC));
    // Sort by field in ascending order
//  sourceBuilder.sort(new FieldSortBuilder("id").order(SortOrder.ASC));

    // Close the suorce query
//  sourceBuilder.fetchSource(false);

    String[] includeFields = new String[]{"title", "user", "innerObject.*"};
    String[] excludeFields = new String[]{"_type"};
    // Contains or excludes fields
//  sourceBuilder.fetchSource(includeFields, excludeFields);

    searchRequest.source(sourceBuilder);
    System.out.println("Ordinary query DSL Sentence:"+sourceBuilder.toString());
    // Synchronized query
    SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);

    // Whether the HTTP status code, execution time, or request terminated or timed out ahead of time
    RestStatus status = searchResponse.status();
    TimeValue took = searchResponse.getTook();
    Boolean terminatedEarly = searchResponse.isTerminatedEarly();
    boolean timedOut = searchResponse.isTimedOut();

    // For statistics on the total number of shards affected by the search, as well as the success and failure of the sharding
    int totalShards = searchResponse.getTotalShards();
    int successfulShards = searchResponse.getSuccessfulShards();
    int failedShards = searchResponse.getFailedShards();
    // Reasons for failure
    for (ShardSearchFailure failure : searchResponse.getShardFailures()) {
        // failures should be handled here
    }
    // Result
    searchResponse.getHits().forEach(hit -> {
        Map<String, Object> map = hit.getSourceAsMap();
        System.out.println("Results of general queries:" + map);
    });
    System.out.println("\n=================\n");
}

Or inquiry

In fact, this or query is also a bool query, where the query statement is equivalent to that in the SQL statement.

SELECT * FROM test1 where (uid = 1 or uid =2) and phone = 12345678919

Code example:

private static void orSearch() throws IOException {
    SearchRequest searchRequest = new SearchRequest();
    searchRequest.indices("test1");
    searchRequest.types("_doc");
    SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
    BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder();
    BoolQueryBuilder boolQueryBuilder2 = new BoolQueryBuilder();
 
      /**
     *  SELECT * FROM test1 where (uid = 1234 or uid =12345)  and phone = 12345678909
     * */
    boolQueryBuilder2.should(QueryBuilders.termQuery("uid", 1234));
    boolQueryBuilder2.should(QueryBuilders.termQuery("uid", 12345));
    boolQueryBuilder.must(boolQueryBuilder2);
    boolQueryBuilder.must(QueryBuilders.termQuery("phone", "12345678909"));
    searchSourceBuilder.query(boolQueryBuilder);
    System.out.println("Or query statement:" + searchSourceBuilder.toString());
    searchRequest.source(searchSourceBuilder);
    // Synchronized query
    SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);

    searchResponse.getHits().forEach(documentFields -> {

        System.out.println("Query results:" + documentFields.getSourceAsMap());
    });

}

Fuzzy Query

Equivalent to like queries in SQL statements.

private static void likeSearch() throws IOException {
    String type = "_doc";
    String index = "test1";
    SearchRequest searchRequest = new SearchRequest();
    searchRequest.indices(index);
    searchRequest.types(type);
    SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
    BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder();

   /**
     *  SELECT * FROM p_test where  message like '%xu%';
     * */
    boolQueryBuilder.must(QueryBuilders.wildcardQuery("message", "*xu*"));
    searchSourceBuilder.query(boolQueryBuilder);
    System.out.println("Fuzzy Query Statement:" + searchSourceBuilder.toString());
    searchRequest.source(searchSourceBuilder);
    // Synchronized query
    SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
    searchResponse.getHits().forEach(documentFields -> {
        System.out.println("Fuzzy query results:" + documentFields.getSourceAsMap());
    });
    System.out.println("\n=================\n");
}

Multivalued query

That is equivalent to in-query in the SQL statement.

     private static void inSearch() throws IOException {
        String type = "_doc";
        String index = "test1";
        // Query the specified index library
        SearchRequest searchRequest = new SearchRequest(index,type);
        SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
        /**
         *  SELECT * FROM p_test where uid in (1,2)
         * */
        // Setting Query Conditions
        sourceBuilder.query(QueryBuilders.termsQuery("uid", 1, 2));
        searchRequest.source(sourceBuilder);
        System.out.println("in Inquiry DSL Sentence:"+sourceBuilder.toString());
        // Synchronized query
        SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
        // Result
        searchResponse.getHits().forEach(hit -> {
            Map<String, Object> map = hit.getSourceAsMap();
            String string = hit.getSourceAsString();
            System.out.println("in Inquiry Map Result:" + map);
            System.out.println("in Inquiry String Result:" + string);
        });

        System.out.println("\n=================\n");
    }

Existence query

Determine whether the field exists, using something similar to exist in an SQL statement.

  private static void existSearch() throws IOException {
    String type = "_doc";
    String index = "test1";
    // Query the specified index library
    SearchRequest searchRequest = new SearchRequest(index);
    searchRequest.types(type);
    SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();

    // Setting Query Conditions
     sourceBuilder.query(QueryBuilders.existsQuery("msgcode"));
    searchRequest.source(sourceBuilder);
    System.out.println("Existing queries DSL Sentence:"+sourceBuilder.toString());
    // Synchronized query
    SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
    // Result
    searchResponse.getHits().forEach(hit -> {
        Map<String, Object> map = hit.getSourceAsMap();
        String string = hit.getSourceAsString();
        System.out.println("Existing queries Map Result:" + map);
        System.out.println("Existing queries String Result:" + string);
    });
    System.out.println("\n=================\n");
}

Scope query

As in the SQL statement <>, where gt is greater than, lt is less than, gte is greater than or equal to, and lte is less than or equal to.

private static void rangeSearch() throws IOException{
    String type = "_doc";
    String index = "test1";
    SearchRequest searchRequest = new SearchRequest(index);
    searchRequest.types(type);
    SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();

    // Setting Query Conditions
    sourceBuilder.query(QueryBuilders.rangeQuery("sendtime").gte("2019-01-01 00:00:00").lte("2019-12-31 23:59:59"));
    searchRequest.source(sourceBuilder);
     System.out.println("Scope query DSL Sentence:"+sourceBuilder.toString());
    // Synchronized query
    SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
    // Result
    searchResponse.getHits().forEach(hit -> {
        String string = hit.getSourceAsString();
        System.out.println("Scope query String Result:" + string);
    });
    System.out.println("\n=================\n");
}

Regular Query

ES can use regular query, and the query method is very simple. The code example is as follows:

 private static void regexpSearch() throws IOException{
    String type = "_doc";
    String index = "test1";
    // Query the specified index library
    SearchRequest searchRequest = new SearchRequest(index);
    searchRequest.types(type);
    SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
    // Setting Query Conditions
    sourceBuilder.query(QueryBuilders.regexpQuery("message","xu[0-9]"));
    searchRequest.source(sourceBuilder);
     System.out.println("Regular query DSL Sentence:"+sourceBuilder.toString());
    // Synchronized query
    SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
    // Result
    searchResponse.getHits().forEach(hit -> {
        Map<String, Object> map = hit.getSourceAsMap();
        String string = hit.getSourceAsString();
        System.out.println("Regular query Map Result:" + map);
        System.out.println("Regular query String Result:" + string);
    });

    System.out.println("\n=================\n");
}

Query test results

Total number of queries: 6
DSL statements for general queries: {from": 0,"size": 5,"timeout":"60s","query": {term": {uid": {value": {1234","boost": 1.0}}}}

=================

Query or query statements: {"query": {"bool": {"bool": {"must": [{"bool": {"should": [{````````` term": [{````term': {`````uid': {```value': 1234, `boost': 1.0}}}},{```````````````````````````````uid''{`````value'12345,12345, ````` boost': 1.0}}}}}}}}}}}}}}],``adjust clean_negative' adjust true true _pure"adjust_pure_negative": true,"boost": 1.0}}}
Or query results: {msgcode = 1, uid = 12345, phone = 12345678909, message = qq, sendtime = 2019-03-1401:57:04}

=================

Fuzzy Query Statement: {"query": {"bool": {"must": [{"wildcard": {"message": {"wildcard": {"xu", "boost": 1.0}}], "adjust_pure_negative": {true","boost": 1.0}}}}}
Fuzzy query results: {msgcode = 2, uid = 12345, phone = 123456789019, sendtime = 2019-03-1401:57:04, message = xuwujing study Elastic search}
Fuzzy query results: {uid = 123456, phone = 12345678909, message = xu1, sendtime = 2019-03-1401:57:04}

=================

DSL statements with queries: {"query":{"exists":{"field":"msgcode","boost":1.0}}}
Map results with queries: {msgcode = 2, uid = 12345, phone = 123456789019, sendtime = 2019-03-1401:57:04, message = xuwujing study Elastic search}
String results with queries: {"uid": 12345, "phone": 123456789019, "msgcode": 2, "sendtime": "2019-03-1401:57:04", "message": "xuwujing study Elastic search"}
Map results with queries: {msgcode = 1, uid = 12345, phone = 12345678909, message = qq, sendtime = 2019-03-1401:57:04}
The String result of the query: {"uid", "12345", "phone", "12345678909", "message", "qq", "msgcode", "1", "sendtime", "2019-03-1401:57:04"}

=================

DSL statements for range queries: {query": {range": {sendtime": {from": "2019-01-01:00:00", "to": "2019-12-31 23:59:59", "include_lower": true, "include_upper": true","boost": 1.0}}}}}}
String results for range queries: {"uid": 12345, "phone": 123456789019, "msgcode": 2, "sendtime": "2019-03-1401:57:04", "message": "xuwujing study Elastic search"}
String results for range queries: {"uid": "123456", "phone", "12345678909", "message", "xu1", "sendtime": "2019-03-1401:57:04"}
String results for range queries: {"uid": "12345", "phone", "12345678909", "message", "qq", "msgcode", "1", "sendtime", "2019-03-1401:57:04"}

=================

DSL statements for regular queries: {"query": {"regexp": {"message": {"value": "xu [0-9],"flags_value": 65535,"max_determinized_states": 10000,"boost": 1.0}}}}
Map result of regular query: {uid = 123456, phone = 12345678909, message = xu1, sendtime = 2019-03-1401:57:04}
The String result of regular query: {uid":"123456","phone","12345678909","message","xu1","sendtime":"2019-03-1401:57:04"}

=================

DLL statements for combined queries: {query": {"bool": {"must": [{"term": {"uid": {"value": 12345,"boost": 1.0}}}, {term": {"msgcode": {value": 1,"boost": 1.0}}], {adjust_pure_negative": {true, `boost": 1.0}}}}}} {
String results of combined queries: {uid":"12345","phone","12345678909","message","qq","msgcode","1","sendtime","2019-03-1401:57:04"}

=================

Other

Refer to ES official documents:
https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/java-rest-high.html

For the use of SpringBook integration Elastic Search and JestClient, see this article: SpringBoot integrates Elastic Search to achieve multi-version compatibility

With regard to the choice of Elastic Search Java API, JestClient is recommended if the Elastic Search version is prior to 6.x. If it is after 6.x and intends to upgrade to 7.x, then use the official Java High Level REST Client of ES directly, because after 7.x, the connection mode of Transport client will be discarded directly. At present, the ES integrated with Spring and SpringBook uses this mode (I wonder if it will be adjusted later).

The code for this article has been included in my own java-study If you are interested in the project, you are welcome to star, fork and issues.
Project address: https://github.com/xuwujing/java-study

Elastic Search Series:
Elastic Search Practical Series I: Elastic Search Cluster + Kinaba Installation Tutorial
Elastic Search Actual Warfare Series II: Elastic Search DSL Statement Use Tutorial - Illustration

Music Recommendation

Originality is not easy, if you feel good, I hope to give a recommendation! Your support is the greatest motivation for my writing!
Copyright Statement:
Author: Nothingness
Blog Garden Origin: http://www.cnblogs.com/xuwujing
CSDN origin: http://blog.csdn.net/qazwsxpcm    
Personal Blog Origin: http://www.panchengming.com

Tags: Java REST ElasticSearch SQL

Posted on Wed, 09 Oct 2019 18:52:19 -0700 by mjohnson025