[Find NNG - SearchGuard] Question Paper - Print access IP after ES installs Searchguard plug-in

Article Directory 1. Description of the problem 2. Solutions Project Recommendation ES Print Access IP 1. Description of the problem After the ES installs the searchguard plug-in, the client connects to ES, the password is entered incorrectly, and the ES log prompts password does not match, ...

Posted on Tue, 25 Feb 2020 17:24:03 -0800 by torleone

Kafka consumer multithreaded implementation

Kafka Producer, the producer of Kafka, is thread-safe, whereas consumer KafkaConsumer is non-thread-safe.KafkaConsumer defines an acquire() method to detect if only one thread is currently operating and throws a ConcurrentModifycationException exception if other threads are operating: java.util.Co ...

Posted on Thu, 06 Feb 2020 19:46:51 -0800 by HUWUWA

Windows Platform Integration SpringBoot+KAFKA_u Environment Configuration Section

Project needs, need to integrate SpringBoot+KAFKA I looked and found that on Linux, zoomkeeper was installed before KAFKA, such as https://blog.csdn.net/zhangcongyi420/article/details/88674491 My CA, my machine can't move, and if we want to do it, we need to do a good job of a few virtual machines to get a big data platform environment together ...

Posted on Mon, 06 Jan 2020 13:31:57 -0800 by elflacodepr

Install kafka and set up access over external network ip

1. kafka installation Install JDK tar xvf jdk1.8.0_231.tar.gz -C /usr/local && cd /usr/local ln -sv jdk1.8.0_231 jdk vim /etc/profile.d/java.sh JAVA_HOME=/usr/local/jdk PATH=$JAVA_HOME/bin:$PATH zookeeper installation (or using kafka's own) vim /usr/local/kafka/zookeeper/conf/zoo.cfg tickTime=2000 initLimit=10 ...

Posted on Sun, 05 Jan 2020 08:20:01 -0800 by LiamS94

Cluster building of kafka1.1.0

Kafka 1.1.0 cluster building (kafka_2.11-1.1.0) zookeeper-3.4.6 kafka_2.11-1.1.0 zookeeper-3.4.6 installation Extract the file zookeeper-3.4.6.tar.gz to the directory / data/zookeeper, and modify the configuration file zoo.cfg wget http://www.apache.org/dist/zookeeper/zookeeper-3.4.6/zookeeper-3.4.6.tar.gz tar -zxvf zook ...

Posted on Mon, 16 Dec 2019 13:38:54 -0800 by cyberlew15

5. Flnk--state state state management mechanism

1. Flnk State Principle 1.1 What is a state in flink?Why do I need state management? There are many intermediate processes when flink runs a calculation task.There are several temporary states in the process of running the whole task, such as some data is executing an operator, but only half of the data is processed. It is also a state that ha ...

Posted on Tue, 10 Dec 2019 20:34:37 -0800 by kb9yjg

Kafka old consumer API example

The main steps to realize low-level API are as follows: Get the metadata information of the corresponding partition of the specified topic Find out the Leader replica node of the specified partition, create a SimpleConsumer, and establish a connection with the Leader replica Construct consumption request Acq ...

Posted on Tue, 10 Dec 2019 15:52:16 -0800 by Phire

The method of analyzing Json format data by Flink SQL

1. Flink version 1.7.2 2. Introduce dependency Use maven to build the project, so pom.xml adds the following dependencies: <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-table_2.11</artifactId> <version>1.7.2</version> </dependency& ...

Posted on Thu, 05 Dec 2019 18:10:03 -0800 by cx3op

Using Docker to quickly build Zookeeper and kafka clusters

Cluster building Mirror selection Zookeeper and Kafka clusters run in different containers respectivelyOfficial image of zookeeper, version 3.4kafka adopts wurstmeister/kafka image Cluster planning hostname Ip addr port listener zoo1 172.19.0.11 2184:2181 zoo2 172.19.0.12 2185:2181 zoo3 172.19.0.13 2186:2181 kafka1 172.19.0.14 ...

Posted on Tue, 03 Dec 2019 16:50:57 -0800 by guymrodgers

Maintaining kafka offset to external media in spark streaming

Take kafka offset maintenance to redis as an example. redis storage format The data structure used is string, where key is topic:partition and value is offset. For example, there are three partitions under the topic of bobo, so the key value structure is as follows: The offset of bobo:0 is x The offset of bobo:1 is y The offset of bobo:2 ...

Posted on Mon, 02 Dec 2019 13:24:58 -0800 by rhunter007