Windows Platform Integration SpringBoot+KAFKA_u Environment Configuration Section

Project needs, need to integrate SpringBoot+KAFKA

I looked and found that on Linux, zoomkeeper was installed before KAFKA, such as https://blog.csdn.net/zhangcongyi420/article/details/88674491

My CA, my machine can't move, and if we want to do it, we need to do a good job of a few virtual machines to get a big data platform environment together. It's too troublesome. Just thinking about it, see below

https://www.jianshu.com/p/5da86afed228

Great. Try it

Downloaded zoopkeeper

apache-zookeeper-3.5.6-bin.tar.gz

Save Locally

D:\apache-zookeeper-3.5.6

Following the configuration instructions on the web, I configure it in my own zoomkeeper first

dataDir=D:\\apache-zookeeper-3.5.6\\data

Then execute zkServer

Report errors

C:\Users\Administrator>call  "-Dzookeeper.log.dir=D:\apache-zookeeper-3.5.6\bin\
..\logs" "-Dzookeeper.root.logger=INFO,CONSOLE" "-Dzookeeper.log.file=zookeeper-
Administrator-server-GAOLE.log" "-XX:+HeapDumpOnOutOfMemoryError" "-XX:OnOutOfMe
moryError=cmd /c taskkill /pid %%p /t /f" -cp "D:\apache-zookeeper-3.5.6\bin\..\
build\classes;D:\apache-zookeeper-3.5.6\bin\..\build\lib\*;D:\apache-zookeeper-3
.5.6\bin\..\*;D:\apache-zookeeper-3.5.6\bin\..\lib\*;D:\apache-zookeeper-3.5.6\b
in\..\conf" org.apache.zookeeper.server.quorum.QuorumPeerMain "D:\apache-zookeep
er-3.5.6\bin\..\conf\zoo.cfg"
//The file name, directory name, or volume label syntax is incorrect.

There's a JAVA_HOME error on it that caught my attention. Anyway, I guess this guy is looking for JAVA_HOME. First satisfy it, after all, the development environment installed on my machine has its own JDK. It doesn't matter who uses the pre-installed JDK.

After JAVA_HOME is set, then zkServer is executed, the visual inspection should have been done;

 

2020-01-03 17:10:56,545 [myid:] - INFO  [main:Environment@109] - Server environm
ent:java.io.tmpdir=C:\Users\ADMINI~1\AppData\Local\Temp\
2020-01-03 17:10:56,547 [myid:] - INFO  [main:Environment@109] - Server environm
ent:java.compiler=<NA>
2020-01-03 17:10:56,547 [myid:] - INFO  [main:Environment@109] - Server environm
ent:os.name=Windows 7
2020-01-03 17:10:56,548 [myid:] - INFO  [main:Environment@109] - Server environm
ent:os.arch=amd64
2020-01-03 17:10:56,548 [myid:] - INFO  [main:Environment@109] - Server environm
ent:os.version=6.1
2020-01-03 17:10:56,549 [myid:] - INFO  [main:Environment@109] - Server environm
ent:user.name=Administrator
2020-01-03 17:10:56,549 [myid:] - INFO  [main:Environment@109] - Server environm
ent:user.home=C:\Users\Administrator
2020-01-03 17:10:56,550 [myid:] - INFO  [main:Environment@109] - Server environm
ent:user.dir=C:\Users\Administrator
2020-01-03 17:10:56,550 [myid:] - INFO  [main:Environment@109] - Server environm
ent:os.memory.free=114MB
2020-01-03 17:10:56,551 [myid:] - INFO  [main:Environment@109] - Server environm
ent:os.memory.max=1801MB
2020-01-03 17:10:56,552 [myid:] - INFO  [main:Environment@109] - Server environm
ent:os.memory.total=123MB
2020-01-03 17:10:56,555 [myid:] - INFO  [main:ZooKeeperServer@938] - minSessionT
imeout set to 4000
2020-01-03 17:10:56,555 [myid:] - INFO  [main:ZooKeeperServer@947] - maxSessionT
imeout set to 40000
2020-01-03 17:10:56,557 [myid:] - INFO  [main:ZooKeeperServer@166] - Created ser
ver with tickTime 2000 minSessionTimeout 4000 maxSessionTimeout 40000 datadir D:
apache-zookeeper-3.5.6data\version-2 snapdir D:apache-zookeeper-3.5.6data\versio
n-2
2020-01-03 17:10:56,605 [myid:] - INFO  [main:Log@193] - Logging initialized @63
1ms to org.eclipse.jetty.util.log.Slf4jLog
2020-01-03 17:10:56,816 [myid:] - WARN  [main:ContextHandler@1588] - o.e.j.s.Ser
vletContextHandler@53b32d7{/,null,UNAVAILABLE} contextPath ends with /*
2020-01-03 17:10:56,817 [myid:] - WARN  [main:ContextHandler@1599] - Empty conte
xtPath
2020-01-03 17:10:56,834 [myid:] - INFO  [main:Server@370] - jetty-9.4.17.v201904
18; built: 2019-04-18T19:45:35.259Z; git: aa1c656c315c011c01e7b21aabb04066635b9f
67; jvm 1.8.0_60-b27
2020-01-03 17:10:56,900 [myid:] - INFO  [main:DefaultSessionIdManager@365] - Def
aultSessionIdManager workerName=node0
2020-01-03 17:10:56,901 [myid:] - INFO  [main:DefaultSessionIdManager@370] - No
SessionScavenger set, using defaults
2020-01-03 17:10:56,906 [myid:] - INFO  [main:HouseKeeper@149] - node0 Scavengin
g every 600000ms
2020-01-03 17:10:56,922 [myid:] - INFO  [main:ContextHandler@855] - Started o.e.
j.s.ServletContextHandler@53b32d7{/,null,AVAILABLE}
2020-01-03 17:10:56,959 [myid:] - INFO  [main:AbstractConnector@292] - Started S
erverConnector@3dd3bcd{HTTP/1.1,[http/1.1]}{0.0.0.0:8080}
2020-01-03 17:10:56,960 [myid:] - INFO  [main:Server@410] - Started @996ms
2020-01-03 17:10:56,960 [myid:] - INFO  [main:JettyAdminServer@112] - Started Ad
minServer on address 0.0.0.0, port 8080 and command URL /commands
2020-01-03 17:10:56,973 [myid:] - INFO  [main:ServerCnxnFactory@135] - Using org
.apache.zookeeper.server.NIOServerCnxnFactory as server connection factory
2020-01-03 17:10:56,988 [myid:] - INFO  [main:NIOServerCnxnFactory@673] - Config
uring NIO connection handler with 10s sessionless connection timeout, 1 selector
 thread(s), 8 worker threads, and 64 kB direct buffers.
2020-01-03 17:10:56,992 [myid:] - INFO  [main:NIOServerCnxnFactory@686] - bindin
g to port 0.0.0.0/0.0.0.0:2181
2020-01-03 17:10:57,012 [myid:] - INFO  [main:ZKDatabase@117] - zookeeper.snapsh
otSizeFactor = 0.33
2020-01-03 17:10:57,017 [myid:] - INFO  [main:FileTxnSnapLog@384] - Snapshotting
: 0x0 to D:apache-zookeeper-3.5.6data\version-2\snapshot.0
2020-01-03 17:10:57,036 [myid:] - INFO  [main:FileTxnSnapLog@384] - Snapshotting
: 0x0 to D:apache-zookeeper-3.5.6data\version-2\snapshot.0
2020-01-03 17:10:57,058 [myid:] - INFO  [main:ContainerManager@64] - Using check
IntervalMs=60000 maxPerMinute=10000

Weekend break, pause updates;

Come over on Monday and follow up with KAFKA

Start downloading a kafka_2.12-2.4.0 as described in this article

Then modify the configuration file, old habits, backup first, and then start modifying server.properties

The operation was successful according to the instructions on the internet; start kafka, everything is normal;

 with path version 1 (kafka.coordinator.transaction.ProducerIdManager)
[2020-01-06 10:49:10,027] INFO [TransactionCoordinator id=0] Starting up. (kafka
.coordinator.transaction.TransactionCoordinator)
[2020-01-06 10:49:10,031] INFO [Transaction Marker Channel Manager 0]: Starting
(kafka.coordinator.transaction.TransactionMarkerChannelManager)
[2020-01-06 10:49:10,032] INFO [TransactionCoordinator id=0] Startup complete. (
kafka.coordinator.transaction.TransactionCoordinator)
[2020-01-06 10:49:10,086] INFO [ExpirationReaper-0-AlterAcls]: Starting (kafka.s
erver.DelayedOperationPurgatory$ExpiredOperationReaper)
[2020-01-06 10:49:10,172] INFO [/config/changes-event-process-thread]: Starting
(kafka.common.ZkNodeChangeNotificationListener$ChangeEventProcessThread)
[2020-01-06 10:49:10,265] INFO [SocketServer brokerId=0] Started data-plane proc
essors for 1 acceptors (kafka.network.SocketServer)
[2020-01-06 10:49:10,280] INFO Kafka version: 2.4.0 (org.apache.kafka.common.uti
ls.AppInfoParser)
[2020-01-06 10:49:10,281] INFO Kafka commitId: 77a89fcf8d7fa018 (org.apache.kafk
a.common.utils.AppInfoParser)
[2020-01-06 10:49:10,282] INFO Kafka startTimeMs: 1578278950272 (org.apache.kafk
a.common.utils.AppInfoParser)
[2020-01-06 10:49:10,286] INFO [KafkaServer id=0] started (kafka.server.KafkaSer
ver)

 

Note: Do not close this window, make sure the ZooKeeper instance is ready and running before enabling Kafka

Set topic s, create producers, consumers, and test as described in the article

 

 

The test was successful and the installation and configuration of kafka was completed.

Tags: Java Apache kafka Zookeeper JDK

Posted on Mon, 06 Jan 2020 13:31:57 -0800 by elflacodepr