Kafka cluster construction and necessary knowledge and meeting

Kafka cluster deployment and startup In this article, we will start by demonstrating how to build a Kafka cluster, and then briefly introduce some basic knowledge points about Kafka cluster. However, this paper only introduces the cluster, and does not explain the basic concept of Kafka too much. It is assumed that the reader has some basic kn ...

Posted on Thu, 04 Jun 2020 18:21:26 -0700 by Stonewall

In depth interpretation of KafkaListener annotation of spring Kafka

brief introduction At present, Kafka is mainly used as a distributed publish subscribe message system, and it is also one of the most popular message queuing systems. Therefore, more and more frameworks integrate Kafka, such as spring Kafka, which will be mentioned in this article. Since Kafka is a message publishing and subscription system, ...

Posted on Sat, 30 May 2020 01:00:05 -0700 by foyer

Quick Start for Kafka - Introduction to Confluent Kafka

Quick Start for Kafka (8) - Introduction to Confluent Kafka 1. Introduction to Confluent Kafka 1. Introduction to Confluent Kafka In 2014, Jay Kreps, NahaNarkhede and Rao Jun, Kafka's founders, left LinkedIn to create Confluent, which focused on providing Kafka-based enterprise stream processing solutions and released Confluent Kafka.Confluent ...

Posted on Wed, 27 May 2020 09:17:52 -0700 by ranam

Detailed explanation of important concepts of kafka and key configuration of cluster

Key concepts broker A broker is an instance of kafka, which is responsible for receiving, forwarding and storing messages. kafka cluster is composed of multiple brokers. topic kafka's topic is a logical concept, which is to group and classify messages, so as to distinguish and process messages of different business logic. The index concepts in ...

Posted on Wed, 27 May 2020 04:56:38 -0700 by mustang

Kafka Quick Start-Kafka Monitoring

Kafka Quick Start (7) - Kafka Monitoring 1. Kafka Monitoring Indicators 1. Kafka host monitoring indicators Host monitoring monitors the performance of the node machine where the Kafka cluster Broker resides.Common host monitoring metrics include:(1) Machine Load(2) CPU utilization(3) Memory usage, including Free Memory and Used Memory(4) Disk ...

Posted on Mon, 25 May 2020 10:37:33 -0700 by meckr

PHP message queuing Kafka uses

Install Kafka service Go directly to kafka website and download the latest wget https://mirror.bit.edu.cn/apache/kafka/2.5.0/kafka_2.13-2.5.0.tgz    Unzip, enter directory tar -zxvf kafka_2.13-2.5.0.tgz cd kafka_2.13-2.5.0    Start Kafka service Starting a single node Zookeeper instance using the script in the installation package ...

Posted on Tue, 19 May 2020 07:36:32 -0700 by dmeade

The problem of Kafka service unavailable (down)

background An online log collection service alarms and opens the domain name to report the 502 error code. The collection service consists of two netty HA servers. The netty server parses the protobuf logs delivered by the client and sends them to kafka, opens the logs of one of the applications, and finds the following errors: org.apache.kaf ...

Posted on Thu, 07 May 2020 00:08:26 -0700 by JonathanS

Serverless actual combat: monitoring alarm is realized through serverless architecture

In actual production, we often need to do some monitoring scripts to monitor whether the website service or API service is available. The traditional method is to use the website monitoring platform (such as DNSPod monitoring, 360 website service monitoring, Alibaba cloud monitoring, etc.), whose principle is to set the service address to be mo ...

Posted on Tue, 21 Apr 2020 02:14:59 -0700 by l0ve2hat3

maxwell collects binlog s and sends data to kafka clusters via nginx in different network environments

maxwell can be used as a data synchronization tool by collecting binlog changes from mysql in real time. However, sometimes when the application is deployed in a different environment, changes in the mysql database cannot be directly sent to the data center via maxwell for parsing and data synchronization. This time, using ngix as a proxy serv ...

Posted on Thu, 09 Apr 2020 15:14:25 -0700 by palace

Talk about CanalAdapterService of canal

order This paper focuses on the CanalAdapterService of canal CanalAdapterService canal-1.1.4/client-adapter/launcher/src/main/java/com/alibaba/otter/canal/adapter/launcher/loader/CanalAdapterService.java @Component @RefreshScope public class CanalAdapterService { private static final Logger logger = LoggerFactory.getLogger(CanalAdapterSer ...

Posted on Sun, 05 Apr 2020 21:06:06 -0700 by dagee