springboot+logback+logstash log collection

Intended for

  • Operation and maintenance personnel
  • Skilled in docker operation and people with basic knowledge of elk
  • Because this article involves a lot of knowledge, and does not describe it, so Xiaobai carefully looks

The operation steps are as follows

  1. Deploy logstash
  2. Introduce net.logstash.logback: logstash logback encoder package into springboot project
  3. Configure logback-spring.xml

Deploy logstash

The author uses docker compose to deploy logstash Create three files first

  • logstash.conf
  • logstash.yml
  • docker-compose.yml

The configuration is as follows:

# logstash.conf
input {
    tcp {
    # The author has enabled multiple logstash, so the default port of logstash is not used here. You can also configure it as you like
	port => 5045
	codec => "json_lines"
    }
}

output {
    elasticsearch {
    # Hosts = > IP: Port
	hosts => "elk:9200"
	# appname is configured in logback, and the prefix "logstash logback" can also be configured by the reader
	index => "logstash-logback-%{[appname]}-%{+YYYY.MM.dd}"
    }
}
# logstash.yml
http.host: "0.0.0.0"
xpack.monitoring.elasticsearch.hosts: [ "http://elasticsearch:9200" ]
# The first two behaviors are logstash default configuration. The author has not changed them, but added the third line to disable the x-pack monitoring function
xpack.monitoring.enabled: false
# docker-compose.yml
version: '3'

services:
  logstash-logback:
    image: docker.elastic.co/logstash/logstash:7.2.0
    volumes:
      - /root/software/docker/elk/logstash/logstash.yml:/usr/share/logstash/config/logstash.yml
      - /root/software/docker/elk/logstash/logstash.conf:/usr/share/logstash/pipeline/logstash.conf
    restart: always
    container_name: logstash-logback
    ports:
    # The port number here is the same as that configured in logstash.conf, format - > host port: container port
      - "5045:5045"
    networks:
    # Because the author's elastic search is also started in docker, in order to ensure that it can be accessed directly with the container name, the elastic search and logstash are placed in a bridge
      - elk
networks:
# First, the author creates a network bridge named elk, which is configured here to use the external bridge
  elk:
    external: true

Then start the container

Net.logstash.logback: logstash logback encoder package

The author uses gradle, the same with maven

compile('net.logstash.logback:logstash-logback-encoder:6.1')

Configure logback-spring.xml

Create logback-spring.xml file under resources folder

# logback-spring.xml
<?xml version="1.0" encoding="UTF-8"?>
<configuration><!--debug="true" scan="true" scanPeriod="30 seconds"-->
    <springProperty scope="context" name="logPath" source="log.path"/>

    <appender name="LOGSTASH" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
   		<!-- Configuration here logstash Of ip And port -->
        <destination>ip:5045</destination>
        <encoder charset="UTF-8" class="net.logstash.logback.encoder.LogstashEncoder">
        	<!-- Configuration here logstash Index name of,stay logstash.conf Using is configured in appname As part of index name -->
            <customFields>{"appname":"entry name"}</customFields>
        </encoder>
    </appender>

    <root level="info">
        <appender-ref ref="LOGSTASH"/>
    </root>
</configuration>

Start the project, and you can see the index in kibana

Tags: Programming Docker xml Spring ElasticSearch

Posted on Fri, 10 Jan 2020 11:01:26 -0800 by Xproterg^vi