Elastic search source code analysis - startup process

The elastic search version is 5.5.0. The following is the main core process, ignoring the exception compensation part

Whole process

Preview the starting process of elastic search in the way of code flow

  1. elasticsearch.main(args, terminal)
  2. mainWithoutErrorHandling(args, terminal);
  3. execute(terminal, options)
  4. execute(terminal, options, createEnv(terminal, settings))
  5. init(daemonize, pidFile, quiet, env)
  6. Bootstrap.init(!daemonize, pidFile, quiet, initialEnv)
  7. INSTANCE = new Bootstrap();
  8. INSTANCE.setup(true, environment);
  9. INSTANCE.start();

1. Configure default settings, such as path.conf,path.data,path.home,path.logs. These parameters can be set in VM options

putSystemPropertyIfSettingIsMissing(settings, "path.conf", "es.path.conf");
putSystemPropertyIfSettingIsMissing(settings, "path.data", "es.path.data");
putSystemPropertyIfSettingIsMissing(settings, "path.home", "es.path.home");
putSystemPropertyIfSettingIsMissing(settings, "path.logs", "es.path.logs");

2. Initialize Elasticsearch. The following code is not under the same class or package

1.initLoggerPrefix(); TODO: why? is it just a bad default somewhere? or is it some BS around 'but the client' garbage <-- my guess
     2.INSTANCE = new Bootstrap();
     3.final SecureSettings keystore = loadSecureSettings(initialEnv);
     4.Environment environment = createEnvironment(foreground, pidFile, keystore, initialEnv.settings());
     5.LogConfigurator.configure(environment);
     6.checkForCustomConfFile();
     7.checkConfigExtension(environment.configExtension());
     8.PidFile.create(environment.pidFile(), true);
     9.checkLucene();
     10.node = new Node(environment)  ##Core divider,The above steps are all preparation for the node creation##
     11.nodeEnvironment = new NodeEnvironment(tmpSettings, environment);
     12.final JvmInfo jvmInfo = JvmInfo.jvmInfo();
     13.this.pluginsService = new PluginsService(tmpSettings, environment.modulesFile(), environment.pluginsFile(), classpathPlugins);
     14. for (final ExecutorBuilder<?> builder : threadPool.builders()) {
                        additionalSettings.addAll(builder.getRegisteredSettings());
                    }
     15. client = new NodeClient(settings, threadPool);
     16. ModulesBuilder modules = new ModulesBuilder(); modules.add(clusterModule);modules.add(indicesModule);....modules.add(new RepositoriesModule(this.environment, pluginsService.filterPlugins(RepositoryPlugin.class), xContentRegistry));
     17. final RestController restController = actionModule.getRestController();
     18. modules.add(b -> {
                             b.bind(NodeService.class).toInstance(nodeService);
                             b.bind(NamedXContentRegistry.class).toInstance(xContentRegistry);
                             b.bind(PluginsService.class).toInstance(pluginsService);
                             b.bind(Client.class).toInstance(client);
                             b.bind(NodeClient.class).toInstance(client);
                             b.bind(Environment.class).toInstance(this.environment);
                             b.bind(ThreadPool.class).toInstance(threadPool);
                             b.bind(NodeEnvironment.class).toInstance(nodeEnvironment);
                             b.bind(TribeService.class).toInstance(tribeService);
                             b.bind(ResourceWatcherService.class).toInstance(resourceWatcherService);
                             b.bind(CircuitBreakerService.class).toInstance(circuitBreakerService);
                             b.bind(BigArrays.class).toInstance(bigArrays);
                             b.bind(ScriptService.class).toInstance(scriptModule.getScriptService());
                             b.bind(AnalysisRegistry.class).toInstance(analysisModule.getAnalysisRegistry());
                             b.bind(IngestService.class).toInstance(ingestService);
                             b.bind(NamedWriteableRegistry.class).toInstance(namedWriteableRegistry);
                             b.bind(MetaDataUpgrader.class).toInstance(metaDataUpgrader);
                             b.bind(MetaStateService.class).toInstance(metaStateService);
                             b.bind(IndicesService.class).toInstance(indicesService);
                             b.bind(SearchService.class).toInstance(newSearchService(clusterService, indicesService,
                                 threadPool, scriptModule.getScriptService(), bigArrays, searchModule.getFetchPhase()));
                             b.bind(SearchTransportService.class).toInstance(searchTransportService);
                             b.bind(SearchPhaseController.class).toInstance(new SearchPhaseController(settings, bigArrays,
                                     scriptModule.getScriptService()));
                             b.bind(Transport.class).toInstance(transport);
                             b.bind(TransportService.class).toInstance(transportService);
                             b.bind(NetworkService.class).toInstance(networkService);
                             b.bind(UpdateHelper.class).toInstance(new UpdateHelper(settings, scriptModule.getScriptService()));
                             b.bind(MetaDataIndexUpgradeService.class).toInstance(new MetaDataIndexUpgradeService(settings, xContentRegistry,
                                 indicesModule.getMapperRegistry(), settingsModule.getIndexScopedSettings(), indexMetaDataUpgraders));
                             b.bind(ClusterInfoService.class).toInstance(clusterInfoService);
                             b.bind(Discovery.class).toInstance(discoveryModule.getDiscovery());
                             {
                                 RecoverySettings recoverySettings = new RecoverySettings(settings, settingsModule.getClusterSettings());
                                 processRecoverySettings(settingsModule.getClusterSettings(), recoverySettings);
                                 b.bind(PeerRecoverySourceService.class).toInstance(new PeerRecoverySourceService(settings, transportService,
                                         indicesService, recoverySettings, clusterService));
                                 b.bind(PeerRecoveryTargetService.class).toInstance(new PeerRecoveryTargetService(settings, threadPool,
                                         transportService, recoverySettings, clusterService));
                             }
                             httpBind.accept(b);
                             pluginComponents.stream().forEach(p -> b.bind((Class) p.getClass()).toInstance(p));
                         }
                     );
                     19. injector = modules.createInjector();
                     20. logger.info("initialized");
                     21. INSTANCE.start();Officially launched Elasticsearch node
                     22. injector.getInstance(MappingUpdatedAction.class).setClient(client);
                                 injector.getInstance(IndicesService.class).start();
                                 injector.getInstance(IndicesClusterStateService.class).start();
                                 injector.getInstance(IndicesTTLService.class).start();
                                 injector.getInstance(SnapshotsService.class).start();
                                 injector.getInstance(SnapshotShardsService.class).start();
                                 injector.getInstance(RoutingService.class).start();
                                 injector.getInstance(SearchService.class).start();
                                 injector.getInstance(MonitorService.class).start();
                      23.  tribeService.startNodes();

The initialization process is as follows:

  1. Initialize the log prefix. As for why, later developers of elastic search gave a guess and used a TODO to guess the part. We ignore it here
  2. Create a Bootstrap instance, register a closed hook internally and use a non daemons thread to ensure that only one Bootstrap instance starts.
  3. If the security module is registered, the relevant configuration will be loaded in
  4. Create the necessary environment and related configurations for Elasticsearch to run, such as loading the configuration directories such as config,scripts,plugins,modules,logs,lib,bin into the running environment
  5. Configure log correlation, create log context
  6. Check the custom configuration files, such as es.default.config,elasticsearch.config,es.config, etc
  7. Check the extension of the configuration file. If the suffix of the configuration file is yaml or json, Elasticsearch will prompt you to use yml for configuration
  8. Create PID file
  9. Check Lucene version
  10. Use the current environment to create a node, the core partition line. The above steps are to prepare for the creation of the node. The subsequent processes can be basically reflected in the Elasticsearch startup log
  11. Create a node environment, such as node name, node ID, partition information, storage element, and allocate memory for the node to use
  12. Print out JVM related information
  13. Use pluginservice to load the corresponding modules and plug-ins. Because no other plug-ins are installed in the elasticsearch 5.5.0 installation directory I downloaded, only the relevant modules under the modules directory will be loaded, such as aggs matrix stats, ingest common, Lang expression, Lang groovy, etc. which modules can be checked under the modules directory
  14. Load some additional configuration parameters
  15. Create a node client (core place)
  16. Cache a series of modules, such as nodemodule, clustermodule, industriesmodule, actionmodule, gatewaymodule, settingsmodule, repositoriesmodule
  17. Get the RestController, which is used to process the rest commands of various Elasticsearch, such as "cat", "all", "cat / health", "clusters", etc. (Elasticsearch is called action)
  18. Binding deals with instances of various services, which is the core of elastic search
  19. Using Guice to inject various modules and services (xxxService) into the elastic search environment
  20. Initialization complete
  21. Officially launch the Elasticsearch node
  22. Use Guice to obtain various wood blocks and services registered above
  23. Add the current node to a cluster and start the current node

The related startup logs from step 10 are as follows:

[2017-07-11T22:50:24,010][INFO ][o.e.n.Node               ] [] initializing ...
[2017-07-11T22:50:24,104][INFO ][o.e.e.NodeEnvironment    ] [iobPZcg] using [1] data paths, mounts [[/ (/dev/disk1)]], net usable_space [133.4gb], net total_space [232.6gb], spins? [unknown], types [hfs]
[2017-07-11T22:50:24,104][INFO ][o.e.e.NodeEnvironment    ] [iobPZcg] heap size [3.5gb], compressed ordinary object pointers [true]
[2017-07-11T22:50:24,120][INFO ][o.e.n.Node               ] node name [iobPZcg] derived from node ID [iobPZcgEQBKodmURFuo_Gw]; set [node.name] to override
[2017-07-11T22:50:24,121][INFO ][o.e.n.Node               ] version[5.5.0-SNAPSHOT], pid[66342], build[Unknown/Unknown], OS[Mac OS X/10.12/x86_64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_101/25.101-b13]
[2017-07-11T22:50:24,121][INFO ][o.e.n.Node               ] JVM arguments [-agentlib:jdwp=transport=dt_socket,address=127.0.0.1:50649,suspend=y,server=n, -Des.path.home=/Users/admin/elk/elasticsearch-5.5.0, -Dlog4j2.disable.jmx=true, -javaagent:/Users/admin/Library/Caches/IntelliJIdea2017.1/groovyHotSwap/gragent.jar, -Dfile.encoding=UTF-8]
[2017-07-11T22:50:24,121][WARN ][o.e.n.Node               ] version [5.5.0-SNAPSHOT] is a pre-release version of Elasticsearch and is not suitable for production
[2017-07-11T22:50:24,781][INFO ][o.e.p.PluginsService     ] [iobPZcg] loaded module [aggs-matrix-stats]
[2017-07-11T22:50:24,781][INFO ][o.e.p.PluginsService     ] [iobPZcg] loaded module [ingest-common]
[2017-07-11T22:50:24,782][INFO ][o.e.p.PluginsService     ] [iobPZcg] loaded module [lang-expression]
[2017-07-11T22:50:24,782][INFO ][o.e.p.PluginsService     ] [iobPZcg] loaded module [lang-groovy]
[2017-07-11T22:50:24,782][INFO ][o.e.p.PluginsService     ] [iobPZcg] loaded module [lang-mustache]
[2017-07-11T22:50:24,782][INFO ][o.e.p.PluginsService     ] [iobPZcg] loaded module [lang-painless]
[2017-07-11T22:50:24,782][INFO ][o.e.p.PluginsService     ] [iobPZcg] loaded module [parent-join]
[2017-07-11T22:50:24,782][INFO ][o.e.p.PluginsService     ] [iobPZcg] loaded module [percolator]
[2017-07-11T22:50:24,782][INFO ][o.e.p.PluginsService     ] [iobPZcg] loaded module [reindex]
[2017-07-11T22:50:24,782][INFO ][o.e.p.PluginsService     ] [iobPZcg] loaded module [transport-netty3]
[2017-07-11T22:50:24,782][INFO ][o.e.p.PluginsService     ] [iobPZcg] loaded module [transport-netty4]
[2017-07-11T22:50:24,783][INFO ][o.e.p.PluginsService     ] [iobPZcg] no plugins loaded
[2017-07-11T22:50:26,679][INFO ][o.e.d.DiscoveryModule    ] [iobPZcg] using discovery type [zen]
[2017-07-11T22:50:27,230][INFO ][o.e.n.Node               ] initialized
[2017-07-11T22:50:27,231][INFO ][o.e.n.Node               ] [iobPZcg] starting ...
[2017-07-11T22:50:27,278][INFO ][i.n.u.i.PlatformDependent] Your platform does not provide complete low-level API for accessing direct buffers reliably. Unless explicitly requested, heap buffer will always be preferred to avoid potential system instability.
[2017-07-11T22:50:27,459][INFO ][o.e.t.TransportService   ] [iobPZcg] publish_address {127.0.0.1:9300}, bound_addresses {[fe80::1]:9300}, {[::1]:9300}, {127.0.0.1:9300}
[2017-07-11T22:50:27,477][WARN ][o.e.b.BootstrapChecks    ] [iobPZcg] initial heap size [268435456] not equal to maximum heap size [4294967296]; this can cause resize pauses and prevents mlockall from locking the entire heap
[2017-07-11T22:50:30,554][INFO ][o.e.c.s.ClusterService   ] [iobPZcg] new_master {iobPZcg}{iobPZcgEQBKodmURFuo_Gw}{Uxli57L-S--Xi5sQtZJlpg}{127.0.0.1}{127.0.0.1:9300}, reason: zen-disco-elected-as-master ([0] nodes joined)
[2017-07-11T22:50:30,599][INFO ][o.e.h.n.Netty4HttpServerTransport] [iobPZcg] publish_address {127.0.0.1:9200}, bound_addresses {[fe80::1]:9200}, {[::1]:9200}, {127.0.0.1:9200}
[2017-07-11T22:50:30,599][INFO ][o.e.n.Node               ] [iobPZcg] started
[2017-07-11T22:50:30,891][INFO ][o.e.g.GatewayService     ] [iobPZcg] recovered [2] indices into cluster_state
[2017-07-11T22:50:31,141][INFO ][o.e.c.r.a.AllocationService] [iobPZcg] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[twitter][4], [twitter][3]] ...]).

3. The general situation of the whole initialization process and startup process is as above. The detailed internal process and method will be analyzed in detail later

 

Welcome to QQ group: 639315446 , discuss the relevant technologies of elastic search together

Published 5 original articles, won praise 2, visited 40000+
Private letter follow

Tags: ElasticSearch jvm snapshot JSON

Posted on Sat, 07 Mar 2020 21:23:53 -0800 by Michael 01