Complete interpretation of Flutter compiling and packaging system

The first thing a Fletter developer needs to know is how to compile and run a Fletter application. Different from the usual compilation of Android engineering projects, the package compilation of flutter is realized by calling the command line of flutter.

In the process of compiling and running over and over again, you may often think about: what has been done behind every fluent command? How does the compilation of flutter connect with the traditional Android gradle compilation process? How can Dart code be compiled into executable code?

We will reveal the mystery behind it.

flutter build apk

In general, for a standard Flutter project, you can complete the packaging by executing the following command,

flutter build apk

The default attribute here is -- release, so the release package will be printed by default. Of course, if you need to type a debug package, you can do this:

flutter build apk --debug

First, let's see what the flitter command is.

The flutter ontology, under the bin of the Flutter SDK directory, that is, / path to Flutter SDK / flutter / bin / flutter, is a command-line script, the core of which is this line:

"$DART" $FLUTTER_TOOL_ARGS "$SNAPSHOT_PATH" "$@"

Among them, the specific meaning of each parameter is as follows:

  • $Dart: Dart executable to start a Dart virtual machine.
  • $shuttle tool args: it is used by Google to debug the shuttle SDK. It is usually empty.
  • $SNAPSHOT_PATH: specify a snapshot file for running, here is the flitter / bin / cache / flitter_tools.snapshot. Snapshot means the jar file in java.
  • $@: it will pass through the parameters passed in by the user. This is build apk.

Therefore, the following commands are actually executed here:

flutter/bin/cache/dart-sdk/bin/dart FLUTTER_TOOL_ARGS= SNAPSHOT_PATH=flutter/bin/cache/flutter_tools.snapshot build apk

It can be seen that this command is actually the same as java's way of executing jar files. However, the logic of compiling and packaging here is implemented in flitter? Tools.snapshot using Dart language, so Dart's execution environment is adopted.

flutter_tool

Dart's snapshot is equivalent to Java's jar. Therefore, when a snapshot is executed, it must have its execution entry, which is similar to the main function.

The entrance here is exactly flutter / packages / flutter ﹣ tools / bin / flutter ﹣ tools.dart

You can see that its main function is as follows:

void main(List<String> args) {
  executable.main(args);
}

It calls the main function of flitter / packages / flitter? Tools / lib / executable.dart. Let's go on to see:

/// Main entry point for commands.
///
/// This function is intended to be used from the `flutter` command line tool.
Future<void> main(List<String> args) async {
  final bool verbose = args.contains('-v') || args.contains('--verbose');

  final bool doctor = (args.isNotEmpty && args.first == 'doctor') ||
      (args.length == 2 && verbose && args.last == 'doctor');
  final bool help = args.contains('-h') || args.contains('--help') ||
      (args.isNotEmpty && args.first == 'help') || (args.length == 1 && verbose);
  final bool muteCommandLogging = help || doctor;
  final bool verboseHelp = help && verbose;

  await runner.run(args, <FlutterCommand>[
    AnalyzeCommand(verboseHelp: verboseHelp),
    AttachCommand(verboseHelp: verboseHelp),
    BuildCommand(verboseHelp: verboseHelp), // Corresponding to shuttle build apk
    ChannelCommand(verboseHelp: verboseHelp),
    CleanCommand(),
    ConfigCommand(verboseHelp: verboseHelp),
    CreateCommand(),
    DaemonCommand(hidden: !verboseHelp),
    DevicesCommand(),
    DoctorCommand(verbose: verbose),
    DriveCommand(),
    EmulatorsCommand(),
    FormatCommand(),
    IdeConfigCommand(hidden: !verboseHelp),
    InjectPluginsCommand(hidden: !verboseHelp),
    InstallCommand(),
    LogsCommand(),
    MakeHostAppEditableCommand(),
    PackagesCommand(),
    PrecacheCommand(),
    RunCommand(verboseHelp: verboseHelp),
    ScreenshotCommand(),
    ShellCompletionCommand(),
    StopCommand(),
    TestCommand(verboseHelp: verboseHelp),
    TraceCommand(),
    UpdatePackagesCommand(hidden: !verboseHelp),
    UpgradeCommand(),
  ], verbose: verbose,
     muteCommandLogging: muteCommandLogging,
     verboseHelp: verboseHelp);
}

Here, the runner is a general class for running and parsing on the command line of flutter. When executing runner.run, a series of XXXCommand methods are passed in. We just need to know that the corresponding commands of flutter build will match the BuildCommand() method.

Let's look at the implementation of BuildCommand:

class BuildCommand extends FlutterCommand {
  BuildCommand({bool verboseHelp = false}) {
    addSubcommand(BuildApkCommand(verboseHelp: verboseHelp));
    addSubcommand(BuildAotCommand());
    addSubcommand(BuildIOSCommand());
    addSubcommand(BuildFlxCommand());
    addSubcommand(BuildBundleCommand(verboseHelp: verboseHelp));
  }

  @override
  final String name = 'build'; // flutter build

  @override
  final String description = 'Flutter build commands.';

  @override
  Future<FlutterCommandResult> runCommand() async => null;
}

A series of subcommands defined here correspond to what we see when we run flitter build-h:

Flutter build commands.

Usage: flutter build <subcommand> [arguments]
-h, --help    Print this usage information.

Available subcommands:
  aot      Build an ahead-of-time compiled snapshot of your app's Dart code.
  apk      Build an Android APK file from your app.
  bundle   Build the Flutter assets directory from your app.
  flx      Deprecated
  ios      Build an iOS application bundle (Mac OS X host only).

BuildCommand inherits FlutterCommand. When BuildCommand is called externally, the run method of the parent FlutterCommand is executed:

  /// Runs this command.
  ///
  /// Rather than overriding this method, subclasses should override
  /// [verifyThenRunCommand] to perform any verification
  /// and [runCommand] to execute the command
  /// so that this method can record and report the overall time to analytics.
  @override
  Future<void> run() {
    final DateTime startTime = systemClock.now();

    return context.run<void>(
      name: 'command',
      overrides: <Type, Generator>{FlutterCommand: () => this},
      body: () async {
... ...
        try {
          commandResult = await verifyThenRunCommand();
        } on ToolExit {
... ...

It mainly calls the verifyThenRunCommand method.

  /// Perform validation then call [runCommand] to execute the command.
  /// Return a [Future] that completes with an exit code
  /// indicating whether execution was successful.
  ///
  /// Subclasses should override this method to perform verification
  /// then call this method to execute the command
  /// rather than calling [runCommand] directly.
  @mustCallSuper
  Future<FlutterCommandResult> verifyThenRunCommand() async {
    await validateCommand();

    // Populate the cache. We call this before pub get below so that the sky_engine
    // package is available in the flutter cache for pub to find.
    if (shouldUpdateCache)
      await cache.updateAll();

    if (shouldRunPub) {
      await pubGet(context: PubContext.getVerifyContext(name));
      final FlutterProject project = await FlutterProject.current();
      await project.ensureReadyForPlatformSpecificTooling();
    }

    setupApplicationPackages();

    final String commandPath = await usagePath;

    if (commandPath != null) {
      final Map<String, String> additionalUsageValues = await usageValues;
      flutterUsage.sendCommand(commandPath, parameters: additionalUsageValues);
    }

    return await runCommand();
  }

This method does three things:

  • First, pubGet will do some validation, mainly to download the configuration dependency in pubspec.yaml. Actually, it is completed by the pub instruction in Dart. The complete command line is as follows: flitter / bin / cache / Dart SDK / bin / pub -- verbosity = warning get -- no precompile
  • Next, ensureReadyForPlatformSpecificTooling and setupApplicationPackages will set the corresponding compilation environment according to the corresponding platform. Here, set the Android gradle environment and add some additional gradle properties.
  • Finally, call the method of the subclass's real runCommand.

Because build apk is executed here, the subclass is BuildApkCommand, so continue to see the runCommand of BuildApkCommand:

class BuildApkCommand extends BuildSubCommand {
  ... ...

  @override
  Future<FlutterCommandResult> runCommand() async {
    await super.runCommand();
    await buildApk(
      project: await FlutterProject.current(),
      target: targetFile,
      buildInfo: getBuildInfo(),
    );
    return null;
  }
}

The core is buildApk:

Future<void> buildApk({
  @required FlutterProject project,
  @required String target,
  BuildInfo buildInfo = BuildInfo.debug
}) async {
  if (!project.android.isUsingGradle) {
    throwToolExit(
        'The build process for Android has changed, and the current project configuration\n'
            'is no longer valid. Please consult\n\n'
            '  https://github.com/flutter/flutter/wiki/Upgrading-Flutter-projects-to-build-with-gradle\n\n'
            'for details on how to upgrade the project.'
    );
  }

  // Detect Android home
  // Validate that we can find an android sdk.
  if (androidSdk == null)
    throwToolExit('No Android SDK found. Try setting the ANDROID_HOME environment variable.');

  final List<String> validationResult = androidSdk.validateSdkWellFormed();
  if (validationResult.isNotEmpty) {
    for (String message in validationResult) {
      printError(message, wrap: false);
    }
    throwToolExit('Try re-installing or updating your Android SDK.');
  }

  return buildGradleProject(
    project: project,
    buildInfo: buildInfo,
    target: target,
  );
}

It is required that Android project must use gradle, otherwise it will exit directly. Then, check whether ANDROID_HOME exists and call buildGradleProject.

The main task of buildgradle project is to add some necessary parameters and execute the command of gradle. The final complete command line is as follows:

flutter_hello/android/gradlew -q -Ptarget=lib/main.dart -Ptrack-widget-creation=false -Ptarget-platform=android-arm assembleRelease

This brings us back to the familiar Android world. It just adds some additional parameters to the familiar gradlew assemblyrelease. Flitter Hello is our sample project of flitter.

Why do you need to go around so much to execute the gradle command? Naturally, due to the positioning of the Flutter itself, it is a cross platform solution, so there is a specific implementation for each platform, so it is necessary to convert a unified flutter build entry into the actual compilation commands required by each platform.

flutter.gradle

Now that we have reached gradlew, of course, we can just look at his build.gradle. His content is generated by the Flutter SDK, which is slightly different from the ordinary Android project. But we only need to understand according to gradle's knowledge system to get through.

The difference between it and the standard Android gradle project is only the beginning of the file:

def localProperties = new Properties()
def localPropertiesFile = rootProject.file('local.properties')
if (localPropertiesFile.exists()) {
    localPropertiesFile.withReader('UTF-8') { reader ->
        localProperties.load(reader)
    }
}

def flutterRoot = localProperties.getProperty('flutter.sdk')
if (flutterRoot == null) {
    throw new GradleException("Flutter SDK not found. Define location with flutter.sdk in the local.properties file.")
}

def flutterVersionCode = localProperties.getProperty('flutter.versionCode')
if (flutterVersionCode == null) {
    flutterVersionCode = '1'
}

def flutterVersionName = localProperties.getProperty('flutter.versionName')
if (flutterVersionName == null) {
    flutterVersionName = '1.0'
}

apply plugin: 'com.android.application'
apply from: "$flutterRoot/packages/flutter_tools/gradle/flutter.gradle"

Here is mainly to obtain some properties related to the Flutter in local.properties, which are flutter.sdk, flutter.versionCode and flutter.versionName. They indicate the path of the flutter sdk and some version number information.

Next, you go to flitter / packages / flitter_tools / gradle / flitter.gradle.

flutter.gradle mainly implements a flutter plugin, which is a standard gradle plugin. Therefore, it must define some tasks and set necessary dependencies. These dependencies are set in addFlutterTask method:

// in addFlutterTask
// We know that the flutter app is a subproject in another Android app when these tasks exist.
Task packageAssets = project.tasks.findByPath(":flutter:package${variant.name.capitalize()}Assets")
Task cleanPackageAssets = project.tasks.findByPath(":flutter:cleanPackage${variant.name.capitalize()}Assets")
Task copyFlutterAssetsTask = project.tasks.create(name: "copyFlutterAssets${variant.name.capitalize()}", type: Copy) {
    dependsOn flutterTask
    dependsOn packageAssets ? packageAssets : variant.mergeAssets
    dependsOn cleanPackageAssets ? cleanPackageAssets : "clean${variant.mergeAssets.name.capitalize()}"
    into packageAssets ? packageAssets.outputDir : variant.mergeAssets.outputDir
    with flutterTask.assets
}
if (packageAssets) {
    // Only include configurations that exist in parent project.
    Task mergeAssets = project.tasks.findByPath(":app:merge${variant.name.capitalize()}Assets")
    if (mergeAssets) {
        mergeAssets.dependsOn(copyFlutterAssetsTask)
    }
} else {
    variant.outputs[0].processResources.dependsOn(copyFlutterAssetsTask)
}

processXXXResources this Task will depend on copyFlutterAssetsTask, which makes it necessary to finish copyFlutterAssetsTask before proceeding to processXXXResources. In this way, the relevant processing of FLUENT is embedded in the compilation process of gradle.

In addition, copyFlutterAssetsTask relies on flutterTask and mergeXXXAssets. That is to say, when the flutterTask completes the compilation of the flutter and the execution of mergeXXXAssets, that is, after the normal Android assets processing is completed, the corresponding products of the flutter will be copied by copyFlutterAssetsTask to the directory build/app/intermediates/merged_assets/debug/mergeXXXAssets/out. XXX here refers to various build variant s, i.e. Debug or Release.

The compiled product of the flutter is specified by the getAssets method of the flutterTask

CopySpec getAssets() {
    return project.copySpec {
        from "${intermediateDir}"

        include "flutter_assets/**" // the working dir and its files

        if (buildMode == 'release' || buildMode == 'profile') {
            if (buildSharedLibrary) {
                include "app.so"
            } else {
                include "vm_snapshot_data"
                include "vm_snapshot_instr"
                include "isolate_snapshot_data"
                include "isolate_snapshot_instr"
            }
        }
    }
}

Specifically, these products are all the contents in the flitter ﹣ assets / directory under build / APP / mediates / flitter / xxx. If it is a release or profile version, it also includes the binary product app.so or **** snapshot of Dart. As you can see, in addition to the default "snapshot", we can also specify the Dart product as a regular so library.

Obviously, the compilation process of flutter is included in flutterTask. Its definition is as follows:

FlutterTask flutterTask = project.tasks.create(name: "${flutterBuildPrefix}${variant.name.capitalize()}", type: FlutterTask) {
    flutterRoot this.flutterRoot
    flutterExecutable this.flutterExecutable
    buildMode flutterBuildMode
    localEngine this.localEngine
    localEngineSrcPath this.localEngineSrcPath
    targetPath target
    verbose verboseValue
    fileSystemRoots fileSystemRootsValue
    fileSystemScheme fileSystemSchemeValue
    trackWidgetCreation trackWidgetCreationValue
    compilationTraceFilePath compilationTraceFilePathValue
    buildHotUpdate buildHotUpdateValue
    buildSharedLibrary buildSharedLibraryValue
    targetPlatform targetPlatformValue
    sourceDir project.file(project.flutter.source)
    intermediateDir project.file("${project.buildDir}/${AndroidProject.FD_INTERMEDIATES}/flutter/${variant.name}")
    extraFrontEndOptions extraFrontEndOptionsValue
    extraGenSnapshotOptions extraGenSnapshotOptionsValue
}

All kinds of parameters required for FlutterTask are passed in here, while the specific core of FlutterTask is just one sentence

class FlutterTask extends BaseFlutterTask {
... ...
    @TaskAction
    void build() {
        buildBundle()
    }
}

In other words, the buildBundle method is called. Let's look at the first half of it:


void buildBundle() {
    if (!sourceDir.isDirectory()) {
        throw new GradleException("Invalid Flutter source directory: ${sourceDir}")
    }

    intermediateDir.mkdirs()

    if (buildMode == "profile" || buildMode == "release") {
        project.exec {
            executable flutterExecutable.absolutePath
            workingDir sourceDir
            if (localEngine != null) {
                args "--local-engine", localEngine
                args "--local-engine-src-path", localEngineSrcPath
            }
            args "build", "aot"
            args "--suppress-analytics"
            args "--quiet"
            args "--target", targetPath
            args "--target-platform", "android-arm"
            args "--output-dir", "${intermediateDir}"
            if (trackWidgetCreation) {
                args "--track-widget-creation"
            }
            if (extraFrontEndOptions != null) {
                args "--extra-front-end-options", "${extraFrontEndOptions}"
            }
            if (extraGenSnapshotOptions != null) {
                args "--extra-gen-snapshot-options", "${extraGenSnapshotOptions}"
            }
            if (buildSharedLibrary) {
                args "--build-shared-library"
            }
            if (targetPlatform != null) {
                args "--target-platform", "${targetPlatform}"
            }
            args "--${buildMode}"
        }
    }
... ...
}

Here, because it is the release version, the binary Dart product of aot will be compiled first, that is, the **** snapshot product. In fact, the following commands are executed:

flutter build aot --suppress-analytics --quiet --target lib/main.dart --target-platform android-arm --output-dir /path-to-project/build/app/intermediates/flutter/release --target-platform android-arm --release

Next, the second half of the buildBundle method calls the fluent command once:

void buildBundle() {
... ....
    project.exec {
        executable flutterExecutable.absolutePath
        workingDir sourceDir
        if (localEngine != null) {
            args "--local-engine", localEngine
            args "--local-engine-src-path", localEngineSrcPath
        }
        args "build", "bundle"
        args "--suppress-analytics"
        args "--target", targetPath
        if (verbose) {
            args "--verbose"
        }
        if (fileSystemRoots != null) {
            for (root in fileSystemRoots) {
                args "--filesystem-root", root
            }
        }
        if (fileSystemScheme != null) {
            args "--filesystem-scheme", fileSystemScheme
        }
        if (trackWidgetCreation) {
            args "--track-widget-creation"
        }
        if (compilationTraceFilePath != null) {
            args "--precompile", compilationTraceFilePath
        }
        if (buildHotUpdate) {
            args "--hotupdate"
        }
        if (extraFrontEndOptions != null) {
            args "--extra-front-end-options", "${extraFrontEndOptions}"
        }
        if (extraGenSnapshotOptions != null) {
            args "--extra-gen-snapshot-options", "${extraGenSnapshotOptions}"
        }
        if (targetPlatform != null) {
            args "--target-platform", "${targetPlatform}"
        }
        if (buildMode == "release" || buildMode == "profile") {
            args "--precompiled"
        } else {
            args "--depfile", "${intermediateDir}/snapshot_blob.bin.d"
        }
        args "--asset-dir", "${intermediateDir}/flutter_assets"
        if (buildMode == "debug") {
            args "--debug"
        }
        if (buildMode == "profile" || buildMode == "dynamicProfile") {
            args "--profile"
        }
        if (buildMode == "release" || buildMode == "dynamicRelease") {
            args "--release"
        }
        if (buildMode == "dynamicProfile" || buildMode == "dynamicRelease") {
            args "--dynamic"
        }
    }
}

That's execution

flutter build bundle --suppress-analytics --target lib/main.dart --target-platform android-arm --precompiled --asset-dir /Users/xl/WorkSpace/FlutterProjects/flutter_hello/build/app/intermediates/flutter/release/flutter_assets --release]

Let's take a look at the specific implementation of the two commands: flitter build AOT and flitter build bundle.

flutter build aot

Review this command:

flutter build aot --suppress-analytics --quiet --target lib/main.dart --target-platform android-arm --output-dir /path-to-project/build/app/intermediates/flutter/release --target-platform android-arm --release

As mentioned in the previous analysis of the fluent build APK, the fluent build will be converted to start a Dart virtual machine through the fluent command line script and execute the fluent UU tool.snapshot. Therefore, the above command will be converted into:

flutter/bin/cache/dart-sdk/bin/dart FLUTTER_TOOL_ARGS= SNAPSHOT_PATH=flutter/bin/cache/flutter_tools.snapshot build aot --suppress-analytics --quiet --target lib/main.dart --target-platform android-arm --output-dir /path-to-project/flutter_hello/build/app/intermediates/flutter/release --target-platform android-arm --release

As mentioned in the previous explanation of fluent tools.snapshot, a series of subcommands are defined in BuildCommand:

class BuildCommand extends FlutterCommand {
  BuildCommand({bool verboseHelp = false}) {
    addSubcommand(BuildApkCommand(verboseHelp: verboseHelp));
    addSubcommand(BuildAotCommand());
    addSubcommand(BuildIOSCommand());
    addSubcommand(BuildFlxCommand());
    addSubcommand(BuildBundleCommand(verboseHelp: verboseHelp));
  }
... ...

The corresponding nature here is buildaaotcommand. Look at its runCommand directly:

class BuildAotCommand extends BuildSubCommand {
... ...

  Future<FlutterCommandResult> runCommand() async {
... ...

      String mainPath = findMainDartFile(targetFile);
      final AOTSnapshotter snapshotter = AOTSnapshotter();

      // Compile to kernel.
      mainPath = await snapshotter.compileKernel(
        platform: platform,
        buildMode: buildMode,
        mainPath: mainPath,
        packagesPath: PackageMap.globalPackagesPath,
        trackWidgetCreation: false,
        outputPath: outputPath,
        extraFrontEndOptions: argResults[FlutterOptions.kExtraFrontEndOptions],
      );
      
... ...

      // Build AOT snapshot.
      if (platform == TargetPlatform.ios) {
      ... ...
      } else {
        // Android AOT snapshot.
        final int snapshotExitCode = await snapshotter.build(
          platform: platform,
          buildMode: buildMode,
          mainPath: mainPath,
          packagesPath: PackageMap.globalPackagesPath,
          outputPath: outputPath,
          buildSharedLibrary: argResults['build-shared-library'],
          extraGenSnapshotOptions: argResults[FlutterOptions.kExtraGenSnapshotOptions],
        );
        if (snapshotExitCode != 0) {
          status?.cancel();
          throwToolExit('Snapshotting exited with non-zero exit code: $snapshotExitCode');
        }
      }
... ...
  }

In fact, two Dart virtual machine commands are called again, which mainly do two things:

  • Generate kernel file
  • Generate AOT executable

Both products are program files generated by Dart code.

The kernel file format is a special data format defined by dart. Dart virtual machine will execute it in an interpreted way. The snapshot files we mentioned earlier, such as flitter? Tool.snapshot, are actually kernel files.

The AOT executable needs to be generated according to the kernel during compilation after the kernel file is generated. It is binary, machine platform (arm, arm64, x86, etc.) executable code, so it is called AOT(Ahead of time compiling). At runtime, its instruction code is already the machine code of the platform, so it no longer needs Dart virtual machine parsing, and the execution speed is faster. In release mode, Dart source code packaged into APK exists in the form of AOT file.

Generate kernel file

Let's analyze the first step, which is Compile to kernel. It generates the kernel file according to the Dart code file. Specifically, it executes the following commands:

flutter/bin/cache/dart-sdk/bin/dart /path-to-flutter-sdk/flutter/bin/cache/artifacts/engine/darwin-x64/frontend_server.dart.snapshot --sdk-root flutter/bin/cache/artifacts/engine/common/flutter_patched_sdk/ --strong --target=flutter --aot --tfa -Ddart.vm.product=true --packages .packages --output-dill /path-to-project/flutter_hello/build/app/intermediates/flutter/release/app.dill --depfile /path-to-project/flutter_hello/build/app/intermediates/flutter/release/kernel_compile.d package:flutter_hello/main.dart

As you can see, this is to start frontend_server.dart.snapshot through the Dart virtual machine to compile the Dart code file into a kernel file named app.dill.

This frontend_server.dart.snapshot is different from the previous flitter_tool.snapshot. It exists in the engine's Dart code, not in the flitter SDK.

The code of engine needs to be downloaded separately: https://github.com/flutter/engine

The entry of frontend_server.dart.snapshot exists in engine/frontend_server/bin/starter.dart. After a series of jumps, the final execution is the compileToKernel method:

Future<Component> compileToKernel(Uri source, CompilerOptions options,
    {bool aot: false,
    bool useGlobalTypeFlowAnalysis: false,
    Map<String, String> environmentDefines,
    bool genBytecode: false,
    bool dropAST: false,
    bool useFutureBytecodeFormat: false,
    bool enableAsserts: false,
    bool enableConstantEvaluation: true}) async {
    
... ...

  final component = await kernelForProgram(source, options);

... ...

  // Run global transformations only if component is correct.
  if (aot && component != null) {
    await _runGlobalTransformations(
        source,
        options,
        component,
        useGlobalTypeFlowAnalysis,
        environmentDefines,
        enableAsserts,
        enableConstantEvaluation,
        errorDetector);
  }

  if (genBytecode && !errorDetector.hasCompilationErrors && component != null) {
    await runWithFrontEndCompilerContext(source, options, component, () {
      generateBytecode(component,
          dropAST: dropAST,
          useFutureBytecodeFormat: useFutureBytecodeFormat,
          environmentDefines: environmentDefines);
    });
  }

  // Restore error handler (in case 'options' are reused).
  options.onDiagnostic = errorDetector.previousErrorHandler;

  return component;
}

There are three main steps: kernel for program, runglobal transformations, and runWithFrontEndCompilerContext.

kernelForProgram

Future<Component> kernelForProgram(Uri source, CompilerOptions options) async {
  var pOptions = new ProcessedOptions(options: options, inputs: [source]);
  return await CompilerContext.runWithOptions(pOptions, (context) async {
    var component = (await generateKernelInternal())?.component;
    if (component == null) return null;

    if (component.mainMethod == null) {
      context.options.report(
          messageMissingMain.withLocation(source, -1, noLength),
          Severity.error);
      return null;
    }
    return component;
  });
}

Let's look at the comments of this method first:

/// Generates a kernel representation of the program whose main library is in
/// the given [source].
///
/// Intended for whole-program (non-modular) compilation.
///
/// Given the Uri of a file containing a program's `main` method, this function
/// follows `import`, `export`, and `part` declarations to discover the whole
/// program, and converts the result to Dart Kernel format.
///
/// If `compileSdk` in [options] is true, the generated component will include
/// code for the SDK.
///
/// If summaries are provided in [options], the compiler will use them instead
/// of compiling the libraries contained in those summaries. This is useful, for
/// example, when compiling for platforms that already embed those sources (like
/// the sdk in the standalone VM).
///
/// The input [source] is expected to be a script with a main method, otherwise
/// an error is reported.

The notes are quite clear. It will find all Dart codes contained in the complete program according to the path of the Dart code file containing the main function and the import, export, and part in it, and finally convert them to the kernel format file. The main purpose of this step is to generate a Component object, which will hold all the information of the Dart program.

Component is defined as follows

/// A way to bundle up libraries in a component.
class Component extends TreeNode {
  final CanonicalName root;

  final List<Library> libraries;

  /// Map from a source file URI to a line-starts table and source code.
  /// Given a source file URI and a offset in that file one can translate
  /// it to a line:column position in that file.
  final Map<Uri, Source> uriToSource;

  /// Mapping between string tags and [MetadataRepository] corresponding to
  /// those tags.
  final Map<String, MetadataRepository<dynamic>> metadata =
      <String, MetadataRepository<dynamic>>{};

  /// Reference to the main method in one of the libraries.
  Reference mainMethodName;
 ... ...

Component is actually just a wrapper Class. Its main function is to organize all libraries in the program. All subsequent processing of the kernel is based on Componet. Library refers to app source files, all package or dart libraries and third-party libraries. Each library contains all the Class, Field, Procedure and other components under its own package.

class Library extends NamedNode implements Comparable<Library>, FileUriNode {
  /// An import path to this library.
  ///
  /// The [Uri] should have the `dart`, `package`, `app`, or `file` scheme.
  ///
  /// If the URI has the `app` scheme, it is relative to the application root.
  Uri importUri;

  /// The URI of the source file this library was loaded from.
  Uri fileUri;

  /// If true, the library is part of another build unit and its contents
  /// are only partially loaded.
  ///
  /// Classes of an external library are loaded at one of the [ClassLevel]s
  /// other than [ClassLevel.Body].  Members in an external library have no
  /// body, but have their typed interface present.
  ///
  /// If the library is non-external, then its classes are at [ClassLevel.Body]
  /// and all members are loaded.
  bool isExternal;

  String name;

  @nocoq
  final List<Expression> annotations;

  final List<LibraryDependency> dependencies;

  /// References to nodes exported by `export` declarations that:
  /// - aren't ambiguous, or
  /// - aren't hidden by local declarations.
  @nocoq
  final List<Reference> additionalExports = <Reference>[];

  @informative
  final List<LibraryPart> parts;

  final List<Typedef> typedefs;
  final List<Class> classes;
  final List<Procedure> procedures;
  final List<Field> fields;

Because composnet is just a wrapper class, we mainly deal with the Library objects by taking it.

The generateKernelInternal method goes through a series of calls to the buildBody method, which processes a single Library. The outer layer will pass the libraries in composnet to this method in turn.

  Future<Null> buildBody(LibraryBuilder library) async {
    if (library is SourceLibraryBuilder) {
      // We tokenize source files twice to keep memory usage low. This is the
      // second time, and the first time was in [buildOutline] above. So this
      // time we suppress lexical errors.
      Token tokens = await tokenize(library, suppressLexicalErrors: true);
      if (tokens == null) return;
      DietListener listener = createDietListener(library);
      DietParser parser = new DietParser(listener);
      parser.parseUnit(tokens);
      for (SourceLibraryBuilder part in library.parts) {
        if (part.partOfLibrary != library) {
          // Part was included in multiple libraries. Skip it here.
          continue;
        }
        Token tokens = await tokenize(part);
        if (tokens != null) {
          listener.uri = part.fileUri;
          listener.partDirectiveIndex = 0;
          parser.parseUnit(tokens);
        }
      }
    }
  }

The buildBody method will tokenize and parse the Library and the part in the Library respectively.

What tokenize does is to parse Dart source code in the Library into lexical unit tokens according to lexical rules.

parseUnit is to further parse the tokens obtained from the previous tokenize into an abstract syntax tree according to Dart syntax rules.

As a result, Dart source code has been transformed into an abstract syntax tree and stored in Component.

_runGlobalTransformations

In the second step, runGlobalTransformations performs a series of transformations:

Future _runGlobalTransformations(
    Uri source,
    CompilerOptions compilerOptions,
    Component component,
    bool useGlobalTypeFlowAnalysis,
    Map<String, String> environmentDefines,
    bool enableAsserts,
    bool enableConstantEvaluation,
    ErrorDetector errorDetector) async {
  if (errorDetector.hasCompilationErrors) return;

  final coreTypes = new CoreTypes(component);
  _patchVmConstants(coreTypes);

  // TODO(alexmarkov, dmitryas): Consider doing canonicalization of identical
  // mixin applications when creating mixin applications in frontend,
  // so all backends (and all transformation passes from the very beginning)
  // can benefit from mixin de-duplication.
  // At least, in addition to VM/AOT case we should run this transformation
  // when building a platform dill file for VM/JIT case.
  mixin_deduplication.transformComponent(component);

  if (enableConstantEvaluation) {
    await _performConstantEvaluation(source, compilerOptions, component,
        coreTypes, environmentDefines, enableAsserts);

    if (errorDetector.hasCompilationErrors) return;
  }

  if (useGlobalTypeFlowAnalysis) {
    globalTypeFlow.transformComponent(
        compilerOptions.target, coreTypes, component);
  } else {
    devirtualization.transformComponent(coreTypes, component);
    no_dynamic_invocations_annotator.transformComponent(component);
  }

  // We don't know yet whether gen_snapshot will want to do obfuscation, but if
  // it does it will need the obfuscation prohibitions.
  obfuscationProhibitions.transformComponent(component, coreTypes);
}

You can see that the last line performs the confusing transform. The main confusion here is to do some mapping, which is actually similar to what proguard does. However, at present, it seems that the support in this area is still relatively rudimentary, and some custom rules and other functions should not be as perfect as proguard.

runWithFrontEndCompilerContext

runWithFrontEndCompilerContext mainly passes in a callback method:

    await runWithFrontEndCompilerContext(source, options, component, () {
      generateBytecode(component,
          dropAST: dropAST,
          useFutureBytecodeFormat: useFutureBytecodeFormat,
          environmentDefines: environmentDefines);
    });

This is the generateBytecode here. It needs to get the Component object generated by the kernel for program and generate the specific kernel bytecode according to the abstract syntax tree.

void generateBytecode(Component component,
    {bool dropAST: false,
    bool omitSourcePositions: false,
    bool useFutureBytecodeFormat: false,
    Map<String, String> environmentDefines,
    ErrorReporter errorReporter}) {
  final coreTypes = new CoreTypes(component);
  void ignoreAmbiguousSupertypes(Class cls, Supertype a, Supertype b) {}
  final hierarchy = new ClassHierarchy(component,
      onAmbiguousSupertypes: ignoreAmbiguousSupertypes);
  final typeEnvironment =
      new TypeEnvironment(coreTypes, hierarchy, strongMode: true);
  final constantsBackend =
      new VmConstantsBackend(environmentDefines, coreTypes);
  final errorReporter = new ForwardConstantEvaluationErrors(typeEnvironment);
  new BytecodeGenerator(
          component,
          coreTypes,
          hierarchy,
          typeEnvironment,
          constantsBackend,
          omitSourcePositions,
          useFutureBytecodeFormat,
          errorReporter)
      .visitComponent(component);
  if (dropAST) {
    new DropAST().visitComponent(component);
  }
}

This is mainly to call BytecodeGenerator.visitComponent. When visitcomponent object is called, all libraries in it will be accessed according to the situation. BytecodeGenerator is a tedious class. It has different processing methods for each syntax type in the syntax tree. Let's take a look at it

class BytecodeGenerator extends RecursiveVisitor<Null> {
... ...

  @override
  visitComponent(Component node) => node.visitChildren(this);

  @override
  visitLibrary(Library node) {
    if (node.isExternal) {
      return;
    }
    visitList(node.classes, this);
    visitList(node.procedures, this);
    visitList(node.fields, this);
  }

  @override
  visitClass(Class node) {
    visitList(node.constructors, this);
    visitList(node.procedures, this);
    visitList(node.fields, this);
  }

The visitxxx series of methods here will recursively access its member nodes, thus completely traversing and processing the entire syntax tree.

A Library contains several basic components such as Constructor, Procedure and Field. When visitxxx accesses them, the defaultMember method will be called eventually.

  @override
  defaultMember(Member node) {
    if (node.isAbstract) {
      return;
    }
    try {
      if (node is Field) {
        if (node.isStatic && !_hasTrivialInitializer(node)) {
          start(node);
          if (node.isConst) {
            _genPushConstExpr(node.initializer);
          } else {
            node.initializer.accept(this);
          }
          _genReturnTOS();
          end(node);
        }
      } else if ((node is Procedure && !node.isRedirectingFactoryConstructor) ||
          (node is Constructor)) {
        start(node);
        if (node is Constructor) {
          _genConstructorInitializers(node);
        }
        if (node.isExternal) {
          final String nativeName = getExternalName(node);
          if (nativeName == null) {
            return;
          }
          _genNativeCall(nativeName);
        } else {
          node.function?.body?.accept(this);
          // BytecodeAssembler eliminates this bytecode if it is unreachable.
          asm.emitPushNull();
        }
        _genReturnTOS();
        end(node);
      }
    } on BytecodeLimitExceededException {
      // Do not generate bytecode and fall back to using kernel AST.
      hasErrors = true;
      end(node);
    }
  }

This method will continue to call the ﹣ genXXXX series of methods, and the ﹣ genXXXX methods will also call each other. Let's take a look at some of them

183:  void _genNativeCall(String nativeName) {
288:  void _genConstructorInitializers(Constructor node) {
314:  void _genFieldInitializer(Field field, Expression initializer) {
330:  void _genArguments(Expression receiver, Arguments arguments) {
339:  void _genPushBool(bool value) {
347:  void _genPushInt(int value) {
370:  void _genPushConstExpr(Expression expr) {
383:  void _genReturnTOS() {
387:  void _genStaticCall(Member target, ConstantArgDesc argDesc, int totalArgCount,
401:  void _genStaticCallWithArgs(Member target, Arguments args,
424:  void _genTypeArguments(List<DartType> typeArgs, {Class instantiatingClass}) {
453:  void _genPushInstantiatorAndFunctionTypeArguments(List<DartType> types) {
469:  void _genPushInstantiatorTypeArguments() {
556:  void _genPushFunctionTypeArguments() {
564:  void _genPushContextForVariable(VariableDeclaration variable,
578:  void _genPushContextIfCaptured(VariableDeclaration variable) {
584:  void _genLoadVar(VariableDeclaration v, {int currentContextLevel}) {
... ...

It is easy to see from the name that their function is to generate various elements of code structure, including assignment statement, return statement, judgment statement, etc.

And_ The genXXXX method is not the final performer. The real hero behind the scenes is asm.emitXXXX Series functions.

Just a bool assignment_ For example, genPushBool:

  void _genPushBool(bool value) {
    if (value) {
      asm.emitPushTrue();
    } else {
      asm.emitPushFalse();
    }
  }

It calls the asm.emitPushTrue and asm.emitPushFalse . Take setting the true value as an example:

  void emitPushTrue() {
    emitWord(_encode0(Opcode.kPushTrue));
  }

  int _encode0(Opcode opcode) => _uint8(opcode.index);

  void emitWord(int word) {
    if (isUnreachable) {
      return;
    }
    _encodeBufferIn[0] = word;
    bytecode.addAll(_encodeBufferOut);
  }

Opcode.kPushTrue Represents a bytecode value of push true. emitPushTrue will convert it to an int sized bytecode and write it to bytecode

Dart defines a set of bytecode instructions,

enum Opcode {
  kTrap,

  // Prologue and stack management.
  kEntry,
  kEntryFixed,
  kEntryOptional,
  kLoadConstant,
  kFrame,
  kCheckFunctionTypeArgs,
  kCheckStack,

  // Object allocation.
  kAllocate,
  kAllocateT,
  kCreateArrayTOS,

  // Context allocation and access.
  kAllocateContext,
  kCloneContext,
  kLoadContextParent,
  kStoreContextParent,
  kLoadContextVar,
  kStoreContextVar,

  // Constants.
  kPushConstant,
  kPushNull,
  kPushTrue,
  kPushFalse,
  kPushInt,

  // Locals and expression stack.
  kDrop1,
  kPush,
  kPopLocal,
  kStoreLocal,

... ...

  // Int operations.
  kNegateInt,
  kAddInt,
  kSubInt,
  kMulInt,
  kTruncDivInt,
  kModInt,
  kBitAndInt,
  kBitOrInt,
  kBitXorInt,
  kShlInt,
  kShrInt,
  kCompareIntEq,
  kCompareIntGt,
  kCompareIntLt,
  kCompareIntGe,
  kCompareIntLe,
}

Each part of the syntax tree will be translated into different instructions above. Finally, the whole syntax tree will be completely parsed into binary instruction stream and stored in bytecode member of bytecode assembler.

class BytecodeAssembler {
... ...

  final List<int> bytecode = new List<int>();
... ...

  void emitWord(int word) {
    if (isUnreachable) {
      return;
    }
    _encodeBufferIn[0] = word;
    bytecode.addAll(_encodeBufferOut); // All add ed to bytecode
  }
  
... ...

At this point, the Dart code has been parsed into a kernel format instruction stream. Next, let's see how it is written into a file.

Write kernel file

Let's review the previous generateBytecode, which is a global function. In it, a BytecodeGenerator is new ly created, and its visitComponent method is used to parse the syntax tree, and generate binary instructions to flow to the bytecode member of BytecodeAssembler. This BytecodeAssembler corresponds to the member field asm of BytecodeGenerator. The main code is as follows:

void generateBytecode(Component component,
    {bool dropAST: false,
    bool omitSourcePositions: false,
    bool useFutureBytecodeFormat: false,
    Map<String, String> environmentDefines,
    ErrorReporter errorReporter}) {
... ...
  new BytecodeGenerator(
          component,
          coreTypes,
          hierarchy,
          typeEnvironment,
          constantsBackend,
          omitSourcePositions,
          useFutureBytecodeFormat,
          errorReporter)
      .visitComponent(component);
... ...
}

// The member field asm of BytecodeGenerator is BytecodeAssembler
class BytecodeGenerator extends RecursiveVisitor<Null> {
... ...
  BytecodeAssembler asm;
... ...
}


// bytecode of BytecodeAssembler stores all binary instruction streams
class BytecodeAssembler {
... ...
  final List<int> bytecode = new List<int>();
... ...
}

When traversing the syntax tree with visitComponent, remember that some columns visitxxx we call will go to defaultMember. Let's take a look at defaultMember.

  @override
  defaultMember(Member node) {
    if (node.isAbstract) {
      return;
    }
    try {
      if (node is Field) {
        if (node.isStatic && !_hasTrivialInitializer(node)) {
          start(node);
          if (node.isConst) {
            _genPushConstExpr(node.initializer);
          } else {
            node.initializer.accept(this);
          }
          _genReturnTOS();
          end(node);
        }
      } else if ((node is Procedure && !node.isRedirectingFactoryConstructor) ||
          (node is Constructor)) {
        start(node);
        if (node is Constructor) {
          _genConstructorInitializers(node);
        }
        if (node.isExternal) {
          final String nativeName = getExternalName(node);
          if (nativeName == null) {
            return;
          }
          _genNativeCall(nativeName);
        } else {
          node.function?.body?.accept(this);
          // BytecodeAssembler eliminates this bytecode if it is unreachable.
          asm.emitPushNull();
        }
        _genReturnTOS();
        end(node);
      }
    } on BytecodeLimitExceededException {
      // Do not generate bytecode and fall back to using kernel AST.
      hasErrors = true;
      end(node);
    }
  }

Note that all branches eventually call end(node),

  void end(Member node) {
    if (!hasErrors) {
      final formatVersion = useFutureBytecodeFormat
          ? futureBytecodeFormatVersion
          : stableBytecodeFormatVersion;
      metadata.mapping[node] = new BytecodeMetadata(formatVersion, cp,
          asm.bytecode, asm.exceptionsTable, nullableFields, closures);
    }

... ...
  }

As you can see, in the end function, asm.bytecode It has been transferred to metadata, which has been added to the component when constructing the method:

  BytecodeGenerator(
      this.component,
      this.coreTypes,
      this.hierarchy,
      this.typeEnvironment,
      this.constantsBackend,
      this.omitSourcePositions,
      this.useFutureBytecodeFormat,
      this.errorReporter)
      : recognizedMethods = new RecognizedMethods(typeEnvironment) {
    component.addMetadataRepository(metadata);
  }

Back to the beginning, compiletockernel is called by compile. It is passed in a component object and finally written to the file through the writeDillFile method.

  @override
  Future<bool> compile(
    String filename,
    ArgResults options, {
    IncrementalCompiler generator,
  }) async {
  
... ...

    Component component;
    
... ...

      component = await _runWithPrintRedirection(() => compileToKernel(
          _mainSource, compilerOptions,
          aot: options['aot'],
          useGlobalTypeFlowAnalysis: options['tfa'],
          environmentDefines: environmentDefines));

... ...

      await writeDillFile(component, _kernelBinaryFilename,
          filterExternal: importDill != null);

... ...
  }

The implementation of writeDillFile is as follows:

  writeDillFile(Component component, String filename,
      {bool filterExternal: false}) async {
    final IOSink sink = new File(filename).openWrite();
    final BinaryPrinter printer = filterExternal
        ? new LimitedBinaryPrinter(
            sink, (lib) => !lib.isExternal, true /* excludeUriToSource */)
        : printerFactory.newBinaryPrinter(sink);

    component.libraries.sort((Library l1, Library l2) {
      return "${l1.fileUri}".compareTo("${l2.fileUri}");
    });

    component.computeCanonicalNames();
    for (Library library in component.libraries) {
      library.additionalExports.sort((Reference r1, Reference r2) {
        return "${r1.canonicalName}".compareTo("${r2.canonicalName}");
      });
    }
    if (unsafePackageSerialization == true) {
      writePackagesToSinkAndTrimComponent(component, sink);
    }

    printer.writeComponentFile(component);
    await sink.close();
  }

filename is the / path to project / shuttle passed in through the command line parameter_ hello/build/app/intermediates/flutter/release/ app.dill . Here we focus on writeComponentFile.

  void writeComponentFile(Component component) {
    computeCanonicalNames(component);
    final componentOffset = getBufferOffset();
    writeUInt32(Tag.ComponentFile);
    writeUInt32(Tag.BinaryFormatVersion);
    indexLinkTable(component);
    indexUris(component);
    _collectMetadata(component);
    if (_metadataSubsections != null) {
      _writeNodeMetadataImpl(component, componentOffset);
    }
    libraryOffsets = <int>[];
    CanonicalName main = getCanonicalNameOfMember(component.mainMethod);
    if (main != null) {
      checkCanonicalName(main);
    }
    writeLibraries(component);
    writeUriToSource(component.uriToSource);
    writeLinkTable(component);
    _writeMetadataSection(component);
    writeStringTable(stringIndexer);
    writeConstantTable(_constantIndexer);
    writeComponentIndex(component, component.libraries);

    _flush();
  }

There are many parts written in it, so we will not analyze them one by one. Here we mainly look at the data saved in BytecodeMetadata after compiling asm.bytecode How the data of is written to the file here. This logic, mainly in_ writeNodeMetadataImpl.

  void _writeNodeMetadataImpl(Node node, int nodeOffset) {
    for (var subsection in _metadataSubsections) {
      final repository = subsection.repository;
      final value = repository.mapping[node];
      if (value == null) {
        continue;
      }

      if (!MetadataRepository.isSupported(node)) {
        throw "Nodes of type ${node.runtimeType} can't have metadata.";
      }

      if (!identical(_sink, _mainSink)) {
        throw "Node written into metadata can't have metadata "
            "(metadata: ${repository.tag}, node: ${node.runtimeType} $node)";
      }

      _sink = _metadataSink;
      subsection.metadataMapping.add(nodeOffset);
      subsection.metadataMapping.add(getBufferOffset());
      repository.writeToBinary(value, node, this);
      _sink = _mainSink;
    }
  }

repository.writeToBinary The corresponding is BytecodeMetadataRepository.writeToBinary

  void writeToBinary(BytecodeMetadata metadata, Node node, BinarySink sink) {
    sink.writeUInt30(metadata.version);
    sink.writeUInt30(metadata.flags);
    metadata.constantPool.writeToBinary(node, sink);
    sink.writeByteList(metadata.bytecodes); // Here bytecodes is written into the file
    if (metadata.hasExceptionsTable) {
      metadata.exceptionsTable.writeToBinary(sink);
    }
    if (metadata.hasNullableFields) {
      sink.writeUInt30(metadata.nullableFields.length);
      metadata.nullableFields.forEach((ref) => sink
          .writeCanonicalNameReference(getCanonicalNameOfMember(ref.asField)));
    }
    if (metadata.hasClosures) {
      sink.writeUInt30(metadata.closures.length);
      metadata.closures.forEach((c) => c.writeToBinary(sink));
    }
  }
  

The code is still relatively clear, that is, the data of metadata is written, sink.writeByteList ( metadata.bytecodes ); write bytecodes, in addition to version, flags and other information are also written to the file. In this case, sink refers to the kernel binary writer. It is a BinaryPrinter class that binds a specific file, that is app.dill .

At this point, the process of compiling and generating kernel files is finished.

Generate AOT executable

Now, you can follow the previous step app.dill To generate the AOT executable. Review the code that calls the command line to generate AOT in the gradle script:

        final int snapshotExitCode = await snapshotter.build(
          platform: platform,
          buildMode: buildMode,
          mainPath: mainPath,
          packagesPath: PackageMap.globalPackagesPath,
          outputPath: outputPath,
          buildSharedLibrary: argResults['build-shared-library'],
          extraGenSnapshotOptions: argResults[FlutterOptions.kExtraGenSnapshotOptions],
        );

The actual corresponding command is

flutter/bin/cache/artifacts/engine/android-arm-release/darwin-x64/gen_snapshot 
--causal_async_stacks 
--packages=.packages 
--deterministic 
--snapshot_kind=app-aot-blobs 
--vm_snapshot_data=/path-to-project/flutter_hello/build/app/intermediates/flutter/release/vm_snapshot_data 
--isolate_snapshot_data=/path-to-project/flutter_hello/build/app/intermediates/flutter/release/isolate_snapshot_data 
--vm_snapshot_instructions=/path-to-project/flutter_hello/build/app/intermediates/flutter/release/vm_snapshot_instr 
--isolate_snapshot_instructions=/path-to-project/flutter_hello/build/app/intermediates/flutter/release/isolate_snapshot_instr 
--no-sim-use-hardfp 
--no-use-integer-division /path-to-project/flutter_hello/build/app/intermediates/flutter/release/app.dill

Here, gen_snapshot is no longer a common Dart command, but a real Native binary executable. It corresponds to the C + + code in Dart virtual machine: dart/runtime/bin/gen_snapshot.cc , the entry is in the main function:

int main(int argc, char** argv) {
... ...
  
  error = Dart_Initialize(&init_params);

... ...

    return GenerateSnapshotFromKernel(kernel_buffer, kernel_buffer_size);

... ...
}

There are two main things to do here. First, initialize a Dart running environment according to the parameters passed in, mainly load the kernel file, and load all Dart classes into the running environment. Then, according to the existing running environment, the binary executable snapshot will be compiled directly.

Let's focus on the next step, generate snapshot from kernel.

gen_ There are many types of snapshots defined in snapshot:

static const char* kSnapshotKindNames[] = {
    "core",
    "core-jit",
    "core-jit-all",
    "app-jit",
    "app-aot-blobs",
    "app-aot-assembly",
    "vm-aot-assembly", NULL,
};

The corresponding enumeration types are:

// Global state that indicates whether a snapshot is to be created and
// if so which file to write the snapshot into. The ordering of this list must
// match kSnapshotKindNames below.
enum SnapshotKind {
  kCore,
  kCoreJIT,
  kCoreJITAll,
  kAppJIT,
  kAppAOTBlobs,
  kAppAOTAssembly,
  kVMAOTAssembly,
};

And our command line argument is -- snapshot_ Kind = app AOT blobs, so you only need to look at the kAppAOTBlobs here.

static int GenerateSnapshotFromKernel(const uint8_t* kernel_buffer,
                                      intptr_t kernel_buffer_size) {
                                        switch (snapshot_kind) {
... ...
    case kAppAOTBlobs:
    case kAppAOTAssembly: {
      if (Dart_IsNull(Dart_RootLibrary())) {
        Log::PrintErr(
            "Unable to load root library from the input dill file.\n");
        return kErrorExitCode;
      }

      CreateAndWritePrecompiledSnapshot();

      CreateAndWriteDependenciesFile();

      break;
    }
... ...

The main logic lies in CreateAndWritePrecompiledSnapshot

static void CreateAndWritePrecompiledSnapshot() {
... ...

  result = Dart_Precompile();

... ...

    result = Dart_CreateAppAOTSnapshotAsBlobs(
        &vm_snapshot_data_buffer, &vm_snapshot_data_size,
        &vm_snapshot_instructions_buffer, &vm_snapshot_instructions_size,
        &isolate_snapshot_data_buffer, &isolate_snapshot_data_size,
        &isolate_snapshot_instructions_buffer,
        &isolate_snapshot_instructions_size, shared_data, shared_instructions);

... ...

      WriteFile(vm_snapshot_data_filename, vm_snapshot_data_buffer,
                vm_snapshot_data_size);
      WriteFile(vm_snapshot_instructions_filename,
                vm_snapshot_instructions_buffer, vm_snapshot_instructions_size);
      WriteFile(isolate_snapshot_data_filename, isolate_snapshot_data_buffer,
                isolate_snapshot_data_size);
      WriteFile(isolate_snapshot_instructions_filename,
                isolate_snapshot_instructions_buffer,
                isolate_snapshot_instructions_size);
... ...
}

There are three steps.

  • Dart_Precompile for AOT compilation
  • Transfer snapshot code to buffer
  • Write buffer to four binaries

Focus on the first step, Dart_Precompile calls Precompiler::CompileAll() to realize compilation. The specific details are complex. Generally speaking, it will first follow the previous Dart_ The data of Dart running environment obtained from initialize generates FlowGraph objects, and then optimizes various execution flow graphs. Finally, the optimized FlowGraph objects are translated into binary instructions of specific architecture (arm/arm64/x86, etc.).

And the next two steps are to finally land the binary data in the memory into the file, that is, isolate_snapshot_data,isolate_snapshot_instr,vm_snapshot_data,vm_snapshot_instr these four files.

At this point, the fluent build AOT is finished, and the Dart code is completely compiled into binary executable.

flutter build bundle

Review this command:

flutter build bundle --suppress-analytics --target lib/main.dart --target-platform android-arm --precompiled --asset-dir /path-to-project/flutter_hello/build/app/intermediates/flutter/release/flutter_assets --release

As mentioned in the previous analysis of the fluent build APK, the fluent build will be converted to start a Dart virtual machine and execute the fluent through the fluent command line script_ tool.snapshot , so the above command is converted to:

flutter/bin/cache/dart-sdk/bin/dart 
FLUTTER_TOOL_ARGS= 
SNAPSHOT_PATH=flutter/bin/cache/flutter_tools.snapshot
build bundle 
--suppress-analytics 
--target lib/main.dart 
--target-platform android-arm 
--precompiled 
--asset-dir /path-to-project/flutter_hello/build/app/intermediates/flutter/release/flutter_assets 
--release

The build bundle corresponds to the BuildBundleCommand. Because it is a release mode, the parameter will bring -- precompiled, so the kernel file will not be compiled here. Its final execution is the following code:

Future<void> writeBundle(
    Directory bundleDir, Map<String, DevFSContent> assetEntries) async {
  if (bundleDir.existsSync())
    bundleDir.deleteSync(recursive: true);
  bundleDir.createSync(recursive: true);

  await Future.wait<void>(
      assetEntries.entries.map<Future<void>>((MapEntry<String, DevFSContent> entry) async {
    final File file = fs.file(fs.path.join(bundleDir.path, entry.key));
    file.parent.createSync(recursive: true);
    await file.writeAsBytes(await entry.value.contentsAsBytes());
  }));
}

Actually, I just put some files in build / APP / mediates / shuttle / release / shuttle_ In the assets directory, these files are:

packages/cupertino_icons/assets/CupertinoIcons.ttf
fonts/MaterialIcons-Regular.ttf
AssetManifest.json
FontManifest.json
LICENSE

Therefore, when the build bundle is completed, all the files required by the fluent have been put into the fluent_ Assets.

We were talking about flutter.gradle It's mentioned at the time. build/app/intermediates/flutter/release/flutter_ Everything in assets will be copied to build / APP / intermediates / merged_ Under assets / debug / mergexxxassets / out. In this way, these fluent files will be entered into APK at the end of the day, along with Android's standard taskmergeXXXAssets.

summary

At this point, the whole process of flutter compiling release package is analyzed. Let's sum up the whole compilation process with a diagram:

If it is the operation of compiling the debug package, fluent build APK -- debug, its process is as follows:

Of course, some of these command line specific parameters are different. In general, there is no build aot step in debug mode, and the step of compiling kernel files is also transferred from build aot in release version to build bundle.

This paper explains the process of compiling apk by Flutter in Android environment, but there are still many details not fully developed, including Dart's pub mechanism, the construction of abstract syntax tree, machine code compilation, etc. if every point needs to be analyzed clearly, it is also a long article.

It can be said that as a new technology, flutter has too many implementation details to savor and explore. In the process of reading the code, we also found that some code implementations are not very stable at present, and the official is also constantly optimizing. Through the study of its deep principles, we can not only apply what we have learned to achieve the improvement of specific needs, but also jointly improve and promote this new technology, making the environment in the field of mobile development technology more diverse and perfect.

Other Android in-depth technical articles can be concerned about the official account.

Tags: Android snapshot SDK Gradle

Posted on Sun, 17 May 2020 23:58:27 -0700 by MisterWebz