Two-Directional Foil in Programming--Dart Metaprogramming

The students who have read "Three Body" must know "dimension reduction attack". Seeing from a higher dimension, they will directly oppose KO.Today, let's chat about metaprogramming, the "two-way foil" in programming.

1. What is metaprogramming

We've heard too many nouns. It seems that our ears are a bit paralyzed.For example, some nouns add prefixes that seem to sparkle with light, such as meta, in order to pretend x (such as the title of the author's article...) or to make sense indefinitely.Computer software This industry has meta data, meta model, meta programming.
Today the protagonist of installing x is meta programming - meta programming.
In fact, many related articles can also be found on the network, for the definition of the word refer to a sentence from wikipedia:

Metaprogramming is a programming technique in which computer programs have the ability to treat other programs as their data. It means that a program can be designed to read, generate, analyze or transform other programs, and even modify itself while running. In some cases, this allows programmers to minimize the number of lines of code to express a solution, in turn reducing development time. It also allows programs greater flexibility to efficiently handle new situations without recompilation.

In short, programs are treated as data that can be used to manipulate programs or themselves while implementing certain capabilities, such as moving runtime work to compile time.
As compilers evolve, metaprogramming can implement such things as code substitution (through macro implementation), generic programming (evolving from macros, deviating from types, highly abstract logic, reducing the amount of code), or, in more advanced languages, run-time manipulation of code logic through introspection/reflection mechanisms, or with compilationThe decoupling and opening of processes enables the ability to manipulate grammar trees and intermediate languages in the intermediate language stage (AST, IL) to achieve greater scalability.

As a modern high-level language, Dart can manipulate code based on an intermediate language in addition to its templating capabilities.This article focuses on how to use AST tree operations for metaprogramming based on its intermediate language (dill) and implement some capabilities that are not available with the existing dart syntax itself.At compile time, this implementation has little impact on program performance at run time.

2. Introduction to metaprogramming in Dart

2.1 Background knowledge

We know that in almost any language, code generates a tree-like intermediate state during compilation (which occurs when an interpretive language is compiled at run time), called AST (Abstract Grammar Tree).AST describes the execution order and execution logic of substatements in each expression/statement, so it can be easily translated into target code.Based on this abstraction, it is reasonable to divide the compiler into three phases: FrontEnd, Optimizer, Backend, so as to implement languages that are compatible with various grammatical forms, and make it easier to migrate and compatible with cpu of different architectures.See the following picture:

These three stages revolve around this IL (intermediate language).The IL language isolates differences in grammar (easily adapted to new languages), platform architecture, and so on.

2.2 Dart Compilation Process

Dart is designed similarly, with Dill as the intermediate language.The difference is that Dill is not as open as either java's IL or DotNet's IL and can be written directly. Instead, Dill is implemented programmatically.

This is based on AST The library manipulates the Dill.

The components in this library contain the definition and access of all the nodes involved in the AST tree, abstracting basic programming concepts such as types, functions, statements, declarations, expressions into objects.Based on these objects, we can traverse the entire AST tree, or generate new types and functions, insert code statements, and implement new logic.

2.3 A few chestnuts

It's really easy to get started. Just look at the example code.

2.3.1. For example, the following statement defines a new Map variable and calls its constructor:

//Assembly parameters
Arguments mapFromArgs = Arguments.empty();
mapFromArgs.positional.add(MapLiteral([], keyType:keyInterType));
//Call from constructor
StaticInvocation mapConstructor = StaticInvocation(MapFromFactoryProc, mapFromArgs);
//Declare a Map type variable named jsonMap
VariableDeclaration mapInstDecl = VariableDeclaration("jsonMap", type:mapInterType);
//Equivalent to var jsonMap = new Map();
VariableSet set_mymap = VariableSet(mapInstDecl, mapConstructor);

2.3.2. Creating a function body

The body of a function is actually a Block.

Block bodyStatements = Block(List<Statement>());
bodyStatements.addStatement(mapInstDecl);
bodyStatements.addStatement(ExpressionStatement(inst));

2.3.3 Create Functions

This example refers to the declarative form of a function to create a new function with limited length and some parameters omitted.

static Procedure createProcedure(Procedure referProcedure ,Statement bodyStatements, DartType returnType) {
      FunctionNode functionNode = new FunctionNode(bodyStatements,
          //... parameter omitted
      );
      Procedure procedure = new Procedure(
        Name(referProcedure.canonicalName.name, referProcedure.name.library),ProcedureKind.Method, functionNode,
        //... parameter omitted
      );
      return procedure;
  }
//Call function creation and add to class definition
Procedure overridedToJsonFunc = createProcedure(jsonTypes.StaticBaseToJsonProc, bodyStatements, InterfaceType(mapClass));
youClazz.addMember(overridedToJsonFunc);

2.3.4 Other

Based on AST, complex expressions and statements can also be created, such as ForInStatement (for...in loop), which can also be converted to each other through ExpressionStatement and BlockExpression.For more tips, refer to the definition of AST.

2.4 How to Debug

The edited Dill appears to be a black box, and besides looking at logs or exception stacks, it is not possible to step through debugging, which makes development difficult.However, Dart provides tools that have turned the kernel dill into readable text for debugging:

$DartHome/dart ../../pkg/vm/bin/dump_kernel.dart /your/dill/file/path /output/dill/text/file.text

The open text file looks like this:

    static method  __from_Json1(core::Map<dynamic, dynamic> m) → aop2::UserDataT {
      aop2::UserDataT inst;
      inst = new aop2::UserDataT::•();
      inst.name = m.[]("name") is core::String ?{core::String} m.[]("name") : null;
      inst.city = m.[]("city") is core::String ?{core::String} m.[]("city") : null;
      inst.age = m.[]("age") is core::int ?{core::int} m.[]("age") : null;
      inst.squres = m.[]("squres") is core::double ?{core::double} m.[]("squres") : null;
      inst.sex = m.[]("sex") is core::bool ?{core::bool} m.[]("sex") : null;
      inst.houses = m.[]("houses") is core::Map<dynamic, dynamic> ?{core::Map<dynamic, dynamic>} block {
        core::Map<core::String, core::String> mymap;
        mymap = col::LinkedHashMap::from<core::String, core::String>(<core::String, core::String>{});
        for (core::String item in (m.[]("houses") as core::Map<dynamic, dynamic>).keys) {
          mymap.[]=(item, (m.[]("houses") as core::Map<dynamic, dynamic>).[](item) is core::String ?{core::String} (m.[]("houses") as core::Map<dynamic, dynamic>).[](item) : null);
        }
      } =>mymap : null;
      return inst;
    }

3. Applying Imagination

Based on Dill's anipulation, we can inject new logic into the code.For example, open source AOP libraries prior to Idle Fish Technology AspectD The principle is to generate an AST by loading a dill file, then traversing the AST to find the function or statement that has been annotated, and then generate a dill to participate in the compiler's subsequent process after the dill level operation, which ultimately implements AOP.

Similarly, we know that Dart is not very convenient for Json parsing, that jsonDecode cannot directly generate business objects, but collections such as Map or List, and that users need to manually code themselves to traverse these collections and load objects.Although the official open source solution is based on Source_gen, it is not user-friendly (there are other options, such as Dason, but it relies on Mirror, as detailed in A comparison here ).Actually, the logic of traversing a Map or List and assembling objects is simple, and we can do it with Dill Manipulation.

It is easy to use, for example:

@JsonModel()
class UserData {
  String name;
  String city;
  UserData son;
}

void main(){
    var str = '''
    {
      "name": "Jim",
      "city": "hangzhou",
      "son":{
        "name": "Kong",
        "city":"Hangzhou"
      }
    }
    ''';

    UserData userData = JsonDrill.fromJson<UserData>(jsonDecode(str));
    var map = JsonDrill.toJson<UserData>(userData);
    print("$map");
  }

More in-depth thinking, Dart's existing mirror capabilities are not yet recommended (reason analysis can refer to) This article ), can we implement a simple and lightweight LiteMirror Library Based on Dill Manipulation?Based on this LiteMirror library, can you implement higher-level Json resolution and AOP or even Hook capabilities?

Of course, smart you may have found that Dill Manipulation inevitably has to customize the compilation process, which requires, for example, that in a Flutter environment, Flutter Tool s be customized to include Dill re-editing.Dramatically, Idle Fish Technology has now implemented the Json parser and is preparing to open source. Please look forward to it


Text Link
This is an original content of Cloud-dwelling Community, which cannot be reproduced without permission.

Tags: Programming JSON network Java

Posted on Wed, 28 Aug 2019 21:24:04 -0700 by michaelk46