Jdk dynamic agent source code

Preface

There are two kinds of java Dynamic agents: Jdk dynamic agent and cglib dynamic agent. This paper mainly introduces the use, operation mechanism and source code analysis of Jdk dynamic agent. When spring does not manually turn on cglib dynamic proxy, that is: < AOP: AspectJ AutoProxy proxy target class = "true" / > or @ EnableAspectJAutoProxy(proxyTargetClass = true), the default is Jdk dynamic proxy. Dynamic agent has a wide range of applications, such as log, transaction management, cache and so on. This article will explain the application of simulating @ Cacheable, that is, buffering in dynamic agent. It should be noted that compared with cglib dynamic agent, the object of Jdk dynamic agent must implement the interface, otherwise an error will be reported. We will also take this question to find the answer in the source code analysis

When @ Cacheable annotation is on method

  1. Before method execution, the Jdk dynamic agent will be called to find redis (or other caches) first
  2. When the cache does not exist, execute methods, such as querying the database
  3. After the method is executed, the Jdk dynamic agent is called again to cache the result in Redis

I. use

step

  1. Create interface UserService
  2. Create interface implementation class UserServiceImpl
  3. Create a Jdk dynamic agent, JdkCacheHandler, to enhance the caching logic before and after the UserServiceImpl method

Code

  1. Create interface UserService
public interface UserService {
    public String getUserByName(String name);
}
  1. Create implementation class UserServiceImpl
public class UserServiceImpl implements UserService {
    @Override
    public String getUserByName(String name) {
        System.out.println("Query from database:" + name);
        return name;
    }
}
  1. Create a Jdk dynamic agent, JdkCacheHandler
public class JdkCacheHandler implements InvocationHandler {

    // Target class object
    private Object target;

    // Get target class object
    public JdkCacheHandler(Object target) {
        this.target = target;
    }

    // Create JDK agent
    public Object createJDKProxy() {
        Class clazz = target.getClass();
        // Three parameters are required to create a JDK agent: target class loader, target class interface, and agent class object (i.e. itself)
        return Proxy.newProxyInstance(clazz.getClassLoader(), clazz.getInterfaces(), this);
    }

    @Override
    public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {

        System.out.println("Before finding the database, find whether it exists in the cache:" + args[0]);
        // Trigger target class method
        Object result = method.invoke(target, args);
        System.out.printf("After finding the database, you will%s Add to cache\r\n", result);
        return result;
    }
}
  1. Create test class
public class JdkTest {

    @Test
    public void test() {
        UserService userService = new UserServiceImpl();
        JdkCacheHandler jdkCacheHandler = new JdkCacheHandler(userService);
        UserService proxy = (UserService) jdkCacheHandler.createJDKProxy();

        System.out.println("==========================");
        proxy.getUserByName("bugpool");
        System.out.println("==========================");

        System.out.println(proxy.getClass());
    }
}
  1. output
==========================
Before looking up the database, look up whether it exists in the cache: bugpool
 Query from database: bugpool
 After finding the database, add the bugpool to the cache
==========================
class com.sun.proxy.$Proxy4

2, Call mechanism

View $Proxy code

It can be seen that after passing through the Jdk dynamic proxy, the produced proxy is no longer of the UserService type, but of the $Proxy4 type. To understand its calling mechanism, you need to get the code of the proxy class first

System.out.println(proxy.getClass());

class com.sun.proxy.$Proxy4
  1. Modify the JVM running parameters and add - Dsun.misc.ProxyGenerator.saveGeneratedFiles=true

  2. Run test to view the code at com.sun.proxy. At this time, the production is class, and when the idea is opened, it will be automatically decompiled
  3. In the above output, you can see that the proxy class is $Proxy4. Now you get the source code of $Proxy4, and then analyze the calling mechanism of the proxy class
public final class $Proxy4 extends Proxy implements UserService {
    private static Method m1;
    private static Method m3;
    private static Method m2;
    private static Method m0;

    public $Proxy4(InvocationHandler var1) throws  {
        super(var1);
    }

    public final boolean equals(Object var1) throws  {
        try {
            return (Boolean)super.h.invoke(this, m1, new Object[]{var1});
        } catch (RuntimeException | Error var3) {
            throw var3;
        } catch (Throwable var4) {
            throw new UndeclaredThrowableException(var4);
        }
    }

    public final String getUserByName(String var1) throws  {
        try {
            return (String)super.h.invoke(this, m3, new Object[]{var1});
        } catch (RuntimeException | Error var3) {
            throw var3;
        } catch (Throwable var4) {
            throw new UndeclaredThrowableException(var4);
        }
    }

    public final String toString() throws  {
        try {
            return (String)super.h.invoke(this, m2, (Object[])null);
        } catch (RuntimeException | Error var2) {
            throw var2;
        } catch (Throwable var3) {
            throw new UndeclaredThrowableException(var3);
        }
    }

    public final int hashCode() throws  {
        try {
            return (Integer)super.h.invoke(this, m0, (Object[])null);
        } catch (RuntimeException | Error var2) {
            throw var2;
        } catch (Throwable var3) {
            throw new UndeclaredThrowableException(var3);
        }
    }

    static {
        try {
            m1 = Class.forName("java.lang.Object").getMethod("equals", Class.forName("java.lang.Object"));
            m3 = Class.forName("proxy.jdk.UserService").getMethod("getUserByName", Class.forName("java.lang.String"));
            m2 = Class.forName("java.lang.Object").getMethod("toString");
            m0 = Class.forName("java.lang.Object").getMethod("hashCode");
        } catch (NoSuchMethodException var2) {
            throw new NoSuchMethodError(var2.getMessage());
        } catch (ClassNotFoundException var3) {
            throw new NoClassDefFoundError(var3.getMessage());
        }
    }
}

Invocation mechanism

  1. From proxy call
// JdkTest.java
proxy.getUserByName("bugpool");
  1. Proxy is of type $Proxy4, so enter the getUserByName method of $Proxy4
// $Proxy4.class
public final class $Proxy4 extends Proxy implements UserService {
    ...

    // Constructor, the object passed into the JdkCacheHandler class is the super.h property called below
    public $Proxy4(InvocationHandler var1) throws  {
        super(var1);
    }

    public final String getUserByName(String var1) throws  {
        try {
            /**
            *   invoke method calling the h attribute of the parent class
            *   In the following source code analysis, you will find that the h attribute is exactly this passed in by the JdkCacheHandler class createJDKProxy method
            *   Proxy.newProxyInstance(clazz.getClassLoader(), clazz.getInterfaces(), this);
            *   The this refers to the object of the JdkCacheHandler class, so the last call is the invoke method of JdkCacheHandler.
            */
            return (String)super.h.invoke(this, m3, new Object[]{var1});
        } catch (RuntimeException | Error var3) {
            throw var3;
        } catch (Throwable var4) {
            throw new UndeclaredThrowableException(var4);
        }
    }

    ...

    static {
        ...
        m1 = Class.forName("java.lang.Object").getMethod("equals", Class.forName("java.lang.Object"));
        m3 = Class.forName("proxy.jdk.UserService").getMethod("getUserByName", Class.forName("java.lang.String"));
        m2 = Class.forName("java.lang.Object").getMethod("toString");
        m0 = Class.forName("java.lang.Object").getMethod("hashCode");
        ...
    }
}
  1. Therefore, h.invok actually calls the invoke method of the JdkCacheHandler class
// JdkCacheHandler.java
public class JdkCacheHandler implements InvocationHandler {
    ...
    @Override
    public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {

        System.out.println("Before finding the database, find whether it exists in the cache:" + args[0]);
        // Trigger target class method
        Object result = method.invoke(target, args);
        System.out.printf("After finding the database, you will%s Add to cache\r\n", result);
        return result;
    }
}
  1. In method.invoke(target, args), method = getUserByName, target = UserServiceImpl object passed in by constructor, args = "bugpool"
// UserServiceImpl.java
public class UserServiceImpl implements UserService {
    @Override
    public String getUserByName(String name) {
        System.out.println("Query from database:" + name);
        return name;
    }
}

3, Source code analysis

principle

After understanding the call mechanism of Jdk dynamic proxy, all the core problems fall on how to generate the object proxy of $Proxy4 class? That is, the following code, here is an overview of the macro view of the source code. Before you start, review the running mechanism of java: 1. All. java files are compiled to generate. Class files; 2. Load the bytecode in. Class into the JVM through classLoad; 3. Run

// Create JDK agent
public Object createJDKProxy() {
    Class clazz = target.getClass();
    // Three parameters are required to create the JDK agent
    // Target class loader: used to load generated bytecode
    // Target class interface: used to generate bytecode, that is, the production of $Proxy can be completed only by the interface array
    // Proxy class object (i.e. itself): used to call back the invoke method and implement method enhancement
    return Proxy.newProxyInstance(clazz.getClassLoader(), clazz.getInterfaces(), this);
}
  1. Get all interfaces through clazz.getInterfaces(). You can generate bytecode similar to the following (note that the code is given below). After careful observation, you will find that the code generated by each interface method is the same. Only the M3 and parameters of (String)super.h.invoke(this, m3, new Object[]{var1} may be different. So in fact, to generate $Proxy bytecode, only the interface array is enough
public final String getUserByName(String var1) throws  {
        try {
            return (String)super.h.invoke(this, m3, new Object[]{var1});
        } catch (RuntimeException | Error var3) {
            throw var3;
        } catch (Throwable var4) {
            throw new UndeclaredThrowableException(var4);
        }
    }
  1. At this time, the bytecode of $Proxy4.class has been obtained, but the bytecode here has not been loaded into the JVM, so you need to call the class loader passed in by clazz.getClassLoader() to load and get the corresponding class, that is, $Proxy class
  2. Gets the constructor of the $Proxy class, which has an important parameter h
  3. Call the constructor of $Proxy class via reflection, cons.newInstance(new Object[]{h}); the h of the constructor is exactly the passed this, that is, the object of JdkCacheHandler class
  4. Put the $Proxy object obtained by reflection back

Source code analysis

  1. Trace the code from Proxy.newProxyInstance (Note: ① represents step 1 outlined above)
// JdkCacheHandler.java
// Create JDK agent
public Object createJDKProxy() {
    Class clazz = target.getClass();
    // Three parameters are required to create a JDK agent: target class loader, target class interface, and agent class object (i.e. itself)
    return Proxy.newProxyInstance(clazz.getClassLoader(), clazz.getInterfaces(), this);
}
  1. The tracking code is newProxyInstance. It should be noted that after ① ② the process is finished, and before ③ ④ the process is called, that is, class <? > cl = getproxyclass0 (loader, intfs); after that, the cl variable is only class, that is, $Proxy4, and no corresponding object is generated. There is no need to confuse class and object
// Proxy.java
// Loader class loader, all interfaces implemented by interfaces target class, h is the object of InvocationHandler class
public static Object newProxyInstance(ClassLoader loader,
                                          Class<?>[] interfaces,
                                          InvocationHandler h)
        throws IllegalArgumentException
    {
        // Verify that the InvocationHandler is empty
        Objects.requireNonNull(h);

        // Array of interfaces implemented by the target class
        final Class<?>[] intfs = interfaces.clone();
        // Security check
        final SecurityManager sm = System.getSecurityManager();
        if (sm != null) {
            checkProxyAccess(Reflection.getCallerClass(), loader, intfs);
        }

        /*
         * Look up or generate the designated proxy class.
         */
        // If there is a proxy class in the cache, get it directly. Otherwise, generate the proxy class
        // ① Step 2, the core code, that is, generate the proxy bytecode and load it here
        Class<?> cl = getProxyClass0(loader, intfs);

        /*
         * Invoke its constructor with the designated invocation handler.
         */
        try {
            if (sm != null) {
                checkNewProxyPermission(Reflection.getCallerClass(), cl);
            }

            // ③ Get the constructor from the generated proxy class
            // constructorParams = { InvocationHandler.class };
            final Constructor<?> cons = cl.getConstructor(constructorParams);
            final InvocationHandler ih = h;
            if (!Modifier.isPublic(cl.getModifiers())) {
                AccessController.doPrivileged(new PrivilegedAction<Void>() {
                    public Void run() {
                        cons.setAccessible(true);
                        return null;
                    }
                });
            }
            // ④ Call the constructor to instantiate the proxy object with InvocationHandler as a parameter
            return cons.newInstance(new Object[]{h});
        } catch (IllegalAccessException|InstantiationException e) {
            throw new InternalError(e.toString(), e);
        } catch (InvocationTargetException e) {
            Throwable t = e.getCause();
            if (t instanceof RuntimeException) {
                throw (RuntimeException) t;
            } else {
                throw new InternalError(t.toString(), t);
            }
        } catch (NoSuchMethodException e) {
            throw new InternalError(e.toString(), e);
        }
    }
  1. Trace class <? > CL = getproxyclass0 (loader, intfs);
// Proxy.java
private static Class<?> getProxyClass0(ClassLoader loader,
                                       Class<?>... interfaces) {
    if (interfaces.length > 65535) {
        throw new IllegalArgumentException("interface limit exceeded");
    }

    // If the proxy class defined by the given loader implementing
    // the given interfaces exists, this will simply return the cached copy;
    // otherwise, it will create the proxy class via the ProxyClassFactory
    // If a proxy class that implements the corresponding interface already exists in the classloader, the proxy class in the cache will be returned directly
    // Otherwise, create a new proxy class through ProxyClassFactory
    return proxyClassCache.get(loader, interfaces);
}
  1. Trace proxyClassCache.get(loader, interfaces); (Note: Jdk dynamic agent caches the generated and loaded proxy classes to improve performance, and the cache related code is not the focus we care about, so we can skip the related code.) in this code, we mainly care about V value = supplier.get(); the essence of supplier is factory, through new Factory(key, parameter, subKey, valuesMap) create
// WeakCache.java
//K and P are generics in the WeakCache definition. key is class loader and parameter is interface class array
public V get(K key, P parameter) {
        // Check whether the interface array is empty
        Objects.requireNonNull(parameter);

        expungeStaleEntries();

        Object cacheKey = CacheKey.valueOf(key, refQueue);

        // lazily install the 2nd level valuesMap for the particular cacheKey
        ConcurrentMap<Object, Supplier<V>> valuesMap = map.get(cacheKey);
        if (valuesMap == null) {
            ConcurrentMap<Object, Supplier<V>> oldValuesMap
                = map.putIfAbsent(cacheKey,
                                  valuesMap = new ConcurrentHashMap<>());
            if (oldValuesMap != null) {
                valuesMap = oldValuesMap;
            }
        }

        // create subKey and retrieve the possible Supplier<V> stored by that
        // subKey from valuesMap
        Object subKey = Objects.requireNonNull(subKeyFactory.apply(key, parameter));
        //supplier is obtained through sub key, which is essentially factory
        Supplier<V> supplier = valuesMap.get(subKey);
        Factory factory = null;

        while (true) {
            if (supplier != null) {
                // supplier might be a Factory or a CacheValue<V> instance
                // ① ② the steps are all here. If the supplier is not empty, directly call the get method to return the proxy class
                V value = supplier.get();
                if (value != null) {
                    return value;
                }
            }
            // else no supplier in cache
            // or a supplier that returned null (could be a cleared CacheValue
            // or a Factory that wasn't successful in installing the CacheValue)

            // lazily construct a Factory
            if (factory == null) {
                // Create the corresponding factory. This code is in the loop. The next time supplier.get() will get the proxy class and exit the loop
                factory = new Factory(key, parameter, subKey, valuesMap);
            }

            if (supplier == null) {
                supplier = valuesMap.putIfAbsent(subKey, factory);
                if (supplier == null) {
                    // successfully installed Factory
                    // Assign to supplier
                    supplier = factory;
                }
                // else retry with winning supplier
            } else {
                if (valuesMap.replace(subKey, supplier, factory)) {
                    // successfully replaced
                    // cleared CacheEntry / unsuccessful Factory
                    // with our Factory
                    supplier = factory;
                } else {
                    // retry with current supplier
                    supplier = valuesMap.get(subKey);
                }
            }
        }
    }
  1. Trace V value = supplier.get(); that is, the get method of the Factory class. Most of the work here is still doing the verification and caching. We only care about the core logic valueFactory.apply(key, parameter); where valueFactory is the ProxyClassFactory passed in the previous step
// Factory.java    
public synchronized V get() { // serialize access
        // re-check
        // Check again if the supplier is the current object
        Supplier<V> supplier = valuesMap.get(subKey);
        if (supplier != this) {
            // something changed while we were waiting:
            // might be that we were replaced by a CacheValue
            // or were removed because of failure ->
            // return null to signal WeakCache.get() to retry
            // the loop
            return null;
        }
        // else still us (supplier == this)

        // create new value
        V value = null;
        try {
            // valueFactory is the new ProxyClassFactory()
            // ① ② step, core logic, call valueFactory.apply to generate the corresponding proxy class and load
            value = Objects.requireNonNull(valueFactory.apply(key, parameter));
        } finally {
            if (value == null) { // remove us on failure
                valuesMap.remove(subKey, this);
            }
        }
        // the only path to reach here is with non-null value
        assert value != null;

        // wrap value with CacheValue (WeakReference)
        CacheValue<V> cacheValue = new CacheValue<>(value);

        // put into reverseMap
        reverseMap.put(cacheValue, Boolean.TRUE);

        // try replacing us with CacheValue (this should always succeed)
        if (!valuesMap.replace(subKey, this, cacheValue)) {
            throw new AssertionError("Should not reach here");
        }

        // successfully replaced us with new CacheValue -> return the value
        // wrapped by it
        return value;
    }
  1. Track core logic
    • Jdk dynamic Proxy pieced together the full class name of $Proxy: com.sun.proxy.$proxy0.class
    • ③ Production bytecode byte [] proxyclassfile = proxygenerator.generateproxyclass (proxyname, interfaces, accessflags); it can be seen that the dynamic agent of Jdk needs the interface array to generate bytecode, which is the reason why the interface must be implemented at the beginning of the article. At the same time, it can be seen from the parameters that only the interface array is needed to generate bytecode, and no other information is needed. In fact, the implementation principle can also be guessed. Through traversing all interface methods, the dynamic agent of Jdk generates the corresponding return (String)super.h.invoke(this, m0~n, new Object[]{var1}) code for the method
    • ④ Load bytecode: obtain the byte array of bytecode in ③, and then call classLoader to read all bytecode into JVM
// ProxyClassFactory.java
private static final class ProxyClassFactory
        implements BiFunction<ClassLoader, Class<?>[], Class<?>>
    {
        // prefix for all proxy class names
        // Proxy class name prefix
        private static final String proxyClassNamePrefix = "$Proxy";

        // next number to use for generation of unique proxy class names
        // Proxy counter
        private static final AtomicLong nextUniqueNumber = new AtomicLong();

        @Override
        public Class<?> apply(ClassLoader loader, Class<?>[] interfaces) {

            Map<Class<?>, Boolean> interfaceSet = new IdentityHashMap<>(interfaces.length);
            // Verify agent class interface
            for (Class<?> intf : interfaces) {
                /*
                 * Verify that the class loader resolves the name of this
                 * interface to the same Class object.
                 */
                Class<?> interfaceClass = null;
                try {
                    interfaceClass = Class.forName(intf.getName(), false, loader);
                } catch (ClassNotFoundException e) {
                }
                if (interfaceClass != intf) {
                    throw new IllegalArgumentException(
                        intf + " is not visible from class loader");
                }
                /*
                 * Verify that the Class object actually represents an
                 * interface.
                 */
                if (!interfaceClass.isInterface()) {
                    throw new IllegalArgumentException(
                        interfaceClass.getName() + " is not an interface");
                }
                /*
                 * Verify that this interface is not a duplicate.
                 */
                if (interfaceSet.put(interfaceClass, Boolean.TRUE) != null) {
                    throw new IllegalArgumentException(
                        "repeated interface: " + interfaceClass.getName());
                }
            }

            // Agent class package name
            String proxyPkg = null;     // package to define proxy class in
            int accessFlags = Modifier.PUBLIC | Modifier.FINAL;

            /*
             * Record the package of a non-public proxy interface so that the
             * proxy class will be defined in the same package.  Verify that
             * all non-public proxy interfaces are in the same package.
             */
            // When the interface modifier is public, all packages can use
            // When the interface is non public, the generated proxy class and interface must be under the same package as the non public interface
            // If all non public interfaces are under the same package, the generated proxy class will be placed under the same package of non public interfaces
            // If there are multiple interfaces other than public and they are in different packages, an exception will be thrown
            for (Class<?> intf : interfaces) {
                int flags = intf.getModifiers();
                if (!Modifier.isPublic(flags)) {
                    accessFlags = Modifier.FINAL;
                    String name = intf.getName();
                    int n = name.lastIndexOf('.');
                    String pkg = ((n == -1) ? "" : name.substring(0, n + 1));
                    if (proxyPkg == null) {
                        proxyPkg = pkg;
                    } else if (!pkg.equals(proxyPkg)) {
                        throw new IllegalArgumentException(
                            "non-public interfaces from different packages");
                    }
                }
            }

            if (proxyPkg == null) {
                // if no non-public proxy interfaces, use com.sun.proxy package
                // If all interfaces are public, the proxy class is set to com.sun.proxy package by default
                proxyPkg = ReflectUtil.PROXY_PACKAGE + ".";
            }

            /*
             * Choose a name for the proxy class to generate.
             */
            // Generate counter, for example $proxy0~n
            long num = nextUniqueNumber.getAndIncrement();
            // Proxy class name, com.sun.proxy.$proxy0.class
            String proxyName = proxyPkg + proxyClassNamePrefix + num;

            /*
             * Generate the specified proxy class.
             */
            // ③ Generate proxy bytecode
            byte[] proxyClassFile = ProxyGenerator.generateProxyClass(
                proxyName, interfaces, accessFlags);
            try {
                // ④ Use the classLoader passed in to load the proxy class bytecode into the JVM
                return defineClass0(loader, proxyName,
                                    proxyClassFile, 0, proxyClassFile.length);
            } catch (ClassFormatError e) {
                /*
                 * A ClassFormatError here means that (barring bugs in the
                 * proxy class generation code) there was some other
                 * invalid aspect of the arguments supplied to the proxy
                 * class creation (such as virtual machine limitations
                 * exceeded).
                 */
                throw new IllegalArgumentException(e.toString());
            }
        }
    }
Published 29 original articles, won praise 5, visited 6331
Private letter follow

Tags: Java JDK Database jvm

Posted on Sat, 14 Mar 2020 00:27:03 -0700 by nightowl