Deadlock in Java class loading? turn

source: Will Java class load deadlock? What's the situation?

 

1, Foreword

First, post a test code. You can guess what the execution result will be

import java.util.concurrent.TimeUnit;

public class TestClassLoading {
    public static class A{
        static {
            System.out.println("class A init");
            try {
                TimeUnit.SECONDS.sleep(1);
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
            new B();
        }

        public static void test() {
            System.out.println("aaa");
        }
    }

    public static class B{
        static {
            System.out.println("class B init");
            new A();
        }

        public static void test() {
            System.out.println("bbb");
        }
    }
    public static void main(String[] args) {
        new Thread(() -> A.test()).start();
        new Thread(() -> B.test()).start();
    }
}

 

I don't know. Have you guessed right? The actual execution result will be as follows:

 

2, Cause analysis

Here, at the beginning, we analyzed that it is related to new, but the following code is completely consistent with the above results, and the suspicion of new can be basically ruled out:

public class TestClassLoadingNew {
    public static class A{
        static {
            System.out.println("class A init");
            try {
                TimeUnit.SECONDS.sleep(1);
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
            B.test();
        }

        public static void test() {
            System.out.println("aaa");
        }
    }

    public static class B{
        static {
            System.out.println("class B init");
            A.test();
        }

        public static void test() {
            System.out.println("bbb");
        }
    }
    public static void main(String[] args) {
        new Thread(() -> A.test()).start();
        new Thread(() -> B.test()).start();
    }
}

Here, the root cause of the problem is actually:

When the classloader initializes a class, it will lock the current class, and then execute the static initialization block of the class.

So, what happens above:

1. Thread 1: Class A locks class A, and then executes the static initialization block of the class (embodied as < clinit > function in the stack). class B is found to be used, so load B;

2. Thread 2: Class B locks class B, and then executes the static initialization block of the class (embodied as < clinit > function in the stack). class A is found to be used, so load a;

3. Deadlock occurs.

 

Experienced students have no fear of deadlock, because we have artifact, jstack. Jstack plus the - l parameter can print out the lock information held by each thread. (on windows, jconsole can be used directly, and deadlock detection can also be performed):

"Thread-1" #15 prio=5 os_prio=0 tid=0x000000002178a000 nid=0x2df8 in Object.wait() [0x0000000021f4e000]
   java.lang.Thread.State: RUNNABLE
        at com.dmtest.netty_learn.TestClassLoading$B.<clinit>(TestClassLoading.java:32)
        at com.dmtest.netty_learn.TestClassLoading.lambda$main$1(TestClassLoading.java:42)
        at com.dmtest.netty_learn.TestClassLoading$$Lambda$2/736709391.run(Unknown Source)
        at java.lang.Thread.run(Thread.java:748)

   Locked ownable synchronizers:
        - None

"Thread-0" #14 prio=5 os_prio=0 tid=0x0000000021787800 nid=0x2618 in Object.wait() [0x00000000213be000]
   java.lang.Thread.State: RUNNABLE
        at com.dmtest.netty_learn.TestClassLoading$A.<clinit>(TestClassLoading.java:21)
        at com.dmtest.netty_learn.TestClassLoading.lambda$main$0(TestClassLoading.java:41)
        at com.dmtest.netty_learn.TestClassLoading$$Lambda$1/611437735.run(Unknown Source)
        at java.lang.Thread.run(Thread.java:748)

   Locked ownable synchronizers:
        - None

Here, one of the strange reasons is that there is a deadlock between these two threads. Why is it not shown?

Because this is the internal lock of the jvm, the jconsole and jstack are all invalid.

 

3, Explore the JVM together

1. Single step tracking

class loading is done by classloader, and part of the work is done at the jvm level. We can see that in the definition of java.lang.classloader ා defineclass1:

 

These are all local methods.

Its actual implementation is in / home/ckl/openjdk-jdk8u/jdk/src/share/native/java/lang/ClassLoader.c,

JNIEXPORT jclass JNICALL
Java_java_lang_ClassLoader_defineClass1(JNIEnv *env,
                                        jobject loader,
                                        jstring name,
                                        jbyteArray data,
                                        jint offset,
                                        jint length,
                                        jobject pd,
                                        jstring source)
{
    jbyte *body;
    char *utfName;
    jclass result = 0;
    char buf[128];
    char* utfSource;
    char sourceBuf[1024];

    if (data == NULL) {
        JNU_ThrowNullPointerException(env, 0);
        return 0;
    }

    /* Work around 4153825. malloc crashes on Solaris when passed a
     * negative size.
     */
    if (length < 0) {
        JNU_ThrowArrayIndexOutOfBoundsException(env, 0);
        return 0;
    }

    body = (jbyte *)malloc(length);

    if (body == 0) {
        JNU_ThrowOutOfMemoryError(env, 0);
        return 0;
    }

    (*env)->GetByteArrayRegion(env, data, offset, length, body);

    if ((*env)->ExceptionOccurred(env))
        goto free_body;

    if (name != NULL) {
        utfName = getUTF(env, name, buf, sizeof(buf));
        if (utfName == NULL) {
            goto free_body;
        }
        VerifyFixClassname(utfName);
    } else {
        utfName = NULL;
    }

    if (source != NULL) {
        utfSource = getUTF(env, source, sourceBuf, sizeof(sourceBuf));
        if (utfSource == NULL) {
            goto free_utfName;
        }
    } else {
        utfSource = NULL;
    }
    result = JVM_DefineClassWithSource(env, utfName, loader, body, length, pd, utfSource);

    if (utfSource && utfSource != sourceBuf)
        free(utfSource);

 free_utfName:
    if (utfName && utfName != buf)
        free(utfName);

 free_body:
    free(body);
    return result;
}

You can follow the code marked in red. Let's take a look at it. The implementation of this method is in / home/ckl/openjdk-jdk8u/hotspot/src/share/vm/prims/jvm.cpp,

JVM_ENTRY(jclass, JVM_DefineClassWithSource(JNIEnv *env, const char *name, jobject loader, const jbyte *buf, jsize len, jobject pd, const char *source))
  JVMWrapper2("JVM_DefineClassWithSource %s", name);

  return jvm_define_class_common(env, name, loader, buf, len, pd, source, true, THREAD);
JVM_END

The implementation of JVM? Define? Class? Common is still in jvm.cpp,

// common code for JVM_DefineClass() and JVM_DefineClassWithSource()
// and JVM_DefineClassWithSourceCond()
static jclass jvm_define_class_common(JNIEnv *env, const char *name,
                                      jobject loader, const jbyte *buf,
                                      jsize len, jobject pd, const char *source,
                                      jboolean verify, TRAPS) {
  if (source == NULL)  source = "__JVM_DefineClass__";

  assert(THREAD->is_Java_thread(), "must be a JavaThread");
  JavaThread* jt = (JavaThread*) THREAD;

  PerfClassTraceTime vmtimer(ClassLoader::perf_define_appclass_time(),
                             ClassLoader::perf_define_appclass_selftime(),
                             ClassLoader::perf_define_appclasses(),
                             jt->get_thread_stat()->perf_recursion_counts_addr(),
                             jt->get_thread_stat()->perf_timers_addr(),
                             PerfClassTraceTime::DEFINE_CLASS);

  if (UsePerfData) {
    ClassLoader::perf_app_classfile_bytes_read()->inc(len);
  }

  // Since exceptions can be thrown, class initialization can take place
  // if name is NULL no check for class name in .class stream has to be made.
  TempNewSymbol class_name = NULL;
  if (name != NULL) {
    const int str_len = (int)strlen(name);
    if (str_len > Symbol::max_length()) {
      // It's impossible to create this class;  the name cannot fit
      // into the constant pool.
      THROW_MSG_0(vmSymbols::java_lang_NoClassDefFoundError(), name);
    }
    class_name = SymbolTable::new_symbol(name, str_len, CHECK_NULL);
  }

  ResourceMark rm(THREAD);
  ClassFileStream st((u1*) buf, len, (char *)source);
  Handle class_loader (THREAD, JNIHandles::resolve(loader));
  if (UsePerfData) {
    is_lock_held_by_thread(class_loader,
                           ClassLoader::sync_JVMDefineClassLockFreeCounter(),
                           THREAD);
  }
  Handle protection_domain (THREAD, JNIHandles::resolve(pd));
  Klass* k = SystemDictionary::resolve_from_stream(class_name, class_loader,
                                                     protection_domain, &st,
                                                     verify != 0,
                                                     CHECK_NULL);

  if (TraceClassResolution && k != NULL) {
    trace_class_resolution(k);
  }

  return (jclass) JNIHandles::make_local(env, k->java_mirror());
}

The implementation of resolve from stream is in SystemDictionary class. Let's look at the following:

Klass* SystemDictionary::resolve_from_stream(Symbol* class_name,
                                             Handle class_loader,
                                             Handle protection_domain,
                                             ClassFileStream* st,
                                             bool verify,
                                             TRAPS) {

  // Classloaders that support parallelism, e.g. bootstrap classloader,
  // or all classloaders with UnsyncloadClass do not acquire lock here
  bool DoObjectLock = true;
  if (is_parallelCapable(class_loader)) {
    DoObjectLock = false;
  }

  ClassLoaderData* loader_data = register_loader(class_loader, CHECK_NULL);

  // Make sure we are synchronized on the class loader before we proceed
  Handle lockObject = compute_loader_lock_object(class_loader, THREAD);
  check_loader_lock_contention(lockObject, THREAD);
  ObjectLocker ol(lockObject, THREAD, DoObjectLock);

  TempNewSymbol parsed_name = NULL;

  // Parse the stream. Note that we do this even though this klass might
  // already be present in the SystemDictionary, otherwise we would not
  // throw potential ClassFormatErrors.
  //
  // Note: "name" is updated.

  instanceKlassHandle k = ClassFileParser(st).parseClassFile(class_name,
                                                             loader_data,
                                                             protection_domain,
                                                             parsed_name,
                                                             verify,
                                                             THREAD);

  const char* pkg = "java/";
  size_t pkglen = strlen(pkg);
  if (!HAS_PENDING_EXCEPTION &&
      !class_loader.is_null() &&
      parsed_name != NULL &&
      parsed_name->utf8_length() >= (int)pkglen &&
      !strncmp((const char*)parsed_name->bytes(), pkg, pkglen)) {
    // It is illegal to define classes in the "java." package from
    // JVM_DefineClass or jni_DefineClass unless you're the bootclassloader
    ResourceMark rm(THREAD);
    char* name = parsed_name->as_C_string();
    char* index = strrchr(name, '/');
    assert(index != NULL, "must be");
    *index = '\0'; // chop to just the package name
    while ((index = strchr(name, '/')) != NULL) {
      *index = '.'; // replace '/' with '.' in package name
    }
    const char* fmt = "Prohibited package name: %s";
    size_t len = strlen(fmt) + strlen(name);
    char* message = NEW_RESOURCE_ARRAY(char, len);
    jio_snprintf(message, len, fmt, name);
    Exceptions::_throw_msg(THREAD_AND_LOCATION,
      vmSymbols::java_lang_SecurityException(), message);
  }

  if (!HAS_PENDING_EXCEPTION) {
    assert(parsed_name != NULL, "Sanity");
    assert(class_name == NULL || class_name == parsed_name, "name mismatch");
    // Verification prevents us from creating names with dots in them, this
    // asserts that that's the case.
    assert(is_internal_format(parsed_name),
           "external class name format used internally");

    // Add class just loaded
    // If a class loader supports parallel classloading handle parallel define requests
    // find_or_define_instance_class may return a different InstanceKlass
    if (is_parallelCapable(class_loader)) {
      k = find_or_define_instance_class(class_name, class_loader, k, THREAD);
    } else {
      define_instance_class(k, THREAD);
    }
  }
96 
  return k();
}

In the above method, there are several noteworthy points:

1: Lines 18-20 are locked. Lines 18 get the lock object. Here is the current class loader (as can be seen from the comments). Line 20 is the syntax of locking

2: Line 37-60, here is to determine whether the package name of the class to be loaded starts with java. Classes starting with java are illegal and cannot be loaded

3: Line 76, define "instance" class (k, thread); follow up

 

Next, let's take a look at the implementation of define ﹣ instance ﹣ class:

void SystemDictionary::define_instance_class(instanceKlassHandle k, TRAPS) {

  ClassLoaderData* loader_data = k->class_loader_data();
  Handle class_loader_h(THREAD, loader_data->class_loader());

  for (uintx it = 0; it < GCExpandToAllocateDelayMillis; it++){}

 // for bootstrap and other parallel classloaders don't acquire lock,
 // use placeholder token
 // If a parallelCapable class loader calls define_instance_class instead of
 // find_or_define_instance_class to get here, we have a timing
 // hole with systemDictionary updates and check_constraints
 if (!class_loader_h.is_null() && !is_parallelCapable(class_loader_h)) {
    assert(ObjectSynchronizer::current_thread_holds_lock((JavaThread*)THREAD,
         compute_loader_lock_object(class_loader_h, THREAD)),
         "define called without lock");
  }

  // Check class-loading constraints. Throw exception if violation is detected.
  // Grabs and releases SystemDictionary_lock
  // The check_constraints/find_class call and update_dictionary sequence
  // must be "atomic" for a specific class/classloader pair so we never
  // define two different instanceKlasses for that class/classloader pair.
  // Existing classloaders will call define_instance_class with the
  // classloader lock held
  // Parallel classloaders will call find_or_define_instance_class
  // which will require a token to perform the define class
  Symbol*  name_h = k->name();
  unsigned int d_hash = dictionary()->compute_hash(name_h, loader_data);
  int d_index = dictionary()->hash_to_index(d_hash);
  check_constraints(d_index, d_hash, k, class_loader_h, true, CHECK);

  // Register class just loaded with class loader (placed in Vector)
  // Note we do this before updating the dictionary, as this can
  // fail with an OutOfMemoryError (if it does, we will *not* put this
  // class in the dictionary and will not update the class hierarchy).
  // JVMTI FollowReferences needs to find the classes this way.
  if (k->class_loader() != NULL) {
    methodHandle m(THREAD, Universe::loader_addClass_method());
    JavaValue result(T_VOID);
    JavaCallArguments args(class_loader_h);
    args.push_oop(Handle(THREAD, k->java_mirror()));
    JavaCalls::call(&result, m, &args, CHECK);
  }

  // Add the new class. We need recompile lock during update of CHA.
  {
    unsigned int p_hash = placeholders()->compute_hash(name_h, loader_data);
    int p_index = placeholders()->hash_to_index(p_hash);

    MutexLocker mu_r(Compile_lock, THREAD);

    // Add to class hierarchy, initialize vtables, and do possible
    // deoptimizations.
    add_to_hierarchy(k, CHECK); // No exception, but can block

    // Add to systemDictionary - so other classes can see it.
    // Grabs and releases SystemDictionary_lock
    update_dictionary(d_index, d_hash, p_index, p_hash,
                      k, class_loader_h, THREAD);
  }
  k->eager_initialize(THREAD);

  // notify jvmti
  if (JvmtiExport::should_post_class_load()) {
      assert(THREAD->is_Java_thread(), "thread->is_Java_thread()");
      JvmtiExport::post_class_load((JavaThread *) THREAD, k());

  }

}

Here, since in our case, it is class A that deadlocks during initialization, we focus on the sixty-second line, eager initialize:

void InstanceKlass::eager_initialize(Thread *thread) {
  if (!EagerInitialization) return;

  if (this->is_not_initialized()) {
    // abort if the the class has a class initializer
    if (this->class_initializer() != NULL) return;

    // abort if it is java.lang.Object (initialization is handled in genesis)
    Klass* super = this->super();
    if (super == NULL) return;

    // abort if the super class should be initialized
    if (!InstanceKlass::cast(super)->is_initialized()) return;

    // call body to expose the this pointer
    instanceKlassHandle this_oop(thread, this);
    eager_initialize_impl(this_oop);
  }
}

We then go to eager ﹣ initialize ﹣ impl, which goes to InstanceKlass:

void InstanceKlass::eager_initialize_impl(instanceKlassHandle this_oop) {
  EXCEPTION_MARK;
  oop init_lock = this_oop->init_lock();
  ObjectLocker ol(init_lock, THREAD, init_lock != NULL);

  // abort if someone beat us to the initialization
  if (!this_oop->is_not_initialized()) return;  // note: not equivalent to is_initialized()

  ClassState old_state = this_oop->init_state();
  link_class_impl(this_oop, true, THREAD);
  if (HAS_PENDING_EXCEPTION) {
    CLEAR_PENDING_EXCEPTION;
    // Abort if linking the class throws an exception.

    // Use a test to avoid redundantly resetting the state if there's
    // no change.  Set_init_state() asserts that state changes make
    // progress, whereas here we might just be spinning in place.
    if( old_state != this_oop->_init_state )
      this_oop->set_init_state (old_state);
  } else {
    // linking successfull, mark class as initialized
    this_oop->set_init_state (fully_initialized);
    this_oop->fence_and_clear_init_lock();
    // trace
    if (TraceClassInitialization) {
      ResourceMark rm(THREAD);
      tty->print_cr("[Initialized %s without side effects]", this_oop->external_name());
    }
  }
}

Here, we focus on lines 3 and 4:

1. Line 3, obtain the initialization lock;

2. Line 4, lock

 

2. Acquire and lock the initialization lock

Here, we first obtain the lock operation,

oop InstanceKlass::init_lock() const {
  // return the init lock from the mirror
  oop lock = java_lang_Class::init_lock(java_mirror());
  // Prevent reordering with any access of initialization state
  OrderAccess::loadload();
  assert((oop)lock != NULL || !is_not_initialized(), // initialized or in_error state
         "only fully initialized state can have a null lock");
  return lock;
}

The java_mirror() method returns the following fields in the Klass class:

// java/lang/Class instance mirroring this class
oop       _java_mirror;

Then look at the init lock method:

oop java_lang_Class::init_lock(oop java_class) {
   assert(_init_lock_offset != 0, "must be set");
   return java_class->obj_field(_init_lock_offset);
 }

Here, it should be to get a field in the java_class we passed in, which serves as init_lock. (personal level is limited, please correct)

 

The following is the statement of lock operation:

ObjectLocker ol(init_lock, THREAD, init_lock != NULL);
/ ObjectLocker enforced balanced locking and can never thrown an
// IllegalMonitorStateException. However, a pending exception may
// have to pass through, and we must also be able to deal with
// asynchronous exceptions. The caller is responsible for checking
// the threads pending exception if needed.
// doLock was added to support classloading with UnsyncloadClass which
// requires flag based choice of locking the classloader lock.
class ObjectLocker : public StackObj {
 private:
  Thread*   _thread;
  Handle    _obj;
  BasicLock _lock;
  bool      _dolock;   // default true
 public:
  ObjectLocker(Handle obj, Thread* thread, bool doLock = true);
// -----------------------------------------------------------------------------
// Internal VM locks on java objects
// standard constructor, allows locking failures
ObjectLocker::ObjectLocker(Handle obj, Thread* thread, bool doLock) {
  _dolock = doLock;
  _thread = thread;
  _obj = obj;

  if (_dolock) {
    TEVENT (ObjectLocker) ;

    ObjectSynchronizer::fast_enter(_obj, &_lock, false, _thread);
  }
}

Next, go to synchronizer.cpp,

// -----------------------------------------------------------------------------
//  Fast Monitor Enter/Exit
// This the fast monitor enter. The interpreter and compiler use
// some assembly copies of this code. Make sure update those code
// if the following function is changed. The implementation is
// extremely sensitive to race condition. Be careful.

void ObjectSynchronizer::fast_enter(Handle obj, BasicLock* lock, bool attempt_rebias, TRAPS) {
 if (UseBiasedLocking) {
    if (!SafepointSynchronize::is_at_safepoint()) {
      BiasedLocking::Condition cond = BiasedLocking::revoke_and_rebias(obj, attempt_rebias, THREAD);
      if (cond == BiasedLocking::BIAS_REVOKED_AND_REBIASED) {
        return;
      }
    } else {
      assert(!attempt_rebias, "can not rebias toward VM thread");
      BiasedLocking::revoke_at_safepoint(obj);
    }
    assert(!obj->mark()->has_bias_pattern(), "biases should be revoked by now");
 }

 slow_enter (obj, lock, THREAD) ;
}

The above will determine whether to use the biased lock. If not, go to "slow" enter.

// -----------------------------------------------------------------------------
// Interpreter/Compiler Slow Case
// This routine is used to handle interpreter/compiler slow case
// We don't need to use fast path here, because it must have been
// failed in the interpreter/compiler code.
void ObjectSynchronizer::slow_enter(Handle obj, BasicLock* lock, TRAPS) {
  markOop mark = obj->mark();
  assert(!mark->has_bias_pattern(), "should not see bias pattern here");

  if (mark->is_neutral()) {
    // Anticipate successful CAS -- the ST of the displaced mark must
    // be visible <= the ST performed by the CAS.
    lock->set_displaced_header(mark);
    if (mark == (markOop) Atomic::cmpxchg_ptr(lock, obj()->mark_addr(), mark)) {
      TEVENT (slow_enter: release stacklock) ;
      return ;
    }
    // Fall through to inflate() ...
  } else
  if (mark->has_locker() && THREAD->is_lock_owned((address)mark->locker())) {
    assert(lock != mark->locker(), "must not re-lock the same lock");
    assert(lock != (BasicLock*)obj->mark(), "don't relock with same BasicLock");
    lock->set_displaced_header(NULL);
    return;
  }


  // The object header will never be displaced to this lock,
  // so it does not matter what the value is, except that it
  // must be non-zero to avoid looking like a re-entrant lock,
  // and must not look locked either.
  lock->set_displaced_header(markOopDesc::unused_mark());
  ObjectSynchronizer::inflate(THREAD, obj())->enter(THREAD);
}

The code here, combined with the comments, can be roughly seen that the front part is a lightweight lock, which will not be expanded here, and the lock can be written separately. Interested readers can read by themselves.

 

4: How to solve the deadlock problem of class loading?


You can explicitly load these classes with forName at the beginning of the main thread, so that class loading becomes serial loading in the main thread, and the problem is solved:

public static void main(String[] args) throws ClassNotFoundException{
        
        Class.forName("com.**.**.A");
        Class.forName("com.**.**.B");

        new Thread(() -> A.test(), "thread-1").start();
        new Thread(() -> B.test(), "thread-2").start();
    }

 

5, Summary

Let's make a conclusion here. During class initialization, class will be locked and class initialization will be performed. If circular dependency occurs at this time, it will lead to deadlock.

Tags: Java jvm Lambda Windows

Posted on Sun, 17 May 2020 04:21:16 -0700 by foobar