Bridging the Gap between Coroutines, JVM Threads, and Concurrency Problems

How often do you hear the phrase “coroutines are lightweight threads”? Does this description help you substantially in understanding coroutines? Probably not. Reading this article, you’ll learn more about how coroutines actually execute in the JVM, how coroutines relate to threads, and the concurrency problems that are inevitable when using the JVM threading model.

Coroutines and JVM threads

Coroutines are designed to simplify the code that performs asynchronous operations. The essence of JVA-based coroutines is that lambda blocks passed to the coroutine builder end up being executed on a specific JVM thread. Consider this simple Fibonacci sequence:

// Computes the 10th Fibonacci number coroutine in the background thread
someScope.launch(Dispatchers.Default) {
    val fibonacci10 = synchronousFibonacci(10)
    saveFibonacciInMemory(10, fibonacci10)
}
private fun synchronousFibonacci(n: Long): Long { / *... * / }
Copy the code

The above block of asynchronous coroutine code performs a synchronous, blocking Fibonacci calculation and saves the result to memory. This code block is dispatched by the thread pool managed by the coroutine library (via the dispatchers.default configuration) and is executed at some future point (depending on the thread pool policy) by threads in the thread pool.

Note that the above code executes in one thread because there is no suspend. Coroutines can be executed in a different thread if the logic executed is moved to a different Dispatcher, or if a block of code may yield/suspend in a scheduler that uses thread pools.

Again, if there is no coroutine, the above logic can be manually executed using threads as follows:

// Create a pool of four threads
val executorService = Executors.newFixedThreadPool(4)
// Schedule and execute the following code on threads in the thread pool
executorService.execute {
    val fibonacci10 = synchronousFibonacci(10)
    saveFibonacciInMemory(10, fibonacci10)
}
Copy the code

While it’s possible to manage thread pools manually, coroutines are recommended for asynchronous programming in Android given their built-in support for cancellation, easier error handling, use of Structured Concurrency, which reduces the likelihood of memory leaks, and support from the Jetpack library.

The principle behind it

What happens when you start creating coroutines to execute in threads? When creating coroutines using the standard coroutine builder, you can specify code to be executed at a specific CoroutineDispatcher, which by Default will use dispatchers.default.

The CoroutineDispatcher is responsible for distributing execution of coroutines to JVM threads. Here’s how it works: when using CoroutineDispatcher, it intercepts coroutines with an interceptContinuation, which wraps the Continuation in a DispatchedContinuation. This is possible because CoroutineDispatcher implements the ContinuationInterceptor interface.

If you’ve read my article on how coroutines work, you already know that the compiler creates a state machine, and that the state machine’s information (what you need to do in the next step) is stored in a Continuation object.

If you need to execute a Continuation in another Dispatcher, the resumeWith method of DispatchedContinuation is responsible for assigning it to the appropriate coroutine!

Also, DispatchedContinuation is DispatchedTask, which in the JVM is a Runnable object that can be run on a JVM thread! Isn’t that cool? When CoroutineDispatcher is specified, the coroutine is converted to DispatchedTask, which is executed as a Runnable on the JVM thread!

How is the dispatch method called when the coroutine is created? To create coroutines using the standard coroutine builder, you can specify the coroutine as a start parameter of type CoroutineStart. For example, you can use coroutinestart.lazy to configure it to start only when needed. By DEFAULT, coroutine execution is scheduled according to its CoroutineDispatcher using coroutine.

A diagram of how a block of code in a coroutine is ultimately executed in a thread

Scheduler and thread pool

You can use the Executor. AsCoroutineDispatcher () extension function converts coroutines CoroutineDispatcher, in your app performs coroutines thread pool. You can also use the default Dispatchers in the coroutine library.

You can see how dispatchers.default is initialized in the createDefaultDispatcher method. DefaultScheduler is used by default. If you look at the implementation of Dispatchers.io, it will also use DefaultScheduler and allow at least 64 threads to be created as needed. Dispatchers.Default and dispatchers. IO are implicitly linked together because they use the same thread pool. Now let’s look at the runtime overhead of using different Dispatcher calls to withContext.

Thread and withContext performance

In a JVM, switching between threads imposes some runtime overhead if more threads are created than the number of CPU cores available. Context switching is not cheap! The operating system needs to save and restore execution context, and the CPU needs to spend time scheduling threads rather than running actual APP work. In addition, context switches can occur if the code the thread is running is blocked. If this is the case for threads, is there a performance penalty for using withContext with different Dispatchers?

Fortunately, as you might expect, thread pools manage these complex scenarios for us and try to optimize the work being performed as much as possible (which is why it’s better to perform work on a thread pool than manually in a thread). Coroutines also benefit from this (because they are scheduled in a thread pool)! Most importantly, coroutines do not block threads, but suspend work! Even more efficient!

By default, CoroutineScheduler is the thread pool used in JVM implementations to allocate dispatched coroutines to worker threads in the most efficient way. Since dispatchers.default and dispatchers.io use the same thread pool, the switching between them is optimized to avoid thread switching as much as possible. The coroutine library optimizes these calls, remaining on the same dispatcher and thread, and following a fast-path.

Since dispatchers. Main is usually a different thread in the UI app, switching between dispatchers. Default and dispatchers. Main in the coroutine does not have a huge performance cost, Because coroutines simply hang (that is, stop execution in one thread) and are scheduled to be executed in another thread.

Concurrency problems in coroutines

Because scheduling on different threads is so simple, coroutines do make asynchronous programming easier. On the other hand, this simplicity can be a double-edged sword: since coroutines run on the JVM threading model, they can’t simply get rid of the concurrency problems that the threading model presents. Therefore, you must take care to avoid concurrency problems.

Over the years, good practices such as immutability have mitigated some of the thread-related problems you might encounter. However, there are some scenarios where immutability is not appropriate. The root of all concurrency problems is state management! In particular, access to mutable state in multithreaded environments.

The order of operations in multithreaded applications is unpredictable. In addition to compiler optimizations causing ordering problems, context switching can also cause atomicity problems. If you don’t take the necessary precautions when accessing mutable state, threads can see stale data, lose updates, or suffer from race conditions.

Note that mutable state and access order issues are not specific to the JVM and can affect coroutines on other platforms as well.

An app that uses coroutines is essentially a multithreaded app. Classes that use coroutines and contain mutable state must take precautions to ensure that execution results are as expected, that is, that code executed in coroutines sees the latest version of the data. This way, the different threads do not interfere with each other. Concurrency problems can lead to very minor errors, difficult to debug, and even heisenbugs!

Such problems are not uncommon. For example, maybe a class needs to keep information about a logged in user in memory, or cache some values while the application is running. Concurrency problems can still occur in coroutines if you are not careful! Suspension functions with withContext(defaultDispatcher) cannot always be executed in the same thread!

Suppose we have a class that caches transactions made by users. A concurrency error may occur if the cache is not accessed correctly, as shown in the following example:

class TransactionsRepository(
  private val defaultDispatcher: CoroutineDispatcher = Dispatchers.Default
) {

  private val transactionsCache = mutableMapOf<User, List<Transaction>()

  private suspend fun addTransaction(user: User, transaction: Transaction) =
    / / be careful! The access cache is not protected.
    // Concurrency errors can occur: threads can see stale data, and race conditions can occur
    withContext(defaultDispatcher) {
      if (transactionsCache.contains(user)) {
        val oldList = transactionsCache[user]
        valnewList = oldList!! .toMutableList() newList.add(transaction) transactionsCache.put(user, newList) }else {
        transactionsCache.put(user, listOf(transaction))
      }
    }
}
Copy the code

Even if we’re talking about Kotlin, the Practice of Concurrent Programming in Java by Brian Goetz is an excellent resource for learning more about this and concurrency issues in JVM systems. Or refer to Jetbrains’ documentation on sharing mutable state and concurrency.

Protected variable state

How to protect mutable state or find a good synchronization strategy all depends on the nature of the data and the operations involved. This section is intended to make you aware of the concurrency problems you might encounter, rather than listing all the different methods and apis for securing mutable state. Still, there are some tricks and apis you can take from this to make mutable thread safe.

encapsulation

Mutable state should be encapsulated and owned by a class. This class centralizes access to state and protects read and write operations with a more appropriate synchronization strategy depending on the scenario.

Thread the constraint

One solution is to restrict read/write access to a thread. Access to mutable state can be accomplished in a producer-consumer manner using queues. JetBrains has a good documentation for this.

Don’t duplicate the wheel

In the JVM, you can use thread-safe data structures to protect mutable variables. For example, AtomicInteger can be used for simple counters. To protect the Map of the code above, use ConcurrentHashMap. ConcurrentHashMap is a thread-safe synchronization set that optimizes the read and write throughput of the Map.

Note that thread-safe data structures do not prevent caller-sorting problems, they only ensure that memory access is atomic. They help avoid locks when the logic is not too complex. For example, they cannot be used in the transactionCache example shown above because the order of operations and the logic between them requires threading and access protection.

Similarly, the data in these thread-safe data structures must be immutable or protected to prevent race conditions when modifying objects already stored in them.

Custom solutions

If you have compound operations that need to be synchronized, @volatile variables or thread-safe data structures won’t help! The built-in @synchronized annotations may not be subtle enough to improve efficiency.

In this scenario, you may need to create your own synchronization mechanism using concurrency tools such as Latch, semaphore, or barrier. In other scenarios, you can use locks or mutex to protect multithreaded access to code.

Mutex in Kotlin has suspend functions for lock and unlock to manually protect coroutine code. The mutex. withLock extension function is simple to use:

class TransactionsRepository(
  private val defaultDispatcher: CoroutineDispatcher = Dispatchers.Default
) {

  // Mutex protects cache mutable state
  private val cacheMutex = Mutex()
  private val transactionsCache = mutableMapOf<User, List<Transaction>()

  private suspend fun addTransaction(user: User, transaction: Transaction) =
    withContext(defaultDispatcher) {
      // Mutex makes read-write caches thread-safe
      cacheMutex.withLock {
        if (transactionsCache.contains(user)) {
          val oldList = transactionsCache[user]
          valnewList = oldList!! .toMutableList() newList.add(transaction) transactionsCache.put(user, newList) }else {
          transactionsCache.put(user, listOf(transaction))
        }
      }
    }
}
Copy the code

Because coroutines using Mutex pause execution until they can continue, they are much more efficient than JVM locks that block threads. Be careful when using JVM synchronization classes in coroutines, as this can block the thread in which the coroutine is executed and cause LIVENESS problems.

The block of code passed to the coroutine builder ends up executing on one or more JVM threads. Therefore, coroutines run in the JVM threading model and are subject to all of its constraints. With coroutines, you can still write wrong multithreaded code. So be careful about accessing shared mutable state in your code!

Finish the translation.

The translator to summarize

  • Jvm-based Kotlin coroutines essentially work based on JVM thread pools
  • Coroutines are recommended for asynchronous programming in Android
  • Coroutines also have concurrency issues that developers need to be aware of and address
  • The root of the concurrency problem is state management
  • Protecting mutable state depends on the situation, but there are a few tricks

Recommended reading

  • The source of concurrency problems

  • The thread pool

  • A glance at Kotlin’s coroutines – Can you learn coroutines? Probably because the tutorials you read are all wrong

  • Kotlin coroutine hanging magic? I took the skin off him today

  • Code on school: What is a “non-blocking” hang? Are coroutines really lighter

About me

People love to do things that give them positive feedback. If this article is helpful to you, please click 👍. This is very important to me

I am Flywith24. Only through discussion with others can we know whether our own experience is real or not. By communicating with others on wechat, we can make progress together.

  • The Denver nuggets
  • Small column
  • Github
  • WeChat: Flywith24