Java multithreading, concurrency, and performance optimization

This is a page that summarizes the lectures I took with support from Udemy while participating in the 9th class.

1. Overview

Multithreading is required for responsiveness and performance. In a multi-threaded environment, responses can be processed simultaneously, so you can process other tasks by moving the mouse through the UI even while the video is in progress. It appears that tasks are being executed simultaneously, but in fact, this type of parallelism is possible because the tasks are divided into threads for each task. With multithreading you don't need as many cores. Because it processes more services than using a single thread, you can reduce the number of machines and reduce hardware costs. The operating system is loaded into memory from the hard disk and the operating system helps interact between hardware CPUs. As each application runs from the disk, the operating system brings the program into memory and creates a program instance. Each process is independent and internally consists of PID, Code, Heap Data, Files, and main thread. At this time, if it has threads other than the main thread, you can tell that it is a multi-threaded process. A thread has a stack, local variables, and an instruction pointer. It serves to point to the next instruction address.

• Additional investigation into the structure of the process
• When a function is executed, the address where the function will be returned is stored on the stack and local variables are sequentially stacked on the stack. After the function is executed, the stack data disappears one by one and the return address is also supported and moved to the return location.
• stack frame
• Process status and hierarchy
• Manage the process status by recording it on the PCB. Numerous processes are managed hierarchically, and each operating system has five process states. Creation, preparation, execution, waiting (blocked status), termination status
• The parent process creates its own copy child process with the fork system call, inheriting resources
• The child process overwrites the exec memory space with a new program, the code/data area is changed to the contents of the program to be executed, and the remaining areas are initialized.

Computer structure operating system to study alone I think it would be a good idea to listen to all of Hanbit’s media lectures..!

Context switch

Handling many threads at the same time is not easy. When switching to another thread, you must save all existing data and restore the other thread's resources to the cpu.

Thread Scheduling

Since starvation can occur, several issues must be taken into consideration when the operating system distributes threads equally to the CPU. The operating system divides time into appropriately sized chunks called epochs. Then, the time slices of the threads are assigned to epochs by type. How time is allocated to threads depends on how the operating system sets priorities. Prioritizing interactive threads, the threads themselves are much easier to create and destroy, and parallel processing using multiple threads within a single process can be performed faster and more efficiently. For security and safety reasons, it is recommended that it runs in one process.

2. Threading Basics - Thread Creation

Thread features and debugging

You can create a thread using the Runnable interface. In addition to using threads here, the mere existence of these threads continues to consume resources, so you need to think about how to properly manage threads and organize the application.

public static void main(String[] args) throws InterruptedException {
Thread thread = new Thread(new Runnable(){
@Override
public void run() {
Thread.currentThread().getName();
}
});
thread.setName("New Worker Thread");
thread.setPriority(Thread.MAX_PRIOPITY);
Thread.currentThread().getName();
thread.start();
Thread.currentThread().getName();
Thread.sleep(1000);
}

public static void main(String[] args) throws InterruptedException {
Thread thread = new Thread(new Runnable(){
@Override
public void run() {
throw new RuntimeExcpetion("Intentional Exception");
}
});
thread.setName("Misbehaving thread");
thread.setUncaughtExceptionHandler(new Thread.UncaughtExcpetionHandler() {
@Override
public void uncaughtException(Thread t, Throuable e) {
t.getName();
e.getMessage();
}
})
}

Thread inheritance

public static void main(String[] args) throws InterruptedException {
Thread thread = new NewThread();
thread.start();
}
private static class NewThread extends Thread {
@Override
public void run() {
Thread.currentThread().getName();
}
}

Quiz

Multiple threads within a process share the following:

• hip
• code
• open files in process
• metadata of the process

Thread thread = new Thread(new Runnable() {
@Override
public void run() {
System.out.println("Executing from a new thread");
}
});

This code creates a thread and, when started via a call to thread.start(), will execute the code inside the run() method of the new thread.

Coding Exercise 1: Creating a Thread - MultiExecutor

import java.util.*;
public class MultiExecutor {
public MultiExecutor(List<Runnable> tasks) {
for(Runnable thread:tasks){
thread.run();
}
}
public void executeAll() {
Thread thread1 = new Thread("thread1");
Thread thread2 = new Thread("thread2");
List<Runnable> list = new ArrayList();
list.add(thread1);
list.add(thread2);
new MultiExecutor(list);
}
}

Coding Exercise 2: Creating Threads - MultiExecutor Solution

import java.util.ArrayList;
import java.util.List;
public class MultiExecutor {
private final List<Runnable> tasks;
/*
* @param tasks to executed concurrently
*/
public MultiExecutor(List<Runnable> tasks) {
this.tasks = tasks;
}
/**
* Executes all the tasks concurrently
*/
public void executeAll() {
List<Thread> threads = new ArrayList<>(tasks.size());
for (Runnable task : tasks) {
Thread thread = new Thread(task);
threads.add(thread);
}
for(Thread thread : threads) {
thread.start();
}
}
}

• Compared to the code I wrote, I felt that the solution was better in many ways. First, the constructor of the class was called from a method within the class.

new MultiExecutor(list);

It seemed inappropriate to express it in such a way.

• Like the answer in the solution, I think it would be better to have a separate local variable in the private final List<Runnable> tasks; class and create a method using that variable.

3. Thread adjustment

It refers to processing using Thread.interrupt(); to control the thread. However, this may not be enough to stop the thread unless there is a way to adjust it. You can check whether each thread is interrupted with Thread.currentThread().isInterrupted(). But is there any other way other than individual checking? Daemon Thread is a thread that can run processing even when the application terminates (termination of the main thread). You can use Daemon Thread in the same way as thread.setDaemon(true);. Alternatively, you can consider thread parallelism using Thread.join() so that the main thread can safely terminate after all threads are executed.

// 스레드가 interrupt된 경우 InterruptedException 사례
@Override
public void run() {
try {
Thread.sleep(500000);
} catch (InterruptedException e) {
System.out.println("Existing blocking thread");
}
}

// 스레드의 interrupt 여부를 checking 방식
for (BigInteger i = BigInteger.ZERO; i.compareTo(power) != 0; i = i.add(BigInteger.ONE)) {
if (Thread.currentThread().isInterrupted()) {
System.out.println("Prematurely interrupted computation");
return BigInteger.ZERO;
}
result = result.multiply(base);
}

// Daemon Thread의 적용
thread.setDaemon(true);

4. Performance optimization

The performance mentioned here can be divided into Latency delay time and Throughput throughput.

Delay time

If the CPU is arranged to perform only one task per core and the number of threads is added (i.e., the number of threads increases), this can generally lead to context switching and poor cache performance. Therefore, we can say that the optimal state is when all threads are runnable. The lecture included information on image processing to reduce latency using threads as an example, and although it was not complicated, it was interesting because you could actually see the image change. First, to extract the RGB color, it is processed in the following way. Depending on the FF position of each bit, the RGB color, which is one of the pixel color encoding groups, can be obtained. For reference, when expressed as ARGB, it is a version of the RGB method, where A stands for alpha, that is, alpha (transparency). All bits in the result of the bitmask must be shifted to the right using the >> operator.

public static int getRed(int rgb) {
return (rgb & 0x00FF0000) >> 16;
}
public static int getGreen(int rgb) {
return (rgb & 0x0000FF00) >> 8;
}
public static int getBlue(int rgb) {
return rgb & 0x000000FF;
}

In the lecture, we handled the image processing task of converting all white flowers into pink flowers. As shown in the following code, we divided the image height by the number of threads so that each thread could process it.

• There is no benefit to increasing the number of threads more than the number of virtual cores.
• Parallel execution of algorithms is expensive.

public static void recolorImage(BufferedImage originalImage, BufferedImage resultImage, int leftCorner, int topCorner, int width, int height) {
for(int x = leftCorner ; x < leftCorner + width && x < originalImage.getWidth() ; x++) {
for(int y = topCorner ; y < topCorner + height && y < originalImage.getHeight() ; y++) {
recolorPixel(originalImage, resultImage, x , y);
}
}
}

Throughput

Throughput is the workload divided by threads (T/N). Here, simply dividing and merging tasks incurs costs, so processing without creating a separate thread above may actually provide better performance. However, as the number of threads increases, the throughput increases, so you can see that performance is better when the number of threads exceeds a certain number.

5. Sharing data between threads

Stack and Heap

The stack is a space in memory where methods are executed in a last-in-first-out manner. The stack contains local primitive types and local references. On the other hand, the heap is a space managed by the JVM that contains Objects, Class Members, and Static Valuables.

Atomic Operations

An atomic task literally means a case where the task itself is executed at once, like a single atom. items++; cannot be considered an atomic operation because it retrieves the current value from memory, then adds 1 to that value, and assigns the new Value to items, and so on.

6. Concurrency problems and solutions

Synchronization with critical section

A critical section is an area that can be processed as a single atomic operation. In order for this area to be processed as a single atomic operation, Java gives the method synchronized processing. Most atomic operations occur when handling operations such as get, sest, and references. However, since most methods are non-atomic operations, you need to worry about synchronization. Atomicity can also be given to variables through volatile processing, but since variables such as long and double are not guaranteed in 64-bit Java, atomicity is lost in the process of being divided into 32 bits and processed.

private static class InventoryCounter {
private int items = 0;
Object lock = new Object();
public void increment() {
synchronized (this.lock) {
items++;
}
}
// ...
}

Race conditions and data races, locking techniques and deadlocks

This is literally a situation caused by race conditions and synchronization issues, and occurs because non-atomic operations are performed on shared resources. This problem can be solved by putting the critical section, which is the point of the race condition, in a synchronization block. In this competitive state, deadlock occurs when mutual exclusion, hold and wait, non-preemption, and circular waiting problems all exist. In order to avoid circular waiting, a solution is proposed to maintain the access order of all methods so that they can be executed in order. Data competition refers to problems occurring due to out-of-order execution, where data is reflected in the database rather than sequential calculations. In the code below, both x++; and y++; in increment() are not atomic operations, so if you check checkForDataRace() after processing, you can see that a data race has occurred.

public class Main {
public static void main(String[] args) {
SharedClass sharedClass = new SharedClass();
Thread thread1 = new Thread(() -> {
for (int i = 0; i < Integer.MAX_VALUE; i++) {
sharedClass.increment();
}
});
Thread thread2 = new Thread(() -> {
for (int i = 0; i < Integer.MAX_VALUE; i++) {
sharedClass.checkForDataRace();
}
});
thread1.start();
thread2.start();
}
public static class SharedClass {
private int x = 0;
private int y = 0;
public void increment() {
x++;
y++;
}
public void checkForDataRace() {
if (y > x) {
System.out.println("y > x - Data Race is detected");
}
}
}
}

7. Advanced locking

ReentrantLock is a concept provided by Java and is a method of controlling the fairness of locks. If you mainly read and write only a little, it is more effective to use ReentrantReadWriteLock to allow simultaneous reading of shared resources.

private ReentrantReadWriteLock reentrantReadWriteLock = new ReentrantReadWriteLock();
private Lock readLock = reentrantReadWriteLock.readLock();
private Lock writeLock = reentrantReadWriteLock.writeLock();
public int getNumberOfItemsInPriceRange(int lowerBound, int upperBound) {
readLock.lock();
try {
Integer fromKey = priceToCountMap.ceilingKey(lowerBound);
Integer toKey = priceToCountMap.floorKey(upperBound);
if (fromKey == null || toKey == null) {
return 0;
}
NavigableMap<Integer, Integer> rangeOfPrices = priceToCountMap.subMap(fromKey, true, toKey, true);
int sum = 0;
for (int numberOfItemsForPrice : rangeOfPrices.values()) {
sum += numberOfItemsForPrice;
}
return sum;
} finally {
readLock.unlock();
}
}