[The Art of Scalability] Book Review

✍️ Book review

The book “Technology that Supports Large-Scale Services” was published in 2011, so it is an old book, so there were some parts that did not fit the current technology, and there was also knowledge that had changed in an unusual way. However, it was good to see how Hatena's large-scale service was managed.

There was a lot of content about load management and scalability. It was an article written to make the content easier to understand and remember with examples and repetitive stories about Hatena. In addition, I thought it would be a good idea to review and read other recent books on operating systems, servers, and infrastructure.

? Book contents summary

I couldn't organize all the contents of the book, but I mainly organized the parts that I wanted to remember and take with me.

Difficulties in processing large data scales

Large-scale data such as more than gigabytes takes a lot of time to process.
Large-scale data is difficult to calculate in memory, so the disk is searched for specific data.

Memory and Disk

Disk search speed is slower than memory because it involves physical movement. This is because the physical structure of the disk involves movement of the head and rotation of the disk.
If the data in memory is an algorithm or data structure that is easy to upload to the CPU cache, the search speed will be faster.
OS solves the structural problem of disk seek speed by stacking consecutive data in the same location.
As you can see in the photo below, you can see that the search speed using memory and cache is faster than the disk. However, because the price is also high, it is necessary to consider scalability depending on the service or company circumstances.

Check load on Linux system

Because the response method is different depending on whether the load is from the AP server or the DB server, it is necessary to understand where the load occurs.
Load average.

Load average indicates the level of waiting for processing on average over 1 minute, 5 minutes, and 15 minutes. It's a shame.

Differences in ideal values occur based on the core.

Generally, when it is 1.00 in a single core, it can be considered to be used at 100%. It is best to consider scalability based on 0.70. A value exceeding 1.00 can be seen as a process that is waiting due to load.

Based on multi-core, 2.00 is used at 100% for dual core. This is the state.

The load average is calculated for each Timer Interrupt that the hardware sends to the CPU at a certain period.
I checked the load average of the server currently hosting.

Currently, there is a slight load based on 1 minute, and even based on 15 minutes, it was found that it is time to consider scalability. Additionally, to determine the cause of bottlenecks among CPU and I/O, use the sar command to check the specific resource location. I need to find out more about other parts, but since it is off topic, I will end this post.

AP server load

AP server load means CPU load, and after expanding the server, it is resolved with a load balancer.
Increase servers with the same structure and distribute requests evenly with a load balancer. (scale-out method)
Program generating load = Bound program

DB server load

Unlike the load on the AP server, it is not a problem that can be solved by creating multiple DB servers.

Assuming that there are multiple servers, if the ratio of memory that cannot be cached remains the same, the load issue may remain.
1) Process in memory, minimize disk seek, and distribute using locality.

2) Use appropriate algorithm.

3) Data compression and Use information retrieval technology (search engine specialized for purpose).
OS Cache Disk caching speeds up the search speed, and old caches are destroyed according to LRU.

If the physical memory is large, all caching must be considered because memory is expensive. It is necessary.
Or, partitioning is required to properly divide the data. However, partitioning is the last factor to be considered. Because partitioning increases the number of units, the failure rate also increases. This is because recovery time increases and additional economic costs are incurred due to difficulty in operation.

1) Partitioning considering table size or access

2) It is necessary to separate and manage the DB responsible for the bot and the basic DB depending on the purpose. There is.
Necessity of index and improvement of number of searches

MySQL's B+ tree table structure provides optimized data It is a structure.

Compared to a bipartite tree, it has only pointer values, and this is because it is a structure that has actual values only in the last leaf node.
Replication

Slave - In the master structure, the content entered as an update query in the master is polled and updated by the slave. In other words, the same DB information is shared. Search queries use slaves.

There is also a way to enable mutual replication using the multi-master function. DB server is converted to Active or Standby state with VRRP (Virtual Router Redundancy Protocol).

Many updates In this case, since master expansion is difficult, expansion can be done using table partitioning, disk partitioning, or key-value store. For reference, 90% of web applications mostly use reference queries.

System instability factors

Increased load due to added functions
Memory leak
Code that causes infinite loops or abnormal memory consumption
User's access pattern
Increase in data volume
Impact of external connection API
Decreased processing power due to hardware instability

대규모 서비스를 지탱하는 기술 - 교보문고

데이터 구조 메모리 OS DB 서버 인프라 | 베스트셀러 『(24시간 365일)서버/인프라를 지탱하는 기술』의 핵심 저자들이 또 한 권의 역작을 펴냈다. 서버 1대부터 시작하여 1,000대의 호스트를 운영하