Using cgroups to manage Linux disk io method

We all know that cgroups in Linux system can be used to manage cpu resources. Today, Xiaobian wants to introduce you how to use cgroups to manage process disk io, if you are interested in it. Let's take a look.

This feature is achieved by blkio subsystem.

There are a lot of things in the blkio subsystem. However, most of them are read-only status reports. The only parameters that can be written are the following:

The code is as follows:

blkio.throttle.read_bps_device

blkio. Throttle.read_iops_device

blkio.throttle.write_bps_device

blkio.throttle.write_iops_device

blkio.weight

blkio.weight_device

These Both are used to control the process of the disk io. Obviously divided into two categories, which with "throttle", as the name suggests is a throttle, limiting the flow to a certain value. And "weight" is the weight assigned to io.

“throttle” The four parameters look at the name to know what to do. Take blkio.throttle.read_bps_device to limit the number of bytes that can be read per second. Run io out first

The code is as follows:

dd if=/dev/sda of=/dev/null &

[1] 2750

Use iotop to see the current io

code as follows:

TID PRIO USER DISK READ DISK WRITE SWAPIN IO" COMMAND

2750 be/4 root 66.76 M/s 0.00 B/s 0.00 % 68.53 % dd if=/dev/sda of=/dev/null

. . .

Then modify the resource limit and add the process to the control group

The code is as follows:

echo ‘8:0 1048576’ ”/sys/fs/cgroup/blkio/Foo/blkio.throttle.read_bps_device

echo 2750 》/sys/fs/cgroup/blkio/foo/tasks

Here 8:0 is the main device number and sub-device of the corresponding block device number. It can be viewed by the ls -l device file name. For example, the

code is as follows:

# ls -l /dev/sda

brw-rw----. 1 root disk 8, 0 Oct 24 11:27 /dev/sda

Here, 8, 0 is the corresponding device number. Therefore, cgroups can impose different restrictions on different devices. Then take a look at the effect

The code is as follows:

TID PRIO USER DISK READ DISK WRITE SWAPIN IO" COMMAND

2750 be/4 root 989.17 K/s 0.00 B/s 0.00 % 96.22 % dd if=/dev/sda of=/dev/null

. . .

It can be seen that the reading of the process per second drops to about 1MB. To remove the restriction, write to “8:0 0” to the file

However, it should be noted that this method is useless for a large amount of io generated in the sampling interval. For example, even if you generate a peak of 100M per second in 1s, it will not be limited.

Look at blkio.weight again. Blkio's throttle and weight methods are a bit like the quota and shares of the cpu subsystem. Both are absolute limits, the other is relative, and resources can be fully utilized when not busy. Weight values ​​range from 10 &ndash ; 1000 between.

Testing weights is a bit more troublesome. Because it is not an absolute limit, it is affected by the file system cache. If you are testing in a virtual machine, you should turn off the caching of the virtual machine such as the VirtualBox I use on the host. To test the effect of reading io, Mr. is a large file of two G /tmp/file_1, /tmp/file_2, which can be done with dd. Then set the two weights

The code is as follows:

# echo 500 》/sys/fs/cgroup/blkio/foo/blkio.weight

# echo 100 》/sys /fs/cgroup/blkio/bar/blkio.weight

Empty the file system cache before testing to avoid interfering with test results

The code is as follows:

sync

echo 3 》/proc/sys/vm/drop_caches

Use dd to generate io test results in these two control groups.

The code is as follows:

# cgexec -g “blkio:foo” dd if=/tmp/file_1 of=/dev/null &

[1] 1838

#cgexec -g “blkio:bar” dd if=/tmp/file_2 of=/dev/null &

[2] 1839

Still using iotop Look at the effect

The code is as follows:

TID PRIO USER DISK READ DISK WRITE SWAPIN IO" COMMAND

1839 be/4 root 48.14 M/s 0.00 B/s 0.00 % 99.21 % dd if=/tmp/file_2 of=/dev/null

1838 be/4 root 223.59 M/s 0.00 B/s 0.00 % 16.44 % dd if=/tmp/file_1 of=/dev/Null

Although the number of bytes read per second by the two processes is constantly changing, the general trend remains at around 1:5, which is consistent with the set weight ratio. Blkio.weight_device is a sub-device. When writing, add the device number in front.

There are many statistics in the blkio subsystem

blkio.time

io for each device Access time, in milliseconds

blkio.sectors

Incoming or Out Number of sectors in the device

blkio.io_serviced

The various types of io operations performed, sub-read, write, sync, async, and total

Blkio.io_service_bytes

Each type of io changer or out of each device Number of bytes

blkio.io_service_time

Execution in each device Each type of io time, in microseconds

Blkio.io_wait_time

Waiting time for each type of io in the queue in the device

blkio.io_merged

Number of times each type of io request merges in each device

blkio.io_queued

The number of io requests currently in the queue for each type _

Better statistics and monitoring of the io status of the process by these statistics.

Use the

code as follows :

echo 1 "blkio.reset_stats

All statistics can be cleared.

The above is the use of cgroups management process disk io in Linux introduced, the blkio subsystem will be used in the management process disk io, you can limit and monitor the process disk io through the above method, you learn Yet?