Which is better for Linux compression tools gzip and pigz?

GZIP is mainly used for file compression, and pigz is its parallel implementation version. Its compression speed is faster than gzip, and it makes full use of multi-processing and multi-core systems. The following small series will introduce you to pigz.

pig is Gesha stuff?

Introduction to the sentence:

The code is as follows:

A parallel implementation of gzip for modern multi-processor, multi-core machines.

Simply put, it supports parallel gzip. Not much nonsense, start testing.

pig defaults to the current logical cpu number for concurrent compression. If the number cannot be detected, then 8 threads are concurrently executed.

First, test the compressed package file 1

The code is as follows:

Before compression

-rw-r--r-- 1 root root 2474772480 Dec 3 19:54 dbs.tar "/p" "p" 1, gzip test

compression time, CPU highest: 101.6%

real 2m18.042s

user 2m10.204s

sys 0m6.876s "/p" "p" decompression time-consuming, CPU highest: 101.6%

real 0m26.756s

user 0m20.175s

sys 0m6.580s "/p" "p" compressed

-rw-r--r-- 1 root root 2349563863 Dec 3 19:54 dbs.tar.gz "/p "p"2, pigz test

concurrent 8 threads (default):

compression time, CPU highest: 832%

real 0m21.710s

user 2m31.229s

sys 0m9.101s "/p" "p" decompression time, CPU highest:199%

real 0m13.239s

User 0m11.791s

sys 0m12.426s "/p" "p" concurrent 4 threads:

Compression time, CPU highest: 427.6%

real 0m38.299s

User 2m30.701s

sys 0m8.369s "/p" "p" decompression time-consuming, CPU highest: 202%

real 0m13.294s

user 0m12. 154s

sys 0m11.862s "/p" "p" compressed

-rw-r--r-- 1 root root 2350161547 Dec 3 19:54 dbs.tar.gz /p" "p" Second, test the compressed package file 2

before compression

-rw-r--r-- 1 root root 5248890880 Dec 3 20:26 mysql-5.5.28 -linux2.6-x86_64.tar "/p" "p" 1, gzip test

compression time, CPU highest: 101%

real 6m20.524s

User 6m2.466s

sys 0m7.410s "/p" "p" decompression time-consuming, CPU highest: 101%

real 0m58.893s

user 0m46. 258s

sys 0m11.944s "/p" "p" compressed

-rw-r--r-- 1 root root 1410045513 Dec 3 20:26 mysql-5.5.28- Linux2.6-x86_64.tar.gz "/p" "p" 2, pigz test

concurrent 8 threads (default):

compression time, CPU highest: 799%

real 1m11.908s

u Ser 6m36.717s

sys 0m9.860s "/p" "p" decompression time-consuming, CPU highest: 179%

real 0m33.188s

user 0m38. 069s

sys 0m16.728s "/p" "p" concurrent 4 threads:

Compression time, CPU highest: 418%

real 1m41.547s

user 6m36.782s

sys 0m9.806s "/p" "p" decompression time-consuming, CPU highest: 179%

real 0m33.120s

User 0m37.500s

sys 0m16.956s "/p" "p" compressed

-rw-r--r-- 1 root root 1406061442 Dec 3 20:26 mysql-5.5 .28-linux2.6-x86_64.tar.gz

Conclusion:

1, the default usage of pigz (the default concurrent thread is the number of logical cpu) can be 5.3 times faster than gzip, CPU consumption It is 8 times of gzip, the compression ratio is quite;

2, concurrent 8 thread comparison 4 thread promotion: 41.2%, 16 thread comparison 8 thread promotion: 27.9%, 32 thread comparison 16 thread promotion: 3%;

3, in the scenario where the compression efficiency is high, but the CPU consumption is not affected in a short time, It is very suitable to use pigz.

The above is the introduction of pigz under Linux. It is easy to see from this article that the compression efficiency of pigz is higher, but the loss of CPU is also higher. In the case that CPU does not affect, it is recommended to use pigz compressed file. .