File compression is an integral part of system administration. Finding the best compression method requires significant determination. Luckily, there are many robust compression tools for Linux that make backing up system data easier. Here, we present some of the best Linux compression tools that can be used to compress files in Linux.
1. LZ4
LZ4 is the compression tool of choice for admins who need lightning-fast compression and decompression speed. It utilizes the LZ4 lossless algorithm, which belongs to the family of LZ77 byte-oriented compression algorithms.
Moreover, LZ4 comes coupled with a high-speed decoder which can process more than 1 Gigabyte/second per CPU core. This makes it ideal for users who are looking to bundle large amounts of data with as little time as possible.
Pros
- Extremely quick compression and decompression algorithm.
- Available in almost every Linux distro today.
Cons
- Prioritizes speed over high compression ratio.
- The default implementation only uses a single CPU core.
2. Zstandard
Zstandard is another fast compression tool for Linux that can be used for personal and enterprise projects. Some of its most compelling features include the adaptive mode, which can control compression ratios based on I/O.
Further, Zstandard has the ability to trade speed for better compression, and the dictionary compression scheme. Lastly, it also has a rich API that is available for all major programming languages.
Pros
- Can adjust the speed-to-compression ratio on the fly.
- Takes little to no system resources during compression.
Cons
- Not readily available in most distributions along with no default support in tar.
- Decompression can take time in high compression ratios.
3. lzop
lzop is a robust compression tool that utilizes the Lempel-Ziv-Oberhumer (LZO) compression algorithm. It provides breakneck compression speed by trading compression ratios. For example, it produces slightly larger files compared to gzip but requires only 10 percent CPU runtime.
Moreover, lzop can deal with system backups in multiple ways, including backup mode, single file mode, archive mode, and pipe mode. Lastly, lzop also aims to be a drop-in replacement for the older gzip algorithm. This means that it is possible to use lzop on scripts that normally use gzip by just creating a Bash alias.
Pros
- Decent compression ratio for good compression speed.
- Easy to retrofit in older archive scripts due to compatibility with gzip.
Cons
- Do not delete the original file after compression.
- Its compression ratio tanks the more it focuses on speed.
4. Gzip
Gzip is certainly one of the most widely used compression tools for Linux admins. It is compatible with every GNU software, making it the perfect compression tool for remote engineers.
Gzip leverages the Lempel-Ziv coding in DEFLATE mode for file compression. It can reduce the size of source code by up to 90 percent. Aside from that, it also preserves a file’s metadata, which includes its modification time and permission bits. Overall, this makes gzip an excellent choice for seasoned Linux users as well as software developers.
Pros
- Available in every Linux distribution as part of GNU coreutils.
- Preserves every relevant metadata of the compressed file.
Cons
- Not as effective when compressing directories of files.
- Mediocre compression ratios and speeds compared to other algorithms.
5. bzip2
bzip2 is a free and open-source compression tool for Linux. It compresses files using the Burrows-Wheeler block-sorting compression algorithm and Huffman coding. It also supports several additional compression methods, such as run-length encoding, delta encoding and sparse bit array.
Bzip2 can also recover data from media drives by reconstructing the missing, damaged data from the available archive. Overall, bzip2 is a suitable compression tool for everyday usage due to its robust compression abilities and fast decompression speed.
Pros
- Has a built-in data recovery function that can reconstruct missing and damaged data on the fly.
- Extremely portable. It can run on both old and new hardware and operating systems.
Cons
- Mediocre compression ratio compared to Zstandard and LZ4.
- It can sometimes cause a file to grow in size instead of reducing it.
6. 7zip
7zip is the port of 7-zip’s command-line utility for Linux. It is a high-performance archiving tool with solid compression ratios. Similar to its Windows counterpart, it has support for many popular formats, including tar, xz, gzip, bzip2, and zip.
7zip also uses the 7z format by default, which provides 30 to 50 percent better compression than standard zip compression. Moreover, you can use this tool for creating self-extracting and dynamically-sized volume archives.
Pros
- Format is easily compatible with other non-Linux operating systems.
- Supports multiple compression formats along with its signature 7z.
Cons
- Does not preserve any file metadata for the compressed archive.
- Not easily obtainable on most Linux distros.
7. pigz
pigz or parallel implementation of gzip is a reliable replacement for the gzip compression tool. It leverages multiple CPU cores to increase the compression speed dramatically. It utilizes the zlib and pthread libraries for implementing the multi-threading compression process. However, pigz can’t decompress archives in parallel. Hence, you will not be able to get similar speeds during compression and decompression.
Pros
- Extremely good compression speeds due to its being able to use multiple cores.
- Cross-compatible with the popular gzip compression binary.
Cons
- Can only compress a single file at a time.
- Slow at decompressing due to a lack of multithreading support on its extraction algorithm.
8. pixz
pixz is a parallel implementation of the XZ compressor with support for data indexing. Instead of producing one big block of compressed data like xz, it creates a set of smaller blocks. This makes randomly accessing the original data straightforward. Moreover, pixz also makes sure that the file permissions are preserved the way they were during compression and decompression.
Pros
- Highly efficient at compressing large blocks of data.
- Utilizes all available CPU cores during file compression.
Cons
- Doesn’t have the ability to pipe data directly to standard output.
- Not fully compatible with its single-core variant: xz.
9. plzip
plzip is a lossless data compressor tool that makes creative use of the multi-threading capabilities supported by modern CPUs. It is built on top of the lzlib library and provides a command-line interface similar to gzip and bzip2.
One key benefit of plzip is its ability to fully leverage multiprocessor machines. Aside from that, the developer of the algorithm designed it to be as data-safe and reliable as possible. This makes it an attractive option for admins who need a high-performance Linux compression tool that supports parallel compression and is suitable for long-term storage.
Pros
- Utilizes all CPU cores during both compression and decompression.
- Built to be safe when handling critical data.
Cons
- It prioritizes speed over compression ratio.
- Requires a minimum file size of 2MB in order to efficiently compress data.
10. XZ Utils
XZ Utils is a suite of compression tools for Linux that can compress and decompress .xz and .lzma files. It primarily uses the LZMA2 algorithm for compression and can perform integrity checks of compressed data at ease. Since this tool is available to popular Linux distributions by default, it can be a viable choice for compression in many situations.
Pros
- Produces files with extremely high compression ratio.
- Available on almost all modern Linux distros.
Cons
- Both its compression and decompression algorithms are CPU-intensive and slow.
- Higher compression ratio settings can be memory-hungry.
11. lrzip
Long Range ZIP (lrzip) is an efficient data compression program for Linux that aims to optimize large binary files. It does this by extending the rzip algorithm to accommodate variable memory sizes as well as making use of multiple compression algorithms.
Aside from that, lrzip is also extremely portable. This means that you can find an lrzip package on almost all modern Linux distros such as Ubuntu, Debian and Arch Linux.
Pros
- Efficient at compressing files with large overall sizes.
- The code is portable enough to be available to most Linux distros.
Cons
- Can only compress one file at a time.
- Using the compression program in a UNIX pipe could result in a less efficient output.
12. Zpaq
Zpaq is a powerful data compression tool that can create incremental backups. Unlike other compression algorithms, Zpaq is a complete software suite that aims to provide a redundant but highly space-efficient archival solution.
Aside from that, Zpaq also has the ability to jump back to an older version of its archive. This is because it keeps track of everything that happens on a Zpaq archive including the exact bytes that changed in-between versions.
Pros
- Can automatically create incremental backups of a directory.
- Excellent compression ratio when dealing with multiple files.
Cons
- Doesn’t preserve the permission bits and modification times of its archives.
- Doesn’t follow symbolic links when archiving data.
A plethora of reliable Linux compression tools make it easy to archive and back up essential data. You can choose from many lossless compressors with high compression ratios such as LZ4, lzop, and bzip2.
On the other hand, tools like Zstandard, plzip and Zpaq allow for more advanced compression workflows. Lastly, compressing archival data is not the end of ensuring your data’s integrity over long periods of time. Learn how you can create a secure cloud backup using rclone.
Image credit: Priscilla Du Preez via Unsplash. All alterations and screenshots by Ramces Red.
Our latest tutorials delivered straight to your inbox
