aboutsummaryrefslogtreecommitdiff
path: root/src/liblzma/check/crc64_fast.c (unfollow)
AgeCommit message (Collapse)AuthorFilesLines
2022-11-14liblzma: Add fast CRC64 for 32/64-bit x86 using SSSE3 + SSE4.1 + CLMUL.Lasse Collin1-5/+444
It also works on E2K as it supports these intrinsics. On x86-64 runtime detection is used so the code keeps working on older processors too. A CLMUL-only build can be done by using -msse4.1 -mpclmul in CFLAGS and this will reduce the library size since the generic implementation and its 8 KiB lookup table will be omitted. On 32-bit x86 this isn't used by default for now because by default on 32-bit x86 the separate assembly file crc64_x86.S is used. If --disable-assembler is used then this new CLMUL code is used the same way as on 64-bit x86. However, a CLMUL-only build (-msse4.1 -mpclmul) won't omit the 8 KiB lookup table on 32-bit x86 due to a currently-missing check for disabled assembler usage. The configure.ac check should be such that the code won't be built if something in the toolchain doesn't support it but --disable-clmul-crc option can be used to unconditionally disable this feature. CLMUL speeds up decompression of files that have compressed very well (assuming CRC64 is used as a check type). It is know that the CLMUL code is significantly slower than the generic code for tiny inputs (especially 1-8 bytes but up to 16 bytes). If that is a real-world problem then there is already a commented-out variant that uses the generic version for small inputs. Thanks to Ilya Kurdyukov for the original patch which was derived from a white paper from Intel [1] (published in 2009) and public domain code from [2] (released in 2016). [1] https://www.intel.com/content/dam/www/public/us/en/documents/white-papers/fast-crc-computation-generic-polynomials-pclmulqdq-paper.pdf [2] https://github.com/rawrunprotected/crc
2022-10-31liblzma: Silence -Wconversion warning from crc64_fast.c.Lasse Collin1-2/+3
2019-12-31Rename read32ne to aligned_read32ne, and similarly for the others.Lasse Collin1-2/+2
Using the aligned methods requires more care to ensure that the address really is aligned, so it's nicer if the aligned methods are prefixed. The next commit will remove the unaligned_ prefix from the unaligned methods which in liblzma are used in more places than the aligned ones.
2009-11-22Add missing consts to pointer casts.Lasse Collin1-2/+3
2009-10-04Use a tuklib module for integer handling.Lasse Collin1-2/+2
This replaces bswap.h and integer.h. The tuklib module uses <byteswap.h> on GNU, <sys/endian.h> on *BSDs and <sys/byteorder.h> on Solaris, which may contain optimized code like inline assembly.
2009-04-13Put the interesting parts of XZ Utils into the public domain.Lasse Collin1-12/+8
Some minor documentation cleanups were made at the same time.
2009-02-02Modify LZMA_API macro so that it works on Windows withLasse Collin1-1/+1
other compilers than MinGW. This may hurt readability of the API headers slightly, but I don't know any better way to do this.
2008-12-31Remove lzma_init() and other init functions from liblzma API.Lasse Collin1-0/+0
Half of developers were already forgetting to use these functions, which could have caused total breakage in some future liblzma version or even now if --enable-small was used. Now liblzma uses pthread_once() to do the initializations unless it has been built with --disable-threads which make these initializations thread-unsafe. When --enable-small isn't used, liblzma currently gets needlessly linked against libpthread (on systems that have it). While it is stupid for now, liblzma will need threads in future anyway, so this stupidity will be temporary only. When --enable-small is used, different code CRC32 and CRC64 is now used than without --enable-small. This made the resulting binary slightly smaller, but the main reason was to clean it up and to handle the lack of lzma_init_check(). The pkg-config file lzma.pc was renamed to liblzma.pc. I'm not sure if it works correctly and portably for static linking (Libs.private includes -pthread or other operating system specific flags). Hopefully someone complains if it is bad. lzma_rc_prices[] is now included as a precomputed array even with --enable-small. It's just 128 bytes now that it uses uint8_t instead of uint32_t. Smaller array seemed to be at least as fast as the more bloated uint32_t array on x86; hopefully it's not bad on other architectures.
2007-12-09Added missing LZMA_API to the C versions of the CRC functions.Lasse Collin1-1/+1
The x86 assembler versions were already OK.