aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)AuthorFilesLines
2024-04-06CVE-2024-3094: import xz-5.6.0.tar.xzjiatan/v5.6.0/autofooBertrand Jacquin144-129006/+19406
2024-04-06CVE-2024-3094: doxygenBertrand Jacquin74-0/+15370
2024-04-06CVE-2024-3094: po4aBertrand Jacquin45-2/+18755
2024-04-06CVE-2024-3094: autoreconf-2.72 -fiBertrand Jacquin59-0/+192656
2024-04-06CVE-2024-3094: remove .gitignoreBertrand Jacquin4-187/+0
2024-02-24Bump version and soname for 5.6.0.jiatan/v5.6.0Jia Tan4-6/+6
2024-02-24Add NEWS for 5.6.0.Jia Tan1-0/+143
2024-02-22Translations: Remove obsolete and fuzzy matches from some translations.Jia Tan2-2839/+6814
The French and Brazilian Portuguese man page translations have not been updated since the switch from public domain to 0BSD. The old GPLv2 strings have now been removed from these files.
2024-02-21Translations: Patch man pages to avoid fuzzy matches.Jia Tan4-4/+4
This will be fixed in the next round of translations, but this avoids having a fuzzy match or not fixing the English version.
2024-02-21xzmore: Fix typo in xzmore.1.Jia Tan1-1/+1
Thanks to Yuri Chornoivan.
2024-02-24Translations: Update the Vietnamese translation.Jia Tan1-196/+309
2024-02-24Translations: Update the Esperanto translation.Jia Tan1-196/+306
2024-02-23Tests: Add a few test files.Jia Tan6-0/+19
2024-02-23Tests: Add MicroLZMA test.Jia Tan2-1/+551
2024-02-23Build: Define HAVE_MICROLZMA when it is configured.Jia Tan2-2/+11
2024-02-23xz: Fix Capsicum sandbox compile error.Jia Tan1-2/+2
user_abort_pipe[] was still being used instead of the parameters.
2024-02-23Build: Fix ARM64 CRC32 instruction feature test.Jia Tan1-0/+10
Old versions of Clang reported the unsupported function attribute and __crc32d() function as warnings instead of errors, so the feature test passed when it shouldn't have, causing a compile error at build time. -Werror was added to this feature test to fix this. The change is not needed for CMake because check_c_source_compiles() also performs linking and the error is caught then. Thanks to Sebastian Andrzej Siewior for reporting this.
2024-02-22CMake: Add LOCALEDIR to the windres workaround.Lasse Collin1-5/+11
LOCALEDIR may contain spaces like in "C:\Program Files".
2024-02-22xz: Landlock: Fix error message if input file is a directory.Lasse Collin1-1/+14
If xz is given a directory, it should look like this: $ xz /usr/bin xz: /usr/bin: Is a directory, skipping The Landlock rules didn't allow opening directories for reading: $ xz /usr/bin xz: /usr/bin: Permission denied The simplest fix was to allow opening directories for reading. While it's a bit silly to allow it solely for the error message, it shouldn't make the sandbox significantly weaker. The single-file use case (like when called from GNU tar) is still as strict as possible: all Landlock restrictions are enabled before (de)compression starts.
2024-02-22liblzma: Disable branchless C version in range decoder.Lasse Collin1-3/+10
Thanks to Sebastian Andrzej Siewior and Sam James for benchmarking on various systems.
2024-02-21INSTALL: Clarify that --disable-assembler affects only 32-bit x86.Lasse Collin1-9/+9
2024-02-21Windows: build.bash: Include COPYING.0BSD in the package.Lasse Collin1-1/+1
2024-02-21Windows: build.bash: include liblzma-crt-mixing.txt in the package.Lasse Collin1-2/+4
2024-02-21Windows: Major update to Windows build instructions.Lasse Collin8-184/+404
2024-02-21Windows: Update windows/README-Windows.txt.Lasse Collin1-63/+41
It's for binary packages built with windows/build.bash.
2024-02-20Windows: Update windows/build.bash.Lasse Collin1-79/+112
Support for the old MinGW was dropped. Only MinGW-w64 with GCC is supported now. The script now supports also cross-compilation from GNU/Linux (tests are not run). MSYS2 and also the old MSYS 1.0.11 work for building on Windows. The i686 and x86_64 toolchains must be in PATH to build both 32-bit and 64-bit versions. Parallel builds are done if "nproc" from GNU coreutils is available. MinGW-w64 runtime copyright information file was renamed from COPYING-Windows.txt to COPYING.MinGW-w64-runtime.txt which is the filename used by MinGW-w64 itself. Its existence is now mandatory, it's checked at the beginning of the script. The file TODO is no longer copied to the package.
2024-02-20Translations: Update the Romanian man page translations.Jia Tan1-840/+875
2024-02-20Translations: Update the Korean man page translations.Jia Tan1-3/+3
2024-02-20Translations: Update the Spanish translation.Jia Tan1-3/+3
2024-02-20Translations: Update the Romanian translation.Jia Tan1-243/+227
2024-02-20Translations: Update the Croatian translation.Jia Tan1-293/+355
2024-02-20Translations: Update the German man page translations.Jia Tan1-823/+873
2024-02-20Translations: Update the German translation.Jia Tan1-202/+225
2024-02-20Translations: Update the Hungarian translation.Jia Tan1-218/+338
2024-02-19CMake: Fix building of lzmainfo when translations are enabled.Lasse Collin1-0/+2
2024-02-19CMake: Don't assume that -fvisibility=hidden is supported outside Windows.Lasse Collin1-4/+22
The original code was good enough for supporting GNU/Linux and a few others but it wasn't very portable. CMake doesn't support Solaris Studio's -xldscope=hidden. If it ever does, things should still work with this commit as Solaris Studio supports not only its own __global but also the GNU C __attribute__((visibility("default"))). Support for the attribute was added in 2007 to Sun Studio 12 compiler version 5.9.
2024-02-19CMake: Revise the component splitting.Lasse Collin1-26/+31
2024-02-19CMake: Update the main comment and document CMAKE_BUILD_TYPE=Release.Lasse Collin1-16/+63
2024-02-19CMake: Use -O2 instead of -O3 in CMAKE_BUILD_TYPE=Release.Lasse Collin1-0/+19
-O3 doesn't seem useful for speed but it makes the code bigger. CMake makes is difficult for users to simply override the optimization level: CFLAGS / CMAKE_C_FLAGS aren't helpful because they go before CMAKE_C_FLAGS_RELEASE. Of course, users can override CMAKE_C_FLAGS_RELEASE directly but then they have to remember to add also -DNDEBUG to disable assertions. This commit changes -O3 to -O2 in CMAKE_C_FLAGS_RELEASE if and only if CMAKE_C_FLAGS_RELEASE cache variable doesn't already exist. So if a custom value is passed on the command line (or reconfiguring an already-configured build), the cache variable won't be modified.
2024-02-19CMake: Handle symbol versioning on MicroBlaze specially.Lasse Collin1-4/+19
This is to match configure.ac.
2024-02-19CMake: Keep build working even if lib/*.[ch] are removed.Lasse Collin1-1/+6
2024-02-19CMake: Install documentation.Lasse Collin1-0/+32
2024-02-19CMake: Bump maximum policy version to 3.28.Lasse Collin1-1/+1
CMP0154 doesn't affect us since we don't use FILE_SET.
2024-02-19CMake: Build lzmainfo.Lasse Collin1-0/+54
2024-02-19CMake: Build lzmadec.Lasse Collin1-34/+42
2024-02-19CMake: Add test_scripts.sh to the tests.Lasse Collin2-5/+22
In contrast to Automake, skipping of this test when decoders are disabled is handled at CMake side instead of test_scripts.sh because CMake-build doesn't create config.h.
2024-02-19CMake: Install scripts.Lasse Collin1-1/+82
Compared to the Autotools-based build, this has simpler handling for the shell (@POSIX_SHELL@) and extra PATH entry for the scripts (configure has --enable-path-for-scripts=PREFIX). The simpler metho should be enough for non-ancient systems and Solaris.
2024-02-19Scripts: Use @PACKAGE_VERSION@ instead of @VERSION@.Lasse Collin4-4/+4
PACKAGE_VERSION was already used in liblzma.pc.in. This way only one version @foo@ is used.
2024-02-19CMake: Simplify symlink creation and install translated man pages.Lasse Collin1-97/+98
It helps that cmake_install.cmake doesn't parallelize installation so symlinks can be created so that the target is always known to exist (a requirement on Windows in some cases). This bumps the minimum CMake version from 3.13 to 3.14 to use file(CREATE_LINK ...). It could be made to work on 3.13 by calling "cmake -E create_symlink" but it's uglier code and slower in "make install". 3.14 should be a reasonable version to require nowadays, especially since the Autotools build is still the primary build system for most OSes.
2024-02-19CMake: Add support for building and installing xz with translations.Lasse Collin1-2/+66
If gettext tools are available, the .po files listed in po/LINGUAS are converted using msgfmt. This allows building with translations directly from xz.git without Autotools. If gettext tools aren't available, the Autotools-created .gmo files in the "po" directory will be used. This allows CMake-based build to use translations from Autotools-generated tarball. If translation support is found (Intl_FOUND) but both the gettext tools and the pre-generated .gmo files are missing, then "make" will fail.
2024-02-19liblzma: Remove commented-out code.Lasse Collin1-3/+0
2024-02-17xz: Delete old commented-out code.Lasse Collin1-19/+0
2024-02-17xz: Use stricter pledge(2) and Landlock sandbox.Lasse Collin3-13/+69
This makes these sandboxing methods stricter when no files are created or deleted. That is, it's a middle ground between the initial sandbox and the strictest single-file-to-stdout sandbox: this allows opening files for reading but output has to go to stdout.
2024-02-17xz: Support Landlock ABI version 4.Lasse Collin1-5/+20
Linux 6.7 added support for ABI version 4 which restricts TCP connections which xz won't need and thus those can be forbidden now. Since the ABI version is handled at runtime, supporting version 4 won't cause any compatibility issues. Note that new enough kernel headers are required to get version 4 support enabled at build time.
2024-02-17xz: Move sandboxing code to sandbox.c and improve Landlock sandbox.Lasse Collin8-213/+357
Landlock is now always used just like pledge(2) is: first in more permissive mode and later (under certain common conditions) in a strict mode that doesn't allow opening more files. I put pledge(2) first in sandbox.c because it's the simplest API to use and still somewhat fine-grained for basic applications. So it's the simplest thing to understand for anyone reading sandbox.c.
2024-02-17xz: Tweak comments.Lasse Collin1-1/+3
2024-02-17xz: Fix message_init() description.Lasse Collin3-3/+7
Also explicitly initialize progress_automatic to make it clear that it can be read before message_init() sets it. Static variable was initialized to false by default already so this is only for clarity.
2024-02-17Build: Makefile.am: Sort EXTRA_DIST.Lasse Collin1-7/+7
Dirs first, then files in case-sensitive ASCII order.
2024-02-17Build: Don't install TODO.Lasse Collin1-1/+1
2024-02-18Translations: Update the Korean man page translations.Jia Tan1-836/+871
2024-02-18Translations: Update the Korean translation.Jia Tan1-200/+223
2024-02-17Build: Install translated lzmainfo man pages.Lasse Collin1-0/+26
All other translated man pages were being installed but lzmainfo had been forgotten.
2024-02-17liblzma: Avoid implementation-defined behavior in the RISC-V filter.Lasse Collin1-8/+22
GCC docs promise that it works and a few other compilers do too. Clang/LLVM is documented source code only but unsurprisingly it behaves the same as others on x86-64 at least. But the certainly-portable way is good enough here so use that.
2024-02-17liblzma: Wrap a line exceeding 80 chars.Lasse Collin1-1/+2
2024-02-17liblzma/rangecoder: Exclude x32 from the x86-64 optimisation.Sebastian Andrzej Siewior1-1/+1
The x32 port has a x86-64 ABI in term of all registers but uses only 32bit pointer like x86-32. The assembly optimisation fails to compile on x32. Given the state of x32 I suggest to exclude it from the optimisation rather than trying to fix it. Signed-off-by: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
2024-02-17Translations: Update the Spanish translation.Jia Tan1-201/+226
2024-02-17Translations: Update the Swedish translation.Jia Tan1-204/+230
2024-02-17Translations: Update the Polish translation.Jia Tan1-200/+224
2024-02-17Translations: Update the Ukrainian translation.Jia Tan1-1/+1
2024-02-16Translations: Use the same sentence in xz.pot-header that the TP uses.Lasse Collin1-1/+1
2024-02-16Fix typos discovered by codespell.Jia Tan3-4/+4
2024-02-16Translations: Update the Ukrainian man page translations.Jia Tan1-837/+873
2024-02-16Translations: Update the Ukrainian translation.Jia Tan1-241/+225
2024-02-15Translations: Omit the generic copyright line from man page headers.Lasse Collin1-0/+1
2024-02-15Update m4/.gitignore.Jia Tan1-0/+1
2024-02-14Tests: tuktest.h: Treat Clang separately from GCC.Lasse Collin1-3/+3
Don't assume that Clang defines __GNUC__ as the extensions are available in clang-cl as well (and possibly in some other Clang variants?).
2024-02-14Tests: tuktest.h: Add a missing word to a comment.Lasse Collin1-2/+2
2024-02-14Tests: tuktest.h: Fix the comment about STest.Lasse Collin1-1/+2
2024-02-15Bump version for 5.5.2beta.larhzu/v5.5.2betaJia Tan3-4/+4
2024-02-14liblzma: Fix validate_map.sh.Lasse Collin1-1/+1
Adding the SPDX license identifier changed the line numbers.
2024-02-14Build: Start the generated ChangeLog from around 5.4.0 instead of 5.2.0.Lasse Collin1-1/+1
2024-02-14Fixed NEWS for 5.5.2beta.Lasse Collin1-2/+6
2024-02-14liblzma: Silence warnings in --enable-small build.Lasse Collin2-0/+3
2024-02-14Build: Install COPYING.0BSD as part of docs.Lasse Collin1-0/+1
2024-02-14Docs: List COPYING.0BSD in README.Lasse Collin1-0/+1
2024-02-14Docs: Include doc/examples/11_file_info.c in tarballs.Lasse Collin1-0/+1
It was added in 2017 in c2e29f06a7d1e3ba242ac2fafc69f5d6e92f62cd but it never got into any release tarballs because it was forgotten to be added to Makefile.am.
2024-02-14liblzma: Silence a warning.Lasse Collin1-1/+1
2024-02-14Add NEWS for 5.5.2beta.Lasse Collin1-0/+60
2024-02-14xz: Mention lzmainfo if trying to use 'lzma --list'.Lasse Collin1-2/+14
This kind of fixes the problem reported here: https://bugs.launchpad.net/ubuntu/+source/xz-utils/+bug/1291020
2024-02-14liblzma: Add comments.Lasse Collin2-2/+18
2024-02-14Scripts: Add lz4 support to xzgrep and xzdiff.Lasse Collin4-10/+19
2024-02-14liblzma: Choose the range decoder variants using a bitmask macro.Lasse Collin1-11/+53
2024-02-14xz: Fix outdated threading related info on the man page.Lasse Collin1-8/+14
2024-02-14liblzma: Range decoder: Add x86-64 inline assembly.Lasse Collin1-0/+491
It's compatible with GCC and Clang.
2024-02-14liblzma: Range decoder: Add branchless C code.Lasse Collin1-0/+76
It's used only for basic bittrees and fixed-size reverse bittree because those showed a clear benefit on x86-64 with GCC and Clang. The other methods were more mixed and thus are commented out but they should be tested on other archs.
2024-02-14liblzma: Clarify a comment.Lasse Collin1-3/+6
2024-02-14liblzma: LZMA decoder: Optimize loop comparison.Lasse Collin2-4/+11
But now it needs one more local variable.
2024-02-14liblzma: Optimize literal_subcoder() macro slightly.Lasse Collin5-22/+24
2024-02-14liblzma: LZ decoder: Add unlikely().Lasse Collin1-1/+1
2024-02-14liblzma: LZ decoder: Remove a useless unlikely().Lasse Collin1-1/+1
2024-02-14liblzma: Optimize LZ decoder slightly.Lasse Collin3-60/+88
Now extra buffer space is reserved so that repeating bytes for any single match will never need to copy from two places (both the beginning and the end of the buffer). This simplifies dict_repeat() and helps a little with speed. This seems to reduce .lzma decompression time about 2 %, so with .xz and CRC it could be slightly less. The small things add up still.
2024-02-14liblzma: LZMA decoder: Get rid of next_state[].Lasse Collin3-24/+24
It's not completely obvious if this is better in the decoder. It should be good if compiler can avoid creating a branch (like using CMOV on x86). This also makes lzma_encoder.c use the new macros.
2024-02-14liblzma: LZMA decoder improvements.Lasse Collin3-200/+210
This adds macros for bittree decoding which prepares the code for alternative C versions and inline assembly.
2024-02-14liblzma: Creates Non-resumable and Resumable modes for lzma_decoder.Jia Tan2-213/+521
The new decoder resumes the first decoder loop in the Resumable mode. Then, the code executes in Non-resumable mode until it detects that it cannot guarantee to have enough input/output to decode another symbol. The Resumable mode is how the decoder has always worked. Before decoding every input bit, it checks if there is enough space and will save its location to be resumed later. When the decoder has more input/output, it jumps back to the correct sequence in the Resumable mode code. When the input/output buffers are large, the Resumable mode is much slower than the Non-resumable because it has more branches and is harder for the compiler to optimize since it is in a large switch block. Early benchmarking shows significant time improvement (8-10% on gcc and clang x86) by using the Non-resumable code as much as possible.
2024-02-14liblzma: Creates separate "safe" range decoder mode.Jia Tan2-103/+82
The new "safe" range decoder mode is the same as old range decoder, but now the default behavior of the range decoder will not check if there is enough input or output to complete the operation. When the buffers are close to fully consumed, the "safe" operations must be used instead. This will improve speed because it will reduce the number of branches needed for most of the range decoder operations.
2024-02-14doxygen/footer.html: Add missing closing tags and don't open a new tab.Lasse Collin1-2/+4
The footer template from Doxygen has the closing </body> </html> as Doxygen doesn't add them otherwise. target="_blank" was omitted as it's not useful here but it can be slightly annoying as one cannot just go back in the browser history. Since the footer links to the license file in the same directory and not to CC website, the rel attributes can be omitted.
2024-02-14Tweak the expressions in AUTHORS.Lasse Collin1-8/+23
2024-02-14Translations: Add the man page translators into man page header comment.Lasse Collin3-7/+26
It looked odd to only have the original English authors listed in the header comments of the translated files.
2024-02-14Translations: Translate also messages of lzmainfo.Lasse Collin1-0/+2
lzmainfo has had translation support since 2009 at least but it was never added to po/POTFILES.in so the messages weren't translated. It's a very rarely needed tool so it's not too bad. This also adds src/xz/mytime.c to po/POTFILES.in although there are no translatable strings. It's simpler this way so that it won't be forgotten if strings were ever added to that file.
2024-02-14Translations: Add custom .pot header with SPDX license identifier.Lasse Collin3-0/+16
The same is used for both po/xz.pot and po4a/xz-man.pot.
2024-02-14Translations: po4a/update-po: Add copyright notice to xz-man.pot.Lasse Collin1-1/+1
All man pages are under 0BSD now so this is simple now.
2024-02-14Update COPYING about the man pages of the scripts.Lasse Collin1-3/+3
2024-02-14xzdiff, xzgrep, and xzmore: Rewrite the man pages.Lasse Collin3-116/+173
The main reason is a kind of silly one: xz-man.pot contains strings from all man pages in XZ Utils. The man pages of xzdiff, xzgrep, and xzmore were under GPLv2 and the rest under 0BSD. Thus xz-man.pot contained strings under two licences. po4a creates the translated man pages from the combined 0BSD+GPLv2 xz-man.pot. I haven't liked this mixing in xz-man.pot but the Translation Project requires that all man pages must be in the same .pot file. So a separate xz-man-gpl.pot wasn't an option. Since these man pages are short, rewriting them was quick enough. Now xz-man.pot is entirely under 0BSD and marking the per-file licenses is simpler. As a bonus, some wording hopefully is now slightly better although it's perhaps a matter of taste. NOTE: In xzgrep.1, the EXIT STATUS section was written by me in the commit d796b6d7fdb8b7238b277056cf9146cce25db604 so that's why that section could be taken as is from the old xzgrep.1.
2024-02-14xzless: Update man page slightly.Lasse Collin1-4/+4
The xz tool can decompress three file formats and xzless has always supported uncompressed files too.
2024-02-14Translations: Change po/Makevars to add a copyright notice to po/xz.pot.Lasse Collin1-2/+2
2024-02-14Translations: Update po/Makevars to use the template from gettext 0.22.4.Lasse Collin1-5/+46
Also add SPDX license identifier now that there is a known license.
2024-02-14liblzma: Include the SPDX license identifier 0BSD to generated files.Lasse Collin11-26/+50
Perhaps the generated files aren't even copyrightable but using the same license for them as for the rest of the liblzma keeps things more consistent for tools that look for license info.
2024-02-14liblzma: Fix compilation of price_tablegen.c.Lasse Collin2-1/+9
It is built and run only manually so this didn't matter unless one wanted to regenerate the price_table.c.
2024-02-14Add SPDX license identifiers to GPL, LGPL, and FSFULLR files.Lasse Collin22-6/+37
2024-02-14Add SPDX license identifier into 0BSD source code files.Lasse Collin290-58/+588
2024-02-14liblzma: Sync the AUTHORS fix about SHA-256 to lzma.h.Lasse Collin1-6/+4
2024-02-14Change most public domain parts to 0BSD.Lasse Collin288-911/+100
Translations and doc/xz-file-format.txt and doc/lzma-file-format.txt were not touched. COPYING.0BSD was added.
2024-02-14Fix SHA-256 authors.Lasse Collin2-14/+6
The initial commit 5d018dc03549c1ee4958364712fb0c94e1bf2741 in 2007 had a comment in sha256.c that the code is based on Crypto++ Library 5.5.1. In 2009 the Authors list in sha256.c and the AUTHORS file was updated with information that the code had come from Crypto++ but via 7-Zip. I know I had viewed 7-Zip's SHA-256 code but back then the C code has been identical enough with Crypto++, so I don't why I thought the author info would need that extra step via 7-Zip for this single file. Another error is that I had mixed sha.* and shacal2.* files when checking for author info in Crypto++. The shacal2.* files aren't related to liblzma's sha256.c and thus Kevin Springle's code in Crypto++ isn't either.
2024-02-14Remove macosx/build.sh.Lasse Collin2-114/+0
It was last updated in 2013.
2024-02-14Doc: Remove doc/examples_old.Lasse Collin3-255/+0
It was good to keep these around in parallel with the newer examples but I think it's OK to remove the old ones at this point.
2024-02-13Tests: Add RISC-V filter support in a few places.Jia Tan2-0/+12
2024-02-13liblzma: Fix build error if only RISC-V BCJ filter is enabled.Jia Tan1-1/+3
If any other BCJ filter was enabled for encoding or decoding, then this was not a problem.
2024-02-13Translations: Update the Korean translation.Jia Tan1-242/+284
2024-02-13Translations: Update the Korean man page translations.Jia Tan1-605/+770
2024-02-13Translations: Update the Chinese (simplified) translation.Jia Tan1-156/+268
2024-02-09xzless: Use ||- in LESSOPEN with with "less" 451 and newer.Lasse Collin1-1/+8
2024-02-09xzless: Use --show-preproc-errors with "less" 632 and newer.Lasse Collin1-2/+9
This makes "less" show a warning if a decompression error occurred.
2024-02-09liblzma: Fix typo discovered by codespell.Jia Tan1-1/+1
2024-02-09Translations: Update the Swedish translation.Jia Tan1-166/+254
2024-02-08Translations: Update the Spanish translation.Jia Tan1-11/+11
2024-02-07Translations: Update the Spanish translation.Jia Tan1-166/+253
2024-02-07Translations: Update the Polish translation.Jia Tan1-162/+249
2024-02-07Translations: Update the German translation.Jia Tan1-154/+242
2024-02-07Translations: Update the German man page translations.Jia Tan1-601/+752
2024-02-06Translations: Update the Romanian translation.Jia Tan1-164/+252
2024-02-06Translations: Update the Romanian man page translations.Jia Tan1-793/+966
2024-02-07Translations: Update the Ukrainian translation.Jia Tan1-155/+242
2024-02-06Translations: Update the Ukrainian man page translations.Jia Tan1-599/+764
2024-02-02Update AUTHORS.Jia Tan1-1/+2
2024-02-02liblzma: Update Authors list in crc32_arm64.h.Jia Tan1-0/+1
2024-02-01liblzma: Check HAVE_USABLE_CLMUL before omitting CRC32 table.Jia Tan1-2/+2
This was split from the prior commit so it could be easily applied to the 5.4 branch. Closes: https://github.com/tukaani-project/xz/pull/77
2024-02-01liblzma: Check HAVE_USABLE_CLMUL before omitting CRC64 table.Jia Tan1-2/+2
If liblzma is configured with --disable-clmul-crc CFLAGS="-msse4.1 -mpclmul", then it will fail to compile because the generic version must be used but the CRC tables were not included.
2024-02-01liblzma: Only use ifunc in crcXX_fast.c if its needed.Jia Tan2-6/+6
The code was using HAVE_FUNC_ATTRIBUTE_IFUNC instead of CRC_USE_IFUNC. With ARM64, ifunc is incompatible because it requires non-inline function calls for runtime detection.
2024-02-01Docs: Add --disable-arm64-crc32 description to INSTALL.Jia Tan1-1/+11
2024-02-01liblzma: Omit CRC tables when not needed with ARM64 optimizations.Jia Tan3-5/+25
This is similar to the existing x86-64 CLMUL conditions to omit the tables. They were slightly refactored to improve readability.
2024-02-01liblzma: Rename crc32_aarch64.h to crc32_arm64.h.Jia Tan6-116/+122
Even though the proper name for the architecture is aarch64, this project uses ARM64 throughout. So the rename is for consistency. Additionally, crc32_arm64.h was slightly refactored for the following changes: * Added MSVC, FreeBSD, and macOS support in is_arch_extension_supported(). * crc32_arch_optimized() now checks the size when aligning the buffer. * crc32_arch_optimized() loop conditions were slightly modified to avoid both decrementing the size and incrementing the buffer pointer. * Use the intrinsic wrappers defined in <arm_acle.h> because GCC and Clang name them differently. * Minor spacing and comment changes.
2024-02-01liblzma: Refactor crc_common.h.Jia Tan3-42/+82
The CRC_GENERIC is now split into CRC32_GENERIC and CRC64_GENERIC, since the ARM64 optimizations will be different between CRC32 and CRC64. For the same reason, CRC_ARCH_OPTIMIZED is split into CRC32_ARCH_OPTIMIZED and CRC64_ARCH_OPTIMIZED. ifunc will only be used with x86-64 CLMUL because the runtime detection methods needed with ARM64 are not compatible with ifunc.
2024-02-01CMake: Add support for ARM64 CRC32 instruction detection.Jia Tan1-0/+50
2024-02-01Build: Add support for ARM64 CRC32 instruction detection.Jia Tan1-0/+52
This adds --enable-arm64-crc32/--disable-arm64-crc32 (enabled by default) for using the ARM64 CRC32 instruction. This can be disabled if one knows the binary will never need to run on an ARM64 machine with this instruction extension.
2024-01-27Speed up CRC32 calculation on ARM64Chenxi Mao6-9/+130
The CRC32 instructions in ARM64 can calculate the CRC32 result for 8 bytes in a single operation, making the use of ARM64 instructions much faster compared to the general CRC32 algorithm. Optimized CRC32 will be enabled if ARM64 has CRC extension running on Linux. Signed-off-by: Chenxi Mao <chenxi.mao2013@gmail.com>
2024-01-26Bump version number for 5.5.1alpha.larhzu/v5.5.1alphaJia Tan3-3/+3
2024-01-26Add NEWS for 5.5.1alphaJia Tan1-0/+80
2024-01-26Add NEWS for 5.4.6.Jia Tan1-0/+22
2024-01-24Move doc/logo/xz-logo.png to "doc" and Doxygen footer to "doxygen".Lasse Collin4-3/+3
The footer isn't a complete HTML file so having it in the doxygen directory is a tiny bit clearer.
2024-01-25README: Add COPYING.CC-BY-SA-4.0 entry to section 1.1.Jia Tan1-18/+20
The Overall documentation section (1.1) table spacing had to be adjusted since the filename was very long.
2024-01-25Build: Add the logo and license to the release.Jia Tan1-0/+2
2024-01-25COPYING: Add the license for the XZ logo.Jia Tan2-0/+432
2024-01-25Doxygen: Added the XZ logo and copyright information.Jia Tan3-3/+14
The PROJECT_LOGO field is now used to include the XZ logo. The footer of each page now lists the copyright information instead of the default footer. The license is also copied to statisfy the copyright and so the link in the documentation can be local.
2024-01-23xz: Use threaded mode by defaut (as if --threads=0 was used).Lasse Collin3-3/+16
This hopefully does more good than bad: + It's faster by default. + Only the threaded compressor creates files that can be decompressed in threaded mode. - Compression ratio is worse, usually not too much though. When it matters, -T1 must be used. - Memory usage increases. - Scripts that assume single-threaded mode but don't use -T1 will possibly use too much resources, for example, if they run multiple xz processes in parallel to compress multiple files. - Output from single-threaded and multi-threaded compressors differ but such changes could happen for other reasons too (they just haven't happened since 5.0.0).
2024-01-23CI: Use RISC-V filter when building with BCJ support.Jia Tan1-2/+2
2024-01-23Tests: Use smaller dictionary size in RISC-V test files.Jia Tan2-0/+0
2024-01-23Tests: Skip RISC-V test files if decoder was not built.Jia Tan1-0/+5
2024-01-23xz: Man page: Add more examples of LZMA2 options with BCJ filters.Lasse Collin1-7/+31
2024-01-23liblzma: RISC-V filter: Use byte-by-byte access.Lasse Collin1-30/+84
Not all RISC-V processors support fast unaligned access so it's better to read only one byte in the main loop. This can be faster even on x86-64 when compared to reading 32 bits at a time as half the time the address is only 16-bit aligned. The downside is larger code size on archs that do support fast unaligned access.
2024-01-23xz: Update xz -lvv for RISC-V filter.Jia Tan1-0/+10
Version 5.6.0 will be shown, even though upcoming alphas and betas will be able to support this filter. 5.6.0 looks nicer in the output and people shouldn't be encouraged to use an unstable version in production in any way.
2024-01-23Tests: Add two RISC-V Filter test files.Jia Tan3-0/+8
These test files achieve 100% code coverage in src/liblzma/simple/riscv.c. They contain all of the instructions that should be filtered and a few cases that should not.
2024-01-23xz: Update message in --long-help for RISC-V Filter.Jia Tan1-0/+1
2024-01-23xz: Update the man page for the RISC-V Filter.Jia Tan1-1/+2
A special note was added to suggest using four-byte alignment when the compressed instruction extension is not present in a RISC-V binary.
2024-01-23Tests: Add RISC-V Filter test in test_compress.sh.Jia Tan1-0/+1
2024-01-23liblzma: Update string_conversion.c to support RISC-V Filter.Jia Tan1-0/+5
2024-01-23CMake: Support RISC-V BCJ Filter for encoding and decoding.Jia Tan1-0/+1
2024-01-23liblzma: Add RISC-V BCJ filter.Jia Tan9-2/+742
The new Filter ID is 0x0B. Thanks to Chien Wong <m@xv97.com> for the initial version of the Filter, the xz CLI updates, and the Autotools build system modifications. Thanks to Igor Pavlov for his many contributions to the design of the filter.
2024-01-19Docs: Update .xz file format specification to 1.2.0.Jia Tan1-12/+17
The new RISC-V filter was added to the specification, in addition to updating the specification URL.
2024-01-19xz: Update website URLs in the man pages.Jia Tan2-5/+5
2024-01-19liblzma: Update website URL.Jia Tan2-4/+4
2024-01-19Docs: Update website URLs.Jia Tan6-15/+17
2024-01-19Build: Update website URL.Jia Tan2-2/+2
2024-01-11liblzma: CRC: Add a comment to crc_x86_clmul.h about BUILDING_ macros.Lasse Collin1-0/+6
2024-01-11liblzma: CRC: Remove crc_always_inline, use lzma_always_inline instead.Lasse Collin2-21/+1
Now crc_simd_body() in crc_x86_clmul.h is only called once in a translation unit, we no longer need to be so cautious about ensuring the always-inline behavior.
2024-01-11liblzma: CRC: Update CLMUL comments to more generic wording.Lasse Collin2-13/+13
2024-01-11liblzma: Rename arch-specific CRC functions and macros.Lasse Collin4-25/+31
CRC_CLMUL was split to CRC_ARCH_OPTIMIZED and CRC_X86_CLMUL. CRC_ARCH_OPTIMIZED is defined when an arch-optimized version is used. Currently the x86 CLMUL implementations are the only arch-optimized versions, and these also use the CRC_x86_CLMUL macro to tell when crc_x86_clmul.h needs to be included. is_clmul_supported() was renamed to is_arch_extension_supported(). crc32_clmul() and crc64_clmul() were renamed to crc32_arch_optimized() and crc64_arch_optimized(). This way the names make sense with arch-specific non-CLMUL implementations as well.
2024-01-11liblzma: Fix a comment in crc_common.h.Lasse Collin1-1/+2
2024-01-11liblzma: Avoid extern lzma_crc32_clmul() and lzma_crc64_clmul().Lasse Collin7-91/+91
A CLMUL-only build will have the crcxx_clmul() inlined into lzma_crcxx(). Previously a jump to the extern lzma_crcxx_clmul() was needed. Notes about shared liblzma on ELF platforms: - On platforms that support ifunc and -fvisibility=hidden, this was silly because CLMUL-only build would have that single extra jump instruction of extra overhead. - On platforms that support neither -fvisibility=hidden nor linker version script (liblzma*.map), jumping to lzma_crcxx_clmul() would go via PLT so a few more instructions of overhead (still not a big issue but silly nevertheless). There was a downside with static liblzma too: if an application only needs lzma_crc64(), static linking would make the linker include the CLMUL code for both CRC32 and CRC64 from crc_x86_clmul.o even though the CRC32 code wouldn't be needed, thus increasing code size of the executable (assuming that -ffunction-sections isn't used). Also, now compilers are likely to inline crc_simd_body() even if they don't support the always_inline attribute (or MSVC's __forceinline). Quite possibly all compilers that build the code do support such an attribute. But now it likely isn't a problem even if the attribute wasn't supported. Now all x86-specific stuff is in crc_x86_clmul.h. If other archs The other archs can then have their own headers with their own is_clmul_supported() and crcxx_clmul(). Another bonus is that the build system doesn't need to care if crc_clmul.c is needed. is_clmul_supported() stays as inline function as it's not needed when doing a CLMUL-only build (avoids a warning about unused function).
2024-01-11liblzma: crc_clmul.c: Add crc_attr_target macro.Lasse Collin1-14/+16
This reduces the number of the complex #if directives.
2024-01-11liblzma: Simplify existing cases with lzma_attr_no_sanitize_address.Lasse Collin1-9/+3
2024-01-11liblzma: #define crc_attr_no_sanitize_address in crc_common.h.Lasse Collin1-0/+10
2024-01-10liblzma: CRC: Add empty lines.Lasse Collin3-1/+5
And remove one too.
2024-01-10liblzma: crc_clmul.c: Tidy up the location of MSVC pragma.Lasse Collin1-2/+2
It makes no difference in practice.
2023-12-28Update THANKS.Lasse Collin1-0/+1
2023-12-28liblzma: Use 8-byte method in memcmplen.h on ARM64.Lasse Collin1-8/+10
It requires fast unaligned access to 64-bit integers and a fast instruction to count leading zeros in a 64-bit integer (__builtin_ctzll()). This perhaps should be enabled on some other archs too. Thanks to Chenxi Mao for the original patch: https://github.com/tukaani-project/xz/pull/75 (the first commit) According to the numbers there, this may improve encoding speed by about 3-5 %. This enables the 8-byte method on MSVC ARM64 too which should work but wasn't tested.
2023-12-28liblzma: Check also for __clang__ in memcmplen.h.Lasse Collin1-1/+2
This change hopefully makes no practical difference as Clang likely was detected via __GNUC__ or _MSC_VER already.
2023-12-21Translations: Update the French translation.Jia Tan1-262/+370
2023-12-21xz: Add a comment to Capsicum sandbox setup.Jia Tan1-0/+1
This comment is repeated in xzdec.c to help remind us why all the capabilities are removed from stdin in certain situations.
2023-12-21Docs: Update --enable-sandbox option in INSTALL.Jia Tan1-7/+10
xzdec now also uses the sandbox when its configured.
2023-12-21CMake: Move sandbox detection outside of xz section.Jia Tan1-80/+98
The sandbox is now enabled for xzdec as well, so it no longer belongs in just the xz section. xz and xzdec are always built, except for older MSVC versions, so there isn't a need to conditionally show the sandbox configuration. CMake will do a little unecessary work on older MSVC versions that can't build xz or xzdec, but this is a very small downside.