aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)AuthorFilesLines
2024-04-09Update THANKS.larhzu/v5.6Lasse Collin1-0/+1
2024-04-09Remove the backdoor found in 5.6.0 and 5.6.1 (CVE-2024-3094).Lasse Collin12-66/+8
While the backdoor was inactive (and thus harmless) without inserting a small trigger code into the build system when the source package was created, it's good to remove this anyway: - The executable payloads were embedded as binary blobs in the test files. This was a blatant violation of the Debian Free Software Guidelines. - On machines that see lots bots poking at the SSH port, the backdoor noticeably increased CPU load, resulting in degraded user experience and thus overwhelmingly negative user feedback. - The maintainer who added the backdoor has disappeared. - Backdoors are bad for security. This reverts the following without making any other changes: 6e636819 Tests: Update two test files. a3a29bbd Tests: Test --single-stream can decompress bad-3-corrupt_lzma2.xz. 0b4ccc91 Tests: Update RISC-V test files. 8c9b8b20 liblzma: Fix typos in crc32_fast.c and crc64_fast.c. 82ecc538 liblzma: Fix false Valgrind error report with GCC. cf44e4b7 Tests: Add a few test files. 3060e107 Tests: Use smaller dictionary size in RISC-V test files. e2870db5 Tests: Add two RISC-V Filter test files. The RISC-V test files also have real content that tests the filter but the real content would fit into much smaller files. A generator program would need to be available as well. Thanks to Andres Freund for finding and reporting it and making it public quickly so others could act without a delay. See: https://www.openwall.com/lists/oss-security/2024/03/29/4
2024-03-09Bump version and soname for 5.6.1.jiatan/v5.6.1Jia Tan2-2/+2
2024-03-09Add NEWS for 5.6.1Jia Tan1-0/+26
2024-03-09Translations: Add missing --riscv option to man page translations.Jia Tan6-1974/+2024
2024-03-09Tests: Update two test files.Jia Tan2-0/+0
The original files were generated with random local to my machine. To better reproduce these files in the future, a constant seed was used to recreate these files.
2024-03-09Tests: Test --single-stream can decompress bad-3-corrupt_lzma2.xz.Jia Tan1-0/+11
The first stream in this file is valid, so this tests that xz properly stops after decompressing it.
2024-03-09Tests: Update RISC-V test files.Jia Tan2-0/+0
This increases code coverage and tests for possible shifting bugs.
2024-03-09liblzma: Fix typos in crc32_fast.c and crc64_fast.c.Jia Tan2-4/+3
2024-03-09Tests: Replace HAVE_MICROLZMA usage in CMake and Autotools builds.Jia Tan4-23/+22
This reverts commit adaacafde6661496ca2814b1e94a3ba5186428cb.
2024-03-09liblzma: Fix false Valgrind error report with GCC.Jia Tan3-10/+31
With GCC and a certain combination of flags, Valgrind will falsely trigger an invalid write. This appears to be due to the omission of instructions to properly save, set up, and restore the frame pointer. The IFUNC resolver is a leaf function since it only calls a function that is inlined. So sometimes GCC omits the frame pointer instructions in the resolver unless this optimization is explictly disabled. This fixes https://bugzilla.redhat.com/show_bug.cgi?id=2267598.
2024-03-05liblzma: Fix a typo in a comment in the RISC-V filter.Lasse Collin1-1/+1
2024-03-05liblzma: Use attribute no_profile_instrument_function with ifunc.Jia Tan2-0/+8
Thanks to Sam James for determining this was the attribute needed to workaround the GCC bug and for his version of the patch in Gentoo.
2024-03-05Build: Require attribute no_profile_instrument_function for ifunc usage.Jia Tan2-0/+14
Using __attribute__((__no_profile_instrument_function__)) on the ifunc resolver works around a bug in GCC -fprofile-generate: it adds profiling code even to ifunc resolvers which can make the ifunc resolver crash at program startup. This attribute was not introduced until GCC 7 and Clang 13, so ifunc won't be used with prior versions of these compilers. This bug was brought to our attention by: https://bugs.gentoo.org/925415 And was reported to upstream GCC by: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=11411
2024-03-04liblzma: Fix a comment in the RISC-V filter.Lasse Collin1-2/+2
2024-02-29CMake: Warn if translated man pages are missing.Lasse Collin1-0/+9
2024-02-29CMake: Warn if gettext tools and pre-created .gmo files are missing.Lasse Collin1-0/+25
It's only done with CMake >= 3.20 and if library support for translation was already found. Sort of fixes: https://github.com/tukaani-project/xz/issues/82
2024-02-28xz: Add comments.Lasse Collin1-0/+10
2024-02-28xz: Change logging level for thread reduction to highest verbosity only.Jia Tan1-2/+2
Now that multi threaded encoding is the default, users do not need to see a warning message everytime the number of threads is reduced. On some machines, this could happen very often. It is not unreasonable for users to need to set double verbose mode to see this kind of information. To see these warning messages -vv or --verbose --verbose must be passed to set xz into the highest possible verbosity mode. These warnings had caused automated testing frameworks to fail when they expected no output to stderr. Thanks to Sebastian Andrzej Siewior for reporting this and for the initial version of the patch.
2024-02-28Fix sorting in THANKS.Lasse Collin1-2/+2
2024-02-28Update THANKS.Jia Tan1-0/+1
2024-02-28xz: Add missing RISC-V on the filter list in the man pageChien Wong1-1/+3
Signed-off-by: Chien Wong <m@xv97.com>
2024-02-28Build: Fix Linux Landlock feature test in Autotools and CMake builds.Jia Tan5-10/+54
The previous Linux Landlock feature test assumed that having the linux/landlock.h header file was enough. The new feature tests also requires that prctl() and the required Landlock system calls are supported.
2024-02-28Tests: Add test_microlzma to .gitignore and CMakeLists.txt.Jia Tan2-0/+2
2024-02-28Tests: Correct license header in test_microlzma.c.Jia Tan1-3/+2
2024-02-28Fix typos in NEWS and CMakeLists.Jia Tan2-2/+2
2024-02-24Bump version and soname for 5.6.0.jiatan/v5.6.0Jia Tan4-6/+6
2024-02-24Add NEWS for 5.6.0.Jia Tan1-0/+143
2024-02-22Translations: Remove obsolete and fuzzy matches from some translations.Jia Tan2-2839/+6814
The French and Brazilian Portuguese man page translations have not been updated since the switch from public domain to 0BSD. The old GPLv2 strings have now been removed from these files.
2024-02-21Translations: Patch man pages to avoid fuzzy matches.Jia Tan4-4/+4
This will be fixed in the next round of translations, but this avoids having a fuzzy match or not fixing the English version.
2024-02-21xzmore: Fix typo in xzmore.1.Jia Tan1-1/+1
Thanks to Yuri Chornoivan.
2024-02-24Translations: Update the Vietnamese translation.Jia Tan1-196/+309
2024-02-24Translations: Update the Esperanto translation.Jia Tan1-196/+306
2024-02-23Tests: Add a few test files.Jia Tan6-0/+19
2024-02-23Tests: Add MicroLZMA test.Jia Tan2-1/+551
2024-02-23Build: Define HAVE_MICROLZMA when it is configured.Jia Tan2-2/+11
2024-02-23xz: Fix Capsicum sandbox compile error.Jia Tan1-2/+2
user_abort_pipe[] was still being used instead of the parameters.
2024-02-23Build: Fix ARM64 CRC32 instruction feature test.Jia Tan1-0/+10
Old versions of Clang reported the unsupported function attribute and __crc32d() function as warnings instead of errors, so the feature test passed when it shouldn't have, causing a compile error at build time. -Werror was added to this feature test to fix this. The change is not needed for CMake because check_c_source_compiles() also performs linking and the error is caught then. Thanks to Sebastian Andrzej Siewior for reporting this.
2024-02-22CMake: Add LOCALEDIR to the windres workaround.Lasse Collin1-5/+11
LOCALEDIR may contain spaces like in "C:\Program Files".
2024-02-22xz: Landlock: Fix error message if input file is a directory.Lasse Collin1-1/+14
If xz is given a directory, it should look like this: $ xz /usr/bin xz: /usr/bin: Is a directory, skipping The Landlock rules didn't allow opening directories for reading: $ xz /usr/bin xz: /usr/bin: Permission denied The simplest fix was to allow opening directories for reading. While it's a bit silly to allow it solely for the error message, it shouldn't make the sandbox significantly weaker. The single-file use case (like when called from GNU tar) is still as strict as possible: all Landlock restrictions are enabled before (de)compression starts.
2024-02-22liblzma: Disable branchless C version in range decoder.Lasse Collin1-3/+10
Thanks to Sebastian Andrzej Siewior and Sam James for benchmarking on various systems.
2024-02-21INSTALL: Clarify that --disable-assembler affects only 32-bit x86.Lasse Collin1-9/+9
2024-02-21Windows: build.bash: Include COPYING.0BSD in the package.Lasse Collin1-1/+1
2024-02-21Windows: build.bash: include liblzma-crt-mixing.txt in the package.Lasse Collin1-2/+4
2024-02-21Windows: Major update to Windows build instructions.Lasse Collin8-184/+404
2024-02-21Windows: Update windows/README-Windows.txt.Lasse Collin1-63/+41
It's for binary packages built with windows/build.bash.
2024-02-20Windows: Update windows/build.bash.Lasse Collin1-79/+112
Support for the old MinGW was dropped. Only MinGW-w64 with GCC is supported now. The script now supports also cross-compilation from GNU/Linux (tests are not run). MSYS2 and also the old MSYS 1.0.11 work for building on Windows. The i686 and x86_64 toolchains must be in PATH to build both 32-bit and 64-bit versions. Parallel builds are done if "nproc" from GNU coreutils is available. MinGW-w64 runtime copyright information file was renamed from COPYING-Windows.txt to COPYING.MinGW-w64-runtime.txt which is the filename used by MinGW-w64 itself. Its existence is now mandatory, it's checked at the beginning of the script. The file TODO is no longer copied to the package.
2024-02-20Translations: Update the Romanian man page translations.Jia Tan1-840/+875
2024-02-20Translations: Update the Korean man page translations.Jia Tan1-3/+3
2024-02-20Translations: Update the Spanish translation.Jia Tan1-3/+3
2024-02-20Translations: Update the Romanian translation.Jia Tan1-243/+227
2024-02-20Translations: Update the Croatian translation.Jia Tan1-293/+355
2024-02-20Translations: Update the German man page translations.Jia Tan1-823/+873
2024-02-20Translations: Update the German translation.Jia Tan1-202/+225
2024-02-20Translations: Update the Hungarian translation.Jia Tan1-218/+338
2024-02-19CMake: Fix building of lzmainfo when translations are enabled.Lasse Collin1-0/+2
2024-02-19CMake: Don't assume that -fvisibility=hidden is supported outside Windows.Lasse Collin1-4/+22
The original code was good enough for supporting GNU/Linux and a few others but it wasn't very portable. CMake doesn't support Solaris Studio's -xldscope=hidden. If it ever does, things should still work with this commit as Solaris Studio supports not only its own __global but also the GNU C __attribute__((visibility("default"))). Support for the attribute was added in 2007 to Sun Studio 12 compiler version 5.9.
2024-02-19CMake: Revise the component splitting.Lasse Collin1-26/+31
2024-02-19CMake: Update the main comment and document CMAKE_BUILD_TYPE=Release.Lasse Collin1-16/+63
2024-02-19CMake: Use -O2 instead of -O3 in CMAKE_BUILD_TYPE=Release.Lasse Collin1-0/+19
-O3 doesn't seem useful for speed but it makes the code bigger. CMake makes is difficult for users to simply override the optimization level: CFLAGS / CMAKE_C_FLAGS aren't helpful because they go before CMAKE_C_FLAGS_RELEASE. Of course, users can override CMAKE_C_FLAGS_RELEASE directly but then they have to remember to add also -DNDEBUG to disable assertions. This commit changes -O3 to -O2 in CMAKE_C_FLAGS_RELEASE if and only if CMAKE_C_FLAGS_RELEASE cache variable doesn't already exist. So if a custom value is passed on the command line (or reconfiguring an already-configured build), the cache variable won't be modified.
2024-02-19CMake: Handle symbol versioning on MicroBlaze specially.Lasse Collin1-4/+19
This is to match configure.ac.
2024-02-19CMake: Keep build working even if lib/*.[ch] are removed.Lasse Collin1-1/+6
2024-02-19CMake: Install documentation.Lasse Collin1-0/+32
2024-02-19CMake: Bump maximum policy version to 3.28.Lasse Collin1-1/+1
CMP0154 doesn't affect us since we don't use FILE_SET.
2024-02-19CMake: Build lzmainfo.Lasse Collin1-0/+54
2024-02-19CMake: Build lzmadec.Lasse Collin1-34/+42
2024-02-19CMake: Add test_scripts.sh to the tests.Lasse Collin2-5/+22
In contrast to Automake, skipping of this test when decoders are disabled is handled at CMake side instead of test_scripts.sh because CMake-build doesn't create config.h.
2024-02-19CMake: Install scripts.Lasse Collin1-1/+82
Compared to the Autotools-based build, this has simpler handling for the shell (@POSIX_SHELL@) and extra PATH entry for the scripts (configure has --enable-path-for-scripts=PREFIX). The simpler metho should be enough for non-ancient systems and Solaris.
2024-02-19Scripts: Use @PACKAGE_VERSION@ instead of @VERSION@.Lasse Collin4-4/+4
PACKAGE_VERSION was already used in liblzma.pc.in. This way only one version @foo@ is used.
2024-02-19CMake: Simplify symlink creation and install translated man pages.Lasse Collin1-97/+98
It helps that cmake_install.cmake doesn't parallelize installation so symlinks can be created so that the target is always known to exist (a requirement on Windows in some cases). This bumps the minimum CMake version from 3.13 to 3.14 to use file(CREATE_LINK ...). It could be made to work on 3.13 by calling "cmake -E create_symlink" but it's uglier code and slower in "make install". 3.14 should be a reasonable version to require nowadays, especially since the Autotools build is still the primary build system for most OSes.
2024-02-19CMake: Add support for building and installing xz with translations.Lasse Collin1-2/+66
If gettext tools are available, the .po files listed in po/LINGUAS are converted using msgfmt. This allows building with translations directly from xz.git without Autotools. If gettext tools aren't available, the Autotools-created .gmo files in the "po" directory will be used. This allows CMake-based build to use translations from Autotools-generated tarball. If translation support is found (Intl_FOUND) but both the gettext tools and the pre-generated .gmo files are missing, then "make" will fail.
2024-02-19liblzma: Remove commented-out code.Lasse Collin1-3/+0
2024-02-17xz: Delete old commented-out code.Lasse Collin1-19/+0
2024-02-17xz: Use stricter pledge(2) and Landlock sandbox.Lasse Collin3-13/+69
This makes these sandboxing methods stricter when no files are created or deleted. That is, it's a middle ground between the initial sandbox and the strictest single-file-to-stdout sandbox: this allows opening files for reading but output has to go to stdout.
2024-02-17xz: Support Landlock ABI version 4.Lasse Collin1-5/+20
Linux 6.7 added support for ABI version 4 which restricts TCP connections which xz won't need and thus those can be forbidden now. Since the ABI version is handled at runtime, supporting version 4 won't cause any compatibility issues. Note that new enough kernel headers are required to get version 4 support enabled at build time.
2024-02-17xz: Move sandboxing code to sandbox.c and improve Landlock sandbox.Lasse Collin8-213/+357
Landlock is now always used just like pledge(2) is: first in more permissive mode and later (under certain common conditions) in a strict mode that doesn't allow opening more files. I put pledge(2) first in sandbox.c because it's the simplest API to use and still somewhat fine-grained for basic applications. So it's the simplest thing to understand for anyone reading sandbox.c.
2024-02-17xz: Tweak comments.Lasse Collin1-1/+3
2024-02-17xz: Fix message_init() description.Lasse Collin3-3/+7
Also explicitly initialize progress_automatic to make it clear that it can be read before message_init() sets it. Static variable was initialized to false by default already so this is only for clarity.
2024-02-17Build: Makefile.am: Sort EXTRA_DIST.Lasse Collin1-7/+7
Dirs first, then files in case-sensitive ASCII order.
2024-02-17Build: Don't install TODO.Lasse Collin1-1/+1
2024-02-18Translations: Update the Korean man page translations.Jia Tan1-836/+871
2024-02-18Translations: Update the Korean translation.Jia Tan1-200/+223
2024-02-17Build: Install translated lzmainfo man pages.Lasse Collin1-0/+26
All other translated man pages were being installed but lzmainfo had been forgotten.
2024-02-17liblzma: Avoid implementation-defined behavior in the RISC-V filter.Lasse Collin1-8/+22
GCC docs promise that it works and a few other compilers do too. Clang/LLVM is documented source code only but unsurprisingly it behaves the same as others on x86-64 at least. But the certainly-portable way is good enough here so use that.
2024-02-17liblzma: Wrap a line exceeding 80 chars.Lasse Collin1-1/+2
2024-02-17liblzma/rangecoder: Exclude x32 from the x86-64 optimisation.Sebastian Andrzej Siewior1-1/+1
The x32 port has a x86-64 ABI in term of all registers but uses only 32bit pointer like x86-32. The assembly optimisation fails to compile on x32. Given the state of x32 I suggest to exclude it from the optimisation rather than trying to fix it. Signed-off-by: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
2024-02-17Translations: Update the Spanish translation.Jia Tan1-201/+226
2024-02-17Translations: Update the Swedish translation.Jia Tan1-204/+230
2024-02-17Translations: Update the Polish translation.Jia Tan1-200/+224
2024-02-17Translations: Update the Ukrainian translation.Jia Tan1-1/+1
2024-02-16Translations: Use the same sentence in xz.pot-header that the TP uses.Lasse Collin1-1/+1
2024-02-16Fix typos discovered by codespell.Jia Tan3-4/+4
2024-02-16Translations: Update the Ukrainian man page translations.Jia Tan1-837/+873
2024-02-16Translations: Update the Ukrainian translation.Jia Tan1-241/+225
2024-02-15Translations: Omit the generic copyright line from man page headers.Lasse Collin1-0/+1
2024-02-15Update m4/.gitignore.Jia Tan1-0/+1
2024-02-14Tests: tuktest.h: Treat Clang separately from GCC.Lasse Collin1-3/+3
Don't assume that Clang defines __GNUC__ as the extensions are available in clang-cl as well (and possibly in some other Clang variants?).
2024-02-14Tests: tuktest.h: Add a missing word to a comment.Lasse Collin1-2/+2
2024-02-14Tests: tuktest.h: Fix the comment about STest.Lasse Collin1-1/+2
2024-02-15Bump version for 5.5.2beta.larhzu/v5.5.2betaJia Tan3-4/+4
2024-02-14liblzma: Fix validate_map.sh.Lasse Collin1-1/+1
Adding the SPDX license identifier changed the line numbers.
2024-02-14Build: Start the generated ChangeLog from around 5.4.0 instead of 5.2.0.Lasse Collin1-1/+1
2024-02-14Fixed NEWS for 5.5.2beta.Lasse Collin1-2/+6
2024-02-14liblzma: Silence warnings in --enable-small build.Lasse Collin2-0/+3
2024-02-14Build: Install COPYING.0BSD as part of docs.Lasse Collin1-0/+1
2024-02-14Docs: List COPYING.0BSD in README.Lasse Collin1-0/+1
2024-02-14Docs: Include doc/examples/11_file_info.c in tarballs.Lasse Collin1-0/+1
It was added in 2017 in c2e29f06a7d1e3ba242ac2fafc69f5d6e92f62cd but it never got into any release tarballs because it was forgotten to be added to Makefile.am.
2024-02-14liblzma: Silence a warning.Lasse Collin1-1/+1
2024-02-14Add NEWS for 5.5.2beta.Lasse Collin1-0/+60
2024-02-14xz: Mention lzmainfo if trying to use 'lzma --list'.Lasse Collin1-2/+14
This kind of fixes the problem reported here: https://bugs.launchpad.net/ubuntu/+source/xz-utils/+bug/1291020
2024-02-14liblzma: Add comments.Lasse Collin2-2/+18
2024-02-14Scripts: Add lz4 support to xzgrep and xzdiff.Lasse Collin4-10/+19
2024-02-14liblzma: Choose the range decoder variants using a bitmask macro.Lasse Collin1-11/+53
2024-02-14xz: Fix outdated threading related info on the man page.Lasse Collin1-8/+14
2024-02-14liblzma: Range decoder: Add x86-64 inline assembly.Lasse Collin1-0/+491
It's compatible with GCC and Clang.
2024-02-14liblzma: Range decoder: Add branchless C code.Lasse Collin1-0/+76
It's used only for basic bittrees and fixed-size reverse bittree because those showed a clear benefit on x86-64 with GCC and Clang. The other methods were more mixed and thus are commented out but they should be tested on other archs.
2024-02-14liblzma: Clarify a comment.Lasse Collin1-3/+6
2024-02-14liblzma: LZMA decoder: Optimize loop comparison.Lasse Collin2-4/+11
But now it needs one more local variable.
2024-02-14liblzma: Optimize literal_subcoder() macro slightly.Lasse Collin5-22/+24
2024-02-14liblzma: LZ decoder: Add unlikely().Lasse Collin1-1/+1
2024-02-14liblzma: LZ decoder: Remove a useless unlikely().Lasse Collin1-1/+1
2024-02-14liblzma: Optimize LZ decoder slightly.Lasse Collin3-60/+88
Now extra buffer space is reserved so that repeating bytes for any single match will never need to copy from two places (both the beginning and the end of the buffer). This simplifies dict_repeat() and helps a little with speed. This seems to reduce .lzma decompression time about 2 %, so with .xz and CRC it could be slightly less. The small things add up still.
2024-02-14liblzma: LZMA decoder: Get rid of next_state[].Lasse Collin3-24/+24
It's not completely obvious if this is better in the decoder. It should be good if compiler can avoid creating a branch (like using CMOV on x86). This also makes lzma_encoder.c use the new macros.
2024-02-14liblzma: LZMA decoder improvements.Lasse Collin3-200/+210
This adds macros for bittree decoding which prepares the code for alternative C versions and inline assembly.
2024-02-14liblzma: Creates Non-resumable and Resumable modes for lzma_decoder.Jia Tan2-213/+521
The new decoder resumes the first decoder loop in the Resumable mode. Then, the code executes in Non-resumable mode until it detects that it cannot guarantee to have enough input/output to decode another symbol. The Resumable mode is how the decoder has always worked. Before decoding every input bit, it checks if there is enough space and will save its location to be resumed later. When the decoder has more input/output, it jumps back to the correct sequence in the Resumable mode code. When the input/output buffers are large, the Resumable mode is much slower than the Non-resumable because it has more branches and is harder for the compiler to optimize since it is in a large switch block. Early benchmarking shows significant time improvement (8-10% on gcc and clang x86) by using the Non-resumable code as much as possible.
2024-02-14liblzma: Creates separate "safe" range decoder mode.Jia Tan2-103/+82
The new "safe" range decoder mode is the same as old range decoder, but now the default behavior of the range decoder will not check if there is enough input or output to complete the operation. When the buffers are close to fully consumed, the "safe" operations must be used instead. This will improve speed because it will reduce the number of branches needed for most of the range decoder operations.
2024-02-14doxygen/footer.html: Add missing closing tags and don't open a new tab.Lasse Collin1-2/+4
The footer template from Doxygen has the closing </body> </html> as Doxygen doesn't add them otherwise. target="_blank" was omitted as it's not useful here but it can be slightly annoying as one cannot just go back in the browser history. Since the footer links to the license file in the same directory and not to CC website, the rel attributes can be omitted.
2024-02-14Tweak the expressions in AUTHORS.Lasse Collin1-8/+23
2024-02-14Translations: Add the man page translators into man page header comment.Lasse Collin3-7/+26
It looked odd to only have the original English authors listed in the header comments of the translated files.
2024-02-14Translations: Translate also messages of lzmainfo.Lasse Collin1-0/+2
lzmainfo has had translation support since 2009 at least but it was never added to po/POTFILES.in so the messages weren't translated. It's a very rarely needed tool so it's not too bad. This also adds src/xz/mytime.c to po/POTFILES.in although there are no translatable strings. It's simpler this way so that it won't be forgotten if strings were ever added to that file.
2024-02-14Translations: Add custom .pot header with SPDX license identifier.Lasse Collin3-0/+16
The same is used for both po/xz.pot and po4a/xz-man.pot.
2024-02-14Translations: po4a/update-po: Add copyright notice to xz-man.pot.Lasse Collin1-1/+1
All man pages are under 0BSD now so this is simple now.
2024-02-14Update COPYING about the man pages of the scripts.Lasse Collin1-3/+3
2024-02-14xzdiff, xzgrep, and xzmore: Rewrite the man pages.Lasse Collin3-116/+173
The main reason is a kind of silly one: xz-man.pot contains strings from all man pages in XZ Utils. The man pages of xzdiff, xzgrep, and xzmore were under GPLv2 and the rest under 0BSD. Thus xz-man.pot contained strings under two licences. po4a creates the translated man pages from the combined 0BSD+GPLv2 xz-man.pot. I haven't liked this mixing in xz-man.pot but the Translation Project requires that all man pages must be in the same .pot file. So a separate xz-man-gpl.pot wasn't an option. Since these man pages are short, rewriting them was quick enough. Now xz-man.pot is entirely under 0BSD and marking the per-file licenses is simpler. As a bonus, some wording hopefully is now slightly better although it's perhaps a matter of taste. NOTE: In xzgrep.1, the EXIT STATUS section was written by me in the commit d796b6d7fdb8b7238b277056cf9146cce25db604 so that's why that section could be taken as is from the old xzgrep.1.
2024-02-14xzless: Update man page slightly.Lasse Collin1-4/+4
The xz tool can decompress three file formats and xzless has always supported uncompressed files too.
2024-02-14Translations: Change po/Makevars to add a copyright notice to po/xz.pot.Lasse Collin1-2/+2
2024-02-14Translations: Update po/Makevars to use the template from gettext 0.22.4.Lasse Collin1-5/+46
Also add SPDX license identifier now that there is a known license.
2024-02-14liblzma: Include the SPDX license identifier 0BSD to generated files.Lasse Collin11-26/+50
Perhaps the generated files aren't even copyrightable but using the same license for them as for the rest of the liblzma keeps things more consistent for tools that look for license info.
2024-02-14liblzma: Fix compilation of price_tablegen.c.Lasse Collin2-1/+9
It is built and run only manually so this didn't matter unless one wanted to regenerate the price_table.c.
2024-02-14Add SPDX license identifiers to GPL, LGPL, and FSFULLR files.Lasse Collin22-6/+37
2024-02-14Add SPDX license identifier into 0BSD source code files.Lasse Collin290-58/+588
2024-02-14liblzma: Sync the AUTHORS fix about SHA-256 to lzma.h.Lasse Collin1-6/+4
2024-02-14Change most public domain parts to 0BSD.Lasse Collin288-911/+100
Translations and doc/xz-file-format.txt and doc/lzma-file-format.txt were not touched. COPYING.0BSD was added.
2024-02-14Fix SHA-256 authors.Lasse Collin2-14/+6
The initial commit 5d018dc03549c1ee4958364712fb0c94e1bf2741 in 2007 had a comment in sha256.c that the code is based on Crypto++ Library 5.5.1. In 2009 the Authors list in sha256.c and the AUTHORS file was updated with information that the code had come from Crypto++ but via 7-Zip. I know I had viewed 7-Zip's SHA-256 code but back then the C code has been identical enough with Crypto++, so I don't why I thought the author info would need that extra step via 7-Zip for this single file. Another error is that I had mixed sha.* and shacal2.* files when checking for author info in Crypto++. The shacal2.* files aren't related to liblzma's sha256.c and thus Kevin Springle's code in Crypto++ isn't either.
2024-02-14Remove macosx/build.sh.Lasse Collin2-114/+0
It was last updated in 2013.
2024-02-14Doc: Remove doc/examples_old.Lasse Collin3-255/+0
It was good to keep these around in parallel with the newer examples but I think it's OK to remove the old ones at this point.
2024-02-13Tests: Add RISC-V filter support in a few places.Jia Tan2-0/+12
2024-02-13liblzma: Fix build error if only RISC-V BCJ filter is enabled.Jia Tan1-1/+3
If any other BCJ filter was enabled for encoding or decoding, then this was not a problem.
2024-02-13Translations: Update the Korean translation.Jia Tan1-242/+284
2024-02-13Translations: Update the Korean man page translations.Jia Tan1-605/+770
2024-02-13Translations: Update the Chinese (simplified) translation.Jia Tan1-156/+268
2024-02-09xzless: Use ||- in LESSOPEN with with "less" 451 and newer.Lasse Collin1-1/+8
2024-02-09xzless: Use --show-preproc-errors with "less" 632 and newer.Lasse Collin1-2/+9
This makes "less" show a warning if a decompression error occurred.
2024-02-09liblzma: Fix typo discovered by codespell.Jia Tan1-1/+1
2024-02-09Translations: Update the Swedish translation.Jia Tan1-166/+254
2024-02-08Translations: Update the Spanish translation.Jia Tan1-11/+11
2024-02-07Translations: Update the Spanish translation.Jia Tan1-166/+253
2024-02-07Translations: Update the Polish translation.Jia Tan1-162/+249
2024-02-07Translations: Update the German translation.Jia Tan1-154/+242
2024-02-07Translations: Update the German man page translations.Jia Tan1-601/+752
2024-02-06Translations: Update the Romanian translation.Jia Tan1-164/+252
2024-02-06Translations: Update the Romanian man page translations.Jia Tan1-793/+966
2024-02-07Translations: Update the Ukrainian translation.Jia Tan1-155/+242
2024-02-06Translations: Update the Ukrainian man page translations.Jia Tan1-599/+764
2024-02-02Update AUTHORS.Jia Tan1-1/+2
2024-02-02liblzma: Update Authors list in crc32_arm64.h.Jia Tan1-0/+1
2024-02-01liblzma: Check HAVE_USABLE_CLMUL before omitting CRC32 table.Jia Tan1-2/+2
This was split from the prior commit so it could be easily applied to the 5.4 branch. Closes: https://github.com/tukaani-project/xz/pull/77
2024-02-01liblzma: Check HAVE_USABLE_CLMUL before omitting CRC64 table.Jia Tan1-2/+2
If liblzma is configured with --disable-clmul-crc CFLAGS="-msse4.1 -mpclmul", then it will fail to compile because the generic version must be used but the CRC tables were not included.
2024-02-01liblzma: Only use ifunc in crcXX_fast.c if its needed.Jia Tan2-6/+6
The code was using HAVE_FUNC_ATTRIBUTE_IFUNC instead of CRC_USE_IFUNC. With ARM64, ifunc is incompatible because it requires non-inline function calls for runtime detection.
2024-02-01Docs: Add --disable-arm64-crc32 description to INSTALL.Jia Tan1-1/+11
2024-02-01liblzma: Omit CRC tables when not needed with ARM64 optimizations.Jia Tan3-5/+25
This is similar to the existing x86-64 CLMUL conditions to omit the tables. They were slightly refactored to improve readability.
2024-02-01liblzma: Rename crc32_aarch64.h to crc32_arm64.h.Jia Tan6-116/+122
Even though the proper name for the architecture is aarch64, this project uses ARM64 throughout. So the rename is for consistency. Additionally, crc32_arm64.h was slightly refactored for the following changes: * Added MSVC, FreeBSD, and macOS support in is_arch_extension_supported(). * crc32_arch_optimized() now checks the size when aligning the buffer. * crc32_arch_optimized() loop conditions were slightly modified to avoid both decrementing the size and incrementing the buffer pointer. * Use the intrinsic wrappers defined in <arm_acle.h> because GCC and Clang name them differently. * Minor spacing and comment changes.
2024-02-01liblzma: Refactor crc_common.h.Jia Tan3-42/+82
The CRC_GENERIC is now split into CRC32_GENERIC and CRC64_GENERIC, since the ARM64 optimizations will be different between CRC32 and CRC64. For the same reason, CRC_ARCH_OPTIMIZED is split into CRC32_ARCH_OPTIMIZED and CRC64_ARCH_OPTIMIZED. ifunc will only be used with x86-64 CLMUL because the runtime detection methods needed with ARM64 are not compatible with ifunc.
2024-02-01CMake: Add support for ARM64 CRC32 instruction detection.Jia Tan1-0/+50
2024-02-01Build: Add support for ARM64 CRC32 instruction detection.Jia Tan1-0/+52
This adds --enable-arm64-crc32/--disable-arm64-crc32 (enabled by default) for using the ARM64 CRC32 instruction. This can be disabled if one knows the binary will never need to run on an ARM64 machine with this instruction extension.
2024-01-27Speed up CRC32 calculation on ARM64Chenxi Mao6-9/+130
The CRC32 instructions in ARM64 can calculate the CRC32 result for 8 bytes in a single operation, making the use of ARM64 instructions much faster compared to the general CRC32 algorithm. Optimized CRC32 will be enabled if ARM64 has CRC extension running on Linux. Signed-off-by: Chenxi Mao <chenxi.mao2013@gmail.com>
2024-01-26Bump version number for 5.5.1alpha.larhzu/v5.5.1alphaJia Tan3-3/+3
2024-01-26Add NEWS for 5.5.1alphaJia Tan1-0/+80
2024-01-26Add NEWS for 5.4.6.Jia Tan1-0/+22
2024-01-24Move doc/logo/xz-logo.png to "doc" and Doxygen footer to "doxygen".Lasse Collin4-3/+3
The footer isn't a complete HTML file so having it in the doxygen directory is a tiny bit clearer.
2024-01-25README: Add COPYING.CC-BY-SA-4.0 entry to section 1.1.Jia Tan1-18/+20
The Overall documentation section (1.1) table spacing had to be adjusted since the filename was very long.
2024-01-25Build: Add the logo and license to the release.Jia Tan1-0/+2
2024-01-25COPYING: Add the license for the XZ logo.Jia Tan2-0/+432
2024-01-25Doxygen: Added the XZ logo and copyright information.Jia Tan3-3/+14
The PROJECT_LOGO field is now used to include the XZ logo. The footer of each page now lists the copyright information instead of the default footer. The license is also copied to statisfy the copyright and so the link in the documentation can be local.
2024-01-23xz: Use threaded mode by defaut (as if --threads=0 was used).Lasse Collin3-3/+16
This hopefully does more good than bad: + It's faster by default. + Only the threaded compressor creates files that can be decompressed in threaded mode. - Compression ratio is worse, usually not too much though. When it matters, -T1 must be used. - Memory usage increases. - Scripts that assume single-threaded mode but don't use -T1 will possibly use too much resources, for example, if they run multiple xz processes in parallel to compress multiple files. - Output from single-threaded and multi-threaded compressors differ but such changes could happen for other reasons too (they just haven't happened since 5.0.0).
2024-01-23CI: Use RISC-V filter when building with BCJ support.Jia Tan1-2/+2
2024-01-23Tests: Use smaller dictionary size in RISC-V test files.Jia Tan2-0/+0
2024-01-23Tests: Skip RISC-V test files if decoder was not built.Jia Tan1-0/+5
2024-01-23xz: Man page: Add more examples of LZMA2 options with BCJ filters.Lasse Collin1-7/+31
2024-01-23liblzma: RISC-V filter: Use byte-by-byte access.Lasse Collin1-30/+84
Not all RISC-V processors support fast unaligned access so it's better to read only one byte in the main loop. This can be faster even on x86-64 when compared to reading 32 bits at a time as half the time the address is only 16-bit aligned. The downside is larger code size on archs that do support fast unaligned access.
2024-01-23xz: Update xz -lvv for RISC-V filter.Jia Tan1-0/+10
Version 5.6.0 will be shown, even though upcoming alphas and betas will be able to support this filter. 5.6.0 looks nicer in the output and people shouldn't be encouraged to use an unstable version in production in any way.
2024-01-23Tests: Add two RISC-V Filter test files.Jia Tan3-0/+8
These test files achieve 100% code coverage in src/liblzma/simple/riscv.c. They contain all of the instructions that should be filtered and a few cases that should not.
2024-01-23xz: Update message in --long-help for RISC-V Filter.Jia Tan1-0/+1
2024-01-23xz: Update the man page for the RISC-V Filter.Jia Tan1-1/+2
A special note was added to suggest using four-byte alignment when the compressed instruction extension is not present in a RISC-V binary.
2024-01-23Tests: Add RISC-V Filter test in test_compress.sh.Jia Tan1-0/+1
2024-01-23liblzma: Update string_conversion.c to support RISC-V Filter.Jia Tan1-0/+5
2024-01-23CMake: Support RISC-V BCJ Filter for encoding and decoding.Jia Tan1-0/+1
2024-01-23liblzma: Add RISC-V BCJ filter.Jia Tan9-2/+742
The new Filter ID is 0x0B. Thanks to Chien Wong <m@xv97.com> for the initial version of the Filter, the xz CLI updates, and the Autotools build system modifications. Thanks to Igor Pavlov for his many contributions to the design of the filter.
2024-01-19Docs: Update .xz file format specification to 1.2.0.Jia Tan1-12/+17
The new RISC-V filter was added to the specification, in addition to updating the specification URL.
2024-01-19xz: Update website URLs in the man pages.Jia Tan2-5/+5