aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)AuthorFilesLines
2023-08-28Translations: Update the Esperanto translation.Jia Tan1-2/+2
2023-08-26Translations: Update the Esperanto translation.Jia Tan1-18/+29
2023-08-14Docs: Update INSTALL for --enable-threads method win95.Jia Tan1-2/+4
The Autotools build allows win95 threads and --enable-small together now if the compiler supports __attribute__((__constructor__)).
2023-08-14CMake: Conditionally allow win95 threads and --enable-small.Jia Tan1-8/+20
2023-08-09Build: Conditionally allow win95 threads and --enable-small.Jia Tan1-8/+13
When the compiler supports __attribute__((__constructor__)) mythread_once() is never used, even with --enable-small. A configuration with win95 threads and --enable-small will compile and be thread safe so it can be allowed. This isn't a very common configuration since MSVC does not support __attribute__((__constructor__)), but MINGW32 and CLANG32 environments for MSYS2 can use win95 threads and have __attribute__((__constructor__)) support.
2023-08-08mythread.h: Fix typo error in Vista threads mythread_once().Jamaika11-1/+1
The "once_" variable was accidentally referred to as just "once". This prevented building with Vista threads when HAVE_FUNC_ATTRIBUTE_CONSTRUCTOR was not defined.
2023-08-04codespell: Add .codespellrc to set default options.Jia Tan1-0/+24
The .codespellrc allows setting default options to avoid false positive matches, set additional dictionaries, etc. For now, codespell can be used locally before committing doc and comment changes. It should help prevent silly errors and fix up commits in the future.
2023-08-03Tests: Style fixes to test_lzip_decoder.c.Jia Tan1-12/+24
2023-08-03Translations: Update the Chinese (simplified) translation.Jia Tan1-1/+1
2023-08-02xz: Omit an empty paragraph on the man page.Lasse Collin1-1/+0
2023-08-02Add NEWS for 5.4.4.Jia Tan1-0/+43
2023-08-02build-aux/manconv.sh: Fix US-ASCII and UTF-8 output.Lasse Collin1-2/+2
groff defaults to SGR escapes. Using -P-c passes -c to grotty which restores the old behavior. Perhaps there is a better way to get pure plain text output but this works for now.
2023-08-01Update THANKS.Lasse Collin1-0/+1
2023-08-01Update THANKS.Lasse Collin1-0/+1
2023-08-01mythread.h: Disable signal functions in builds targeting Wasm + WASI.ChanTsune1-1/+1
signal.h in WASI SDK doesn't currently provide sigprocmask() or sigset_t. liblzma doesn't need them so this change makes liblzma and xzdec build against WASI SDK. xz doesn't build yet and the tests don't either as tuktest needs setjmp() which isn't (yet?) implemented in WASI SDK. Closes: https://github.com/tukaani-project/xz/pull/57 See also: https://github.com/tukaani-project/xz/pull/56 (The original commit was edited a little by Lasse Collin.)
2023-08-01Add newline to end of .gitignore.Jia Tan1-1/+1
Newline was accidentally removed in commit 01cbb7f023ee7fda8ddde04bd17cf7d3c2418706.
2023-07-31Docs: Fix typos found by codespellDimitri Papadopoulos Orfanos18-29/+29
2023-07-28Update .gitignore.Jia Tan1-0/+4
2023-07-28CMake: Conditionally allow the creation of broken symlinks.Jia Tan1-7/+75
The CMake build will try to create broken symlinks on Unix and Unix-like platforms. Cygwin and MSYS2 are Unix-like, but may not be able to create broken symlinks. The value of the CYGWIN or MSYS environment variables determine if broken symlinks are valid. The default for MSYS2 does not allow for broken symlinks, so the CMake build has been broken for MSYS2 since commit 80a1a8bb838842a2be343bd88ad1462c21c5e2c9.
2023-07-28CI: Fix windows-ci dependency installation.Jia Tan1-3/+3
All of the MSYS2 environments need make, and it does not come with the toolchain package. The toolchain package will install the needed compiler toolchains since without this package CMake cannot properly generate the Makefiles.
2023-07-28CI: Update ci_build.sh CMake to always make Unix Makefiles.Jia Tan1-2/+2
The default for many of the MSYS2 environments is for CMake to create Ninja build files. This would complicate the build script since we would need a different command to run the tests. Its simpler to always use Unix Makefiles so that "make test" is always a usable target for testing.
2023-07-25CI: Test CMake builds and test framework with MSYS2.Jia Tan1-12/+20
2023-07-25CI: Windows CI rename system matrix variable -> msys2_env.Jia Tan1-10/+5
Calling the MSYS2 environment "system" was a bit vague and should be more specific.
2023-07-24CI: Add Clang64 MSYS2 environment to Windows CI.Jia Tan1-0/+1
2023-07-24liblzma: Prevent an empty translation unit in Windows builds.Jia Tan1-1/+5
To workaround Automake lacking Windows resource compiler support, an empty source file is compiled to overwrite the resource files for static library builds. Translation units without an external declaration are not allowed by the C standard and result in a warning when used with -Wempty-translation-unit (Clang) or -pedantic (GCC).
2023-07-22Translations: Update the Vietnamese translation.Jia Tan1-17/+28
2023-07-22CI: Add Windows runner for Autotools builds with MSYS2.Jia Tan1-0/+119
Only a subset of the tests run by the Linux and MacOS Autotools builds are run. The most interesting tests are the ones that disable threads, encoders, and decoders. The Windows runner will only be run manually since these tests will likely take much longer than the Linux and MacOS runners. This runner should be used before merging any large features and before releases. Currently the clang64 environment fails to due to a warning and -Werror is enabled for the CI tests. This is still an early version since the CMake build can be done for MSVC and optionally each of the MSYS2 environments. GitHub does not allow manually running the CI tests unless the workflow is checked on the default branch so checking in a minimum version is a good idea. Thanks to Arthur S for the original proposing the original patch. Closes: https://github.com/tukaani-project/xz/pull/34
2023-07-22CI: Add argument to ci_build.sh to pass flags to autogen.sh.Jia Tan1-2/+7
2023-07-21Tests: Skip .lz files in test_files.sh if not configured.Jia Tan1-1/+1
Previously if the lzip decoder was not configured then test_files.sh would pass the lzip tests instead of skipping them.
2023-07-20Tests: Add ARM64 filter test to test_compress.sh.Jia Tan1-0/+1
2023-07-20Translations: Update the Croatian translation.Jia Tan1-19/+30
2023-07-20Translations: Update the Korean man page translations.Jia Tan1-626/+629
2023-07-20Translations: Update the Korean translation.Jia Tan1-17/+28
2023-07-20Translations: Update the Polish translation.Jia Tan1-18/+29
2023-07-20Translations: Update the German man page translations.Jia Tan1-626/+629
2023-07-20Translations: Update the German translation.Jia Tan1-18/+29
2023-07-20Translations: Update the Chinese (simplified) translation.Jia Tan1-18/+29
2023-07-20Translations: Update the Swedish translation.Jia Tan1-18/+29
2023-07-20Translations: Update the Ukrainian man page translations.Jia Tan1-625/+628
2023-07-20Translations: Update the Ukrainian translation.Jia Tan1-17/+28
2023-07-20Translations: Update the Spanish translation.Jia Tan1-18/+29
2023-07-20Translations: Update the Romanian translation.Jia Tan1-18/+30
2023-07-20Translations: Update the Romanian man page translations.Jia Tan1-625/+629
2023-07-19liblzma: Suppress -Wunused-function warning.Jia Tan1-0/+10
Clang 16.0.0 and earlier have a bug that the ifunc resolver function triggers the -Wunused-function warning. The resolver function is static and only "used" by the __attribute__((__ifunc()__)). At this time, the bug is still unresolved, but has been reported: https://github.com/llvm/llvm-project/issues/63957 This is not a problem in GCC.
2023-07-18liblzma: Reword lzma_str_list_filters() documentation.Jia Tan1-1/+1
This further improves the documentation from commit f36ca7982f6bd5e9827219ed4f3c5a1fbf5d7bdf. The previous wording of "supported options" was slightly misleading since the options that are printed are the ones that are relevant for encoding/decoding. It is not about which options can or must be specified.
2023-07-18liblzma: Improve comment in string_conversion.c.Jia Tan1-2/+2
The comment used "flag" when referring to decoder options. Just referring to them as options is more clear and consistent.
2023-07-18xz: Translate the second "%s: " in message.c since French needs "%s : ".Lasse Collin1-1/+1
This string is used to print a filename when using "xz -v" and stderr isn't a terminal.
2023-07-18xz: Make "%s: %s" translatable because French needs "%s : %s".Lasse Collin4-14/+18
2023-07-18liblzma: Tweak #if condition in memcmplen.h.Lasse Collin1-2/+2
Maybe ICC always #defines _MSC_VER on Windows but now it's very clear which code will get used.
2023-07-18liblzma: Omit unnecessary parenthesis in a preprocessor directive.Lasse Collin1-2/+2
2023-07-18xz: Update Authors list in a few files.Jia Tan5-5/+10
2023-07-18Docs: Add a new section to INSTALL for Tests.Jia Tan1-17/+64
The new Tests section describes basic information about the tests, how to run them, and important details when cross compiling. We have had a few questions about how to compile the tests without running them, so hopefully this information will help others with the same question in the future. Fixes: https://github.com/tukaani-project/xz/issues/54
2023-07-17Docs: Update README.Jia Tan1-0/+4
This adds an entry to "Other implementations of the .xz format" for XZ for Java.
2023-07-17xz: Fix typo in man page.Jia Tan1-1/+1
The Memory limit information section described three output columns when it actually has six. This was reworded to "multiple" to make it more future proof.
2023-07-17xz: Minor clean up for coder.cJia Tan1-32/+21
* Moved max_block_list_size from a global to local variable. * Reworded error message in validate_block_list_filter(). * Removed helper function filter_chain_error(). * Changed 1 << X to 1U << X in many places
2023-07-17xz: Update man page Authors and date.Jia Tan1-2/+3
2023-07-17xz: Add a section to man page for robot mode --filters-help.Jia Tan1-2/+30
2023-07-17xz: Slight reword in xz man page for consistency.Jia Tan1-1/+1
Changed will print => prints in xz --robot --version description to match --robot --info-memory description.
2023-07-17xz: Reorder robot mode subsections in the man page.Jia Tan1-96/+96
The order is now consistent with the order the command line arguments are documented earlier in the man page. The new order is: 1. --list 2. --info-memory 3. --version Instead of the previous order: 1. --version 2. --info-memory 3. --list
2023-07-17xz: Update man page for new --filters-help option.Jia Tan1-0/+10
2023-07-17xz: Add a new --filters-help option.Jia Tan3-0/+43
The --filters-help can be used to help create filter chains with the --filters and --filtersX options. The message in --long-help is too short to fully explain the syntax to construct complex filter chains. In --robot mode, xz will only print the output from liblzma function lzma_str_list_filters.
2023-07-17xz: Update the man page for --block-list and --filtersXJia Tan1-26/+80
The --block-list option description needed updating since the new --filtersX option changes how it can be used. The new entry for --filters1=FILTERS ... --filter9=FILTERS was created right after the --filters option.
2023-07-17xz: Update --long-help for the new --filtersX option.Jia Tan1-2/+10
2023-07-17xz: Ignore filter chains that are set but never used in --block-list.Jia Tan1-18/+48
If a filter chain is set but not used in --block-list, it introduced unexpected behavior such as requiring an unneeded amount of memory to compress, reducing the number of threads in multi-threaded encoding, and printing an incorrect amount of memory needed to decompress. This also renames filters_init_mask => filters_used_mask. A filter is assumed to be used if it is specified in --filtersX until coder_set_compression_settings() determines which filters are referenced in --block-list.
2023-07-17xz: Set the Block size for mt encoding correctly.Jia Tan1-1/+67
When opt_block_size is not used, the Block size for mt encoder is derived from the minimum of the largest Block specified by --block-list and the recommended Block size on all filter chains calculated by lzma_mt_block_size(). This avoids using unnecessary memory and ensures that all Blocks are large enough for the most memory needy filter chain.
2023-07-17xz: Validate --flush-timeout for all specified filter chains.Jia Tan1-8/+16
2023-07-17xz: Allows --block-list filters to scale down memory usage.Jia Tan1-55/+214
Previously, only the default filter chain could have its memory usage adjusted. The filter chains specified with --filtersX were not checked for memory usage. Now, all used filter chains will be adjusted if necessary.
2023-07-17xz: Do not include block splitting if encoders are disabled.Jia Tan1-9/+20
The block splitting logic and split_block() function are not needed if encoders are disabled. This will help slightly reduce the binary size when built without encoders and allow split_block() to use functions that require encoders being enabled.
2023-07-17xz: Free filters[] in debug mode.Jia Tan1-0/+10
This will only free filter chains created with --filters1-9 since the default filter chain may be set from a static function variable. The complexity to free the default filter chain is not worth the burden on code maintenance.
2023-07-17xz: Add a message if --block-list is used outside of xz compresssion.Jia Tan1-0/+11
--block-list is only supported with compression in xz format. This avoids silently ignoring when --block-list is unused.
2023-07-17xz: Create command line options for filters[1-9].Jia Tan3-60/+230
The new command line options are meant to be combined with --block-list. They work as an optional extension to --block-list to specify a custom filter chain for each block listed. The new options allow the creation of up to 9 reusable filter chains. For instance: xz --block-list=1:10MiB,3:5MiB,,2:5MiB,1:0 --filters1=delta--lzma2 \ --filters2=x86--lzma2 --filters3=arm64--lzma2 Will create the following blocks: 1. A block of size 10 MiB with filter chain delta, lzma2. 2. A block of size 5 MiB with filter chain arm64, lzma2. 3. A block of size 5 MiB with filter chain arm64, lzma2. 4. A block of size 5 MiB with filter chain x86, lzma2. 5. A block containing the rest of the file contents with filter chain delta, lzma2.
2023-07-17xz: Use lzma_filters_free() in forget_filter_chain().Jia Tan1-8/+10
This is a little cleaner than the previous implementation of forget_filter_chain(). It is also more consistent since lzma_str_to_filters() will always terminate the filter chain so there is no need to terminate it later in coder_set_compression_settings().
2023-07-17xz: Separate string to filter conversion into a helper function.Jia Tan1-13/+20
Converting from string to filter will also need to be done for block specific filter chains.
2023-07-17Tests: Use new --filters option in test_compress.shJia Tan1-10/+10
2023-07-17xz: Update --long-help and man page for new --filters option.Jia Tan2-5/+42
2023-07-17xz: Add --filters option to CLI.Jia Tan3-4/+58
The --filters option uses the new lzma_str_to_filters() function to convert a string into a full filter chain. Using this option will reset all previous filters set by --preset, --[filter], or --filters.
2023-07-14Tests: Improve feature testing for skipping.Jia Tan2-3/+3
Fixed a bug where test_compress_* would all fail if arm64 or armthumb filters were enabled for compression but arm was disabled. Since the grep tests only checked for "define HAVE_ENCODER_ARM", this would match on HAVE_ENCODER_ARM64 or HAVE_ENCODER_ARMTHUMB. Now the config.h feature test requires " 1" at the end to prevent the prefix problem. have_feature() was also updated for this even though there were known current bugs affecting it. This is just in case future features have a similar prefix problem.
2023-07-10Translations: Update the Chinese (traditional) translation.Jia Tan1-282/+377
2023-07-08liblzma: Remove non-portable empty initializer.Jia Tan1-1/+1
Commit 78704f36e74205857c898a351c757719a6c8b666 added an empty initializer {} to prevent a warning. The empty initializer is a GNU extension and results in a build failure on MSVC. The -wpedantic flag warns about empty initializers.
2023-07-08Translations: Update the Vietnamese translation.Jia Tan1-271/+349
2023-06-29Tests: Fix memory leaks in test_index.Jia Tan1-0/+11
Several tests were missing calls to lzma_index_end() to clean up the lzma_index structs. The memory leaks were discovered by using -fsanitize=address with GCC.
2023-06-29Tests: Fix memory leaks in test_block_header.Jia Tan1-16/+22
test_block_header was not properly freeing the filter options between calls to lzma_block_header_decode(). The memory leaks were discovered by using -fsanitize=address with GCC.
2023-06-29liblzma: Prevent uninitialzed warning in mt stream encoder.Jia Tan1-1/+1
This change only impacts the compiler warning since it was impossible for the wait_abs struct in stream_encode_mt() to be used before it was initialized since mythread_condtime_set() will always be called before mythread_cond_timedwait(). Since the mythread.h code is different between the POSIX and Windows versions, this warning was only present on Windows builds. Thanks to Arthur S for reporting the warning and providing an initial patch.
2023-06-28liblzma: Prevent warning for MSYS2 Windows build.Jia Tan1-2/+4
In lzma_memcmplen(), the <intrin.h> header file is only included if _MSC_VER and _M_X64 are both defined but _BitScanForward64() was previously used if _M_X64 was defined. GCC for MSYS2 defines _M_X64 but not _MSC_VER so _BitScanForward64() was used without including <intrin.h>. Now, lzma_memcmplen() will use __builtin_ctzll() for MSYS2 GCC builds as expected.
2023-06-28CI: Add test with -fsanitize=address,undefined.Jia Tan2-5/+26
ci_build.sh was updated to accept disabling of __attribute__ ifunc and CLMUL. This will allow -fsanitize=address to pass because ifunc is incompatible with -fsanitize=address. The CLMUL implementation has optimizations that potentially read past the buffer and mask out the unwanted bytes. This test will only run on Autotools Linux.
2023-06-28CI: Upgrade checkout action from v2 to v3.Jia Tan1-1/+1
2023-06-27Update THANKS.Jia Tan1-0/+1
2023-06-27Docs: Document the configure option --disable-ifunc in INSTALL.Jia Tan1-0/+8
2023-06-27Minor tweaks to style and comments.Lasse Collin2-8/+9
2023-06-27CMake: Rename CHECK_ATTR_IFUNC to ALLOW_ATTR_IFUNC.Lasse Collin1-3/+3
It's so that there's a clear difference in wording compared to liblzma's integrity check types.
2023-06-27liblzma: Add ifunc implementation to crc64_fast.c.Lasse Collin1-9/+26
The ifunc method avoids indirection via the function pointer crc64_func. This works on GNU/Linux and probably on FreeBSD too. The previous __attribute((__constructor__)) method is kept for compatibility with ELF platforms which do support ifunc. The ifunc method has some limitations, for example, building liblzma with -fsanitize=address will result in segfaults. The configure option --disable-ifunc must be used for such builds. Thanks to Hans Jansen for the original patch. Closes: https://github.com/tukaani-project/xz/pull/53
2023-06-27Add ifunc check to CMakeLists.txtHans Jansen1-0/+19
CMake build system will now verify if __attribute__((__ifunc__())) can be used in the build system. If so, HAVE_FUNC_ATTRIBUTE_IFUNC will be defined to 1.
2023-06-27Add ifunc check to configure.acHans Jansen1-0/+28
configure.ac will now verify if __attribute__((__ifunc__())) can be used in the build system. If so, HAVE_FUNC_ATTRIBUTE_IFUNC will be defined to 1.
2023-06-07CI: Add apt update command before installing dependencies.Jia Tan1-2/+6
Without the extra command, all of the CI tests were automatically failing because the Ubuntu servers could not be reached properly.
2023-06-07Update THANKS.Jia Tan1-0/+1
2023-06-06CMake: Protects against double find_packageBenjamin Buch1-7/+9
Boost iostream uses `find_package` in quiet mode and then again uses `find_package` with required. This second call triggers a `add_library cannot create imported target "ZLIB::ZLIB" because another target with the same name already exists.` This can simply be fixed by skipping the alias part on secondary `find_package` runs.
2023-05-31Translations: Update the Esperanto translation.Jia Tan1-93/+92
2023-05-31Translations: Update the Croatian translation.Jia Tan1-1/+1
2023-05-31Translations: Update the Chinese (simplified) translation.Jia Tan1-160/+157
2023-05-17Translations: Update German translation of man pages.Jia Tan1-40/+12
2023-05-17Translations: Update the German translation.Jia Tan1-95/+94
2023-05-17Translations: Update the Croatian translation.Jia Tan1-94/+93
2023-05-17Translations: Update Korean translation of man pages.Jia Tan1-2446/+567
2023-05-17Translations: Update the Korean translation.Jia Tan1-161/+158
2023-05-16Translations: Update the Spanish translation.Jia Tan1-161/+158
2023-05-16Translations: Update the Romanian translation.Jia Tan1-97/+98
2023-05-16Translations: Update Romanian translation of man pages.Jia Tan1-9/+10
2023-05-16Translations: Update Ukrainian translation of man pages.Jia Tan1-6/+6
2023-05-16Translations: Update the Ukrainian translation.Jia Tan1-162/+159
2023-05-16Translations: Update the Polish translation.Jia Tan1-161/+155
2023-05-16Translations: Update the Swedish translation.Jia Tan1-161/+158
2023-05-16Translations: Update the Esperanto translation.Jia Tan1-17/+17
2023-05-13liblzma: Slightly rewords lzma_str_list_filters() documentation.Jia Tan1-1/+1
Reword "options required" to "supported options". The previous may have suggested that the options listed were all required anytime a filter is used for encoding or decoding. The reword makes this more clear that adjusting the options is optional.
2023-05-12liblzma: Adds lzma_nothrow to MicroLZMA API functions.Jia Tan1-2/+3
None of the liblzma functions may throw an exception, so this attribute should be applied to all liblzma API functions.
2023-05-11liblzma: Exports lzma_mt_block_size() as an API function.Jia Tan7-22/+61
The lzma_mt_block_size() was previously just an internal function for the multithreaded .xz encoder. It is used to provide a recommended Block size for a given filter chain. This function is helpful to determine the maximum Block size for the multithreaded .xz encoder when one wants to change the filters between blocks. Then, this determined Block size can be provided to lzma_stream_encoder_mt() in the lzma_mt options parameter when intializing the coder. This requires one to know all the filter chains they are using before starting to encode (or at least the filter chain that will need the largest Block size), but that isn't a bad limitation.
2023-05-11liblzma: Creates IS_ENC_DICT_SIZE_VALID() macro.Jia Tan2-3/+9
This creates an internal liblzma macro to test if the dictionary size is valid for encoding.
2023-05-04Add NEWS for 5.4.3.Jia Tan1-0/+10
2023-05-04Add NEWS for 5.2.12.Jia Tan1-0/+14
2023-05-04Translations: Update the Croatian translation.Jia Tan1-3/+3
2023-05-04tuklib_integer.h: Reverts previous commit.Jia Tan1-2/+2
Previous commit 6be460dde07113fe3f08f814b61ddc3264125a96 would cause an error if the integer size was 32 bit.
2023-05-04tuklib_integer.h: Changes two other UINT_MAX == UINT32_MAX to >=.Jia Tan1-2/+2
2023-05-03tuklib_integer.h: Fix a recent copypaste error in Clang detection.Lasse Collin1-2/+2
Wrong line was changed in 7062348bf35c1e4cbfee00ad9fffb4a21aa6eff7. Also, this has >= instead of == since ints larger than 32 bits would work too even if not relevant in practice.
2023-04-25CI: Adds a build and test for small configuration.Jia Tan1-0/+5
2023-04-25CI: ci_build.sh allows configuring small build.Jia Tan1-1/+6
2023-04-20Update THANKS.Jia Tan1-0/+1
2023-04-19Windows: Include <intrin.h> when needed.Jia Tan2-0/+16
Legacy Windows did not need to #include <intrin.h> to use the MSVC intrinsics. Newer versions likely just issue a warning, but the MSVC documentation says to include the header file for the intrinsics we use. GCC and Clang can "pretend" to be MSVC on Windows, so extra checks are needed in tuklib_integer.h to only include <intrin.h> when it will is actually needed.
2023-04-19tuklib_integer: Use __builtin_clz() with Clang.Jia Tan1-3/+3
Clang has support for __builtin_clz(), but previously Clang would fallback to either the MSVC intrinsic or the regular C code. This was discovered due to a bug where a new version of Clang required the <intrin.h> header file in order to use the MSVC intrinsics. Thanks to Anton Kochkov for notifying us about the bug.
2023-04-14liblzma: Update project maintainers in lzma.h.Lasse Collin1-1/+1
AUTHORS was updated earlier, lzma.h was simply forgotten.
2023-04-13liblzma: Cleans up old commented out code.Jia Tan1-11/+0
2023-04-07Docs: Add missing word to SECURITY.md.Jia Tan1-1/+1
2023-04-07Update THANKS.Jia Tan1-0/+1
2023-04-07Docs: Minor edits to SECURITY.md.Jia Tan1-5/+20
2023-04-07Docs: Create SECURITY.mdGabriela Gutierrez1-0/+14
Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>
2023-03-29CI: Tests for disabling threading on CMake builds.Jia Tan2-5/+2
2023-03-29CI: Removes CMakeCache.txt between builds.Jia Tan1-0/+2
If the cache file is not removed, CMake will not reset configurations back to their default values. In order to make the tests independent, it is simplest to purge the cache. Unfortunatly, this will slow down the tests a little and repeat some checks.
2023-03-29CMake: Update liblzma-config.cmake generation.Jia Tan1-11/+22
Now that the threading is configurable, the liblzma CMake package only needs the threading library when using POSIX threads.
2023-03-29CMake: Allows setting thread method.Jia Tan1-40/+104
The thread method is now configurable for the CMake build. It matches the Autotools build by allowing ON (pick the best threading method), OFF (no threading), posix, win95, and vista. If both Windows and posix threading are both available, then ON will choose Windows threading. Windows threading will also not use: target_link_libraries(liblzma Threads::Threads) since on systems like MinGW-w64 it would link the posix threads without purpose.
2023-03-24CI: Runs CMake feature tests.Jia Tan1-114/+55
Now, CMake will run similar feature disable tests that the Autotools version did before. In order to do this without repeating lines in ci.yml, it now makes sense to use the GitHub Workflow matrix to create a loop.
2023-03-24CI: ci_build.sh allows CMake features to be configured.Jia Tan1-90/+143
Also included various clean ups for style and helper functions for repeated work.
2023-03-24CI: Change ci_build.sh to use bash instead of sh.Jia Tan1-1/+1
This script is only meant to be run as part of the CI build/test process on machines that are known to have bash (Ubuntu and MacOS). If this assumption changes in the future, then the bash specific commands will need to be replaced with a more portable option. For now, it is convenient to use bash commands.
2023-03-24CMake: Only build xzdec if decoders are enabled.Jia Tan1-1/+1
2023-03-23Build: Removes redundant check for LZMA1 filter support.Jia Tan1-4/+1
2023-03-23CMake: Bump maximum policy version to 3.26.Lasse Collin1-1/+1
It adds only one new policy related to FOLDERS which we don't use. This makes it clear that the code is compatible with the policies up to 3.26.
2023-03-23CMake: Conditionally build xz list.* files if decoders are enabled.Jia Tan1-2/+7
2023-03-23CMake: Allow configuring features as cache variables.Jia Tan1-137/+391
This allows users to change the features they build either in CMakeCache.txt or by using a CMake GUI. The sources built for liblzma are affected by this too, so only the necessary files will be compiled.
2023-03-21Build: Add a comment that AC_PROG_CC_C99 is needed for Autoconf 2.69.Lasse Collin1-0/+3
It's obsolete in Autoconf >= 2.70 and just an alias for AC_PROG_CC but Autoconf 2.69 requires AC_PROG_CC_C99 to get a C99 compiler.
2023-03-21Build: configure.ac: Use AS_IF and AS_CASE where required.Lasse Collin1-15/+15
This makes no functional difference in the generated configure (at least with the Autotools versions I have installed) but this change might prevent future bugs like the one that was just fixed in the commit 5a5bd7f871818029d5ccbe189f087f591258c294.
2023-03-21Update THANKS.Lasse Collin1-0/+1
2023-03-21Build: Fix --disable-threads breaking the building of shared libs.Lasse Collin1-8/+8
This is broken in the releases 5.2.6 to 5.4.2. A workaround for these releases is to pass EGREP='grep -E' as an argument to configure in addition to --disable-threads. The problem appeared when m4/ax_pthread.m4 was updated in the commit 6629ed929cc7d45a11e385f357ab58ec15e7e4ad which introduced the use of AC_EGREP_CPP. AC_EGREP_CPP calls AC_REQUIRE([AC_PROG_EGREP]) to set the shell variable EGREP but this was only executed if POSIX threads were enabled. Libtool code also has AC_REQUIRE([AC_PROG_EGREP]) but Autoconf omits it as AC_PROG_EGREP has already been required earlier. Thus, if not using POSIX threads, the shell variable EGREP would be undefined in the Libtool code in configure. ax_pthread.m4 is fine. The bug was in configure.ac which called AX_PTHREAD conditionally in an incorrect way. Using AS_CASE ensures that all AC_REQUIREs get always run. Thanks to Frank Busse for reporting the bug. Fixes: https://github.com/tukaani-project/xz/issues/45
2023-03-19liblzma: Silence -Wsign-conversion in SSE2 code in memcmplen.h.Lasse Collin1-1/+2
Thanks to Christian Hesse for reporting the issue. Fixes: https://github.com/tukaani-project/xz/issues/44
2023-03-18Add NEWS for 5.4.2.Jia Tan1-0/+48
2023-03-18Add NEWS for 5.2.11.Jia Tan1-0/+27
2023-03-18Update the copy of GNU GPLv3 from gnu.org to COPYING.GPLv3.Lasse Collin1-4/+4
2023-03-18Change a few HTTP URLs to HTTPS.Lasse Collin8-19/+19
The xz man page timestamp was intentionally left unchanged.
2023-03-18CMake: Fix typo in a comment.Jia Tan1-1/+1
2023-03-17Windows: build.bash: Copy liblzma API docs to the output package.Lasse Collin1-1/+2
2023-03-17Windows: Add microlzma_*.c to the VS project files.Lasse Collin6-0/+12
These should have been included in 5.3.2alpha already.
2023-03-17CMake: Add microlzma_*.c to the build.Lasse Collin1-0/+2
These should have been included in 5.3.2alpha already.
2023-03-17Build: Update comments about unaligned access to mention 64-bit.Lasse Collin2-6/+5
2023-03-17Tests: Update .gitignore.Lasse Collin1-1/+2
2023-03-17po4a/update-po: Display the script name consistently in error messages.Lasse Collin1-1/+1
2023-03-17Doc: Rename Doxygen HTML doc directory name liblzma => api.Jia Tan5-22/+22
When the docs are installed, calling the directory "liblzma" is confusing since multiple other files in the doc directory are for liblzma. This should also make it more natural for distros when they package the documentation.
2023-03-17liblzma: Remove note from lzma_options_bcj about the ARM64 exception.Jia Tan1-1/+1
This was left in by mistake since an early version of the ARM64 filter used a different struct for its options.
2023-03-17CI: Add doxygen as a dependency.Jia Tan1-3/+2
Autogen now requires --no-doxygen or having doxygen installed to run without errors.
2023-03-17COPYING: Add a note about the included Doxygen-generated HTML.Lasse Collin1-0/+11
2023-03-17Doc: Update PACKAGERS with details about liblzma API docs install.Jia Tan1-6/+16
2023-03-17liblzma: Add set lzma.h as the main page for Doxygen documentation.Jia Tan15-29/+2
The \mainpage command is used in the first block of comments in lzma.h. This changes the previously nearly empty index.html to use the first comment block in lzma.h for its contents. lzma.h is no longer documented separately, but this is for the better since lzma.h only defined a few macros that users do not need to use. The individual API header files all have a disclaimer that they should not be #included directly, so there should be no confusion on the fact that lzma.h should be the only header used by applications. Additionally, the note "See ../lzma.h for information about liblzma as a whole." was removed since lzma.h is now the main page of the generated HTML and does not have its own page anymore. So it would be confusing in the HTML version and was only a "nice to have" when browsing the source files.
2023-03-17Build: Generate doxygen documentation in autogen.sh.Jia Tan1-6/+29
Another command line option (--no-doxygen) was added to disable creating the doxygen documenation in cases where it not wanted or if the doxygen tool is not installed.
2023-03-17Build: Create doxygen/update-doxygen script.Jia Tan2-0/+112
This is a helper script to generate the Doxygen documentation. It can be run in 'liblzma' or 'internal' mode by setting the first argument. It will default to 'liblzma' mode and only generate documentation for the liblzma API header files. The helper script will be run during the custom mydist hook when we create releases. This hook already alters the source directory, so its fine to do it here too. This way, we can include the Doxygen generated files in the distrubtion and when installing. In 'liblzma' mode, the JavaScript is stripped from the .html files and the .js files are removed. This avoids license hassle from jQuery and other libraries that Doxygen 1.9.6 puts into jquery.js in minified form.
2023-03-17Build: Install Doxygen docs and include in distribution if generated.Jia Tan1-0/+18
Added a install-data-local target to install the Doxygen documentation only when it has been generated. In order to correctly remove the docs, a corresponding uninstall-local target was added. If the doxygen docs exist in the source tree, they will also be included in the distribution now too.
2023-03-17Doxygen: Refactor Doxyfile.in to doxygen/Doxyfile.Lasse Collin4-309/+456
Instead of having Doxyfile.in configured by Autoconf, the Doxyfile can have the tags that need to be configured piped into the doxygen command through stdin with the overrides after Doxyfile's contents. Going forward, the documentation should be generated in two different modes: liblzma or internal. liblzma is useful for most users. It is the documentation for just the liblzma API header files. This is the default. internal is for people who want to understand how xz and liblzma work. It might be useful for people who want to contribute to the project.
2023-03-13Tests: Remove unused macros and functions.Jia Tan1-75/+0
2023-03-13liblzma: Defines masks for return values from lzma_index_checks().Jia Tan2-11/+34
2023-03-13Tests: Refactors existing lzma_index tests.Jia Tan1-544/+1492
Converts the existing lzma_index tests into tuktests and covers every API function from index.h except for lzma_file_info_decoder, which can be tested in the future.
2023-03-11xz: Simplify the error-label in Capsicum sandbox code.Lasse Collin1-15/+12
Also remove unneeded "sandbox_allowed = false;" as this code will never be run more than once (making it work with multiple input files isn't trivial).
2023-03-08xz: Make Capsicum sandbox more strict with stdin and stdout.Lasse Collin1-0/+8
2023-03-08Revert: "Add warning if Capsicum sandbox system calls are unsupported."Jia Tan1-6/+4
The warning causes the exit status to be 2, so this will cause problems for many scripted use cases for xz. The sandbox usage is already very limited already, so silently disabling this allows it to be more usable.
2023-03-07xz: Fix -Wunused-label in io_sandbox_enter().Jia Tan1-2/+2
Thanks to Xin Li for recommending the fix.
2023-03-06xz: Add warning if Capsicum sandbox system calls are unsupported.Jia Tan1-0/+2
The warning is only used when errno == ENOSYS. Otherwise, xz still issues a fatal error.
2023-03-06xz: Skip Capsicum sandbox system calls when they are unsupported.Jia Tan1-5/+17
If a system has the Capsicum header files but does not actually implement the system calls, then this would render xz unusable. Instead, we can check if errno == ENOSYS and not issue a fatal error.
2023-03-06xz: Reorder cap_enter() to beginning of capsicum sandbox code.Jia Tan1-3/+3
cap_enter() puts the process into the sandbox. If later calls to cap_rights_limit() fail, then the process can still have some extra protections.
2023-03-01liblzma: Clarify lzma_lzma_preset() documentation in lzma12.h.Jia Tan1-0/+5
lzma_lzma_preset() does not guarentee that the lzma_options_lzma are usable in an encoder even if it returns false (success). If liblzma is built with default configurations, then the options will always be usable. However if the match finders hc3, hc4, or bt4 are disabled, then the options may not be usable depending on the preset level requested. The documentation was updated to reflect this complexity, since this behavior was unclear before.
2023-02-27CMake: Require that the C compiler supports C99 or a newer standard.Lasse Collin1-0/+8
Thanks to autoantwort for reporting the issue and suggesting a different patch: https://github.com/tukaani-project/xz/pull/42
2023-02-24Tests: Small tweak to test-vli.c.Jia Tan1-0/+2
The static global variables can be disabled if encoders and decoders are not built. If they are not disabled and -Werror is used, it will cause an usused warning as an error.
2023-02-24liblzma: Replace '\n' -> newline in filter.h documentation.Jia Tan1-1/+1
The '\n' renders as a newline when the comments are converted to html by Doxygen.
2023-02-24liblzma: Shorten return description for two functions in filter.h.Jia Tan1-6/+2
Shorten the description for lzma_raw_encoder_memusage() and lzma_raw_decoder_memusage().
2023-02-24liblzma: Reword a few lines in filter.hJia Tan1-5/+5
2023-02-24liblzma: Improve documentation in filter.h.Jia Tan1-83/+143
All functions now explicitly specify parameter and return values. The notes and code annotations were moved before the parameter and return value descriptions for consistency. Also, the description above lzma_filter_encoder_is_supported() about not being able to list available filters was removed since lzma_str_list_filters() will do this.
2023-02-23Update THANKS.Lasse Collin1-0/+1
2023-02-23liblzma: Avoid null pointer + 0 (undefined behavior in C).Lasse Collin10-23/+77
In the C99 and C17 standards, section 6.5.6 paragraph 8 means that adding 0 to a null pointer is undefined behavior. As of writing, "clang -fsanitize=undefined" (Clang 15) diagnoses this. However, I'm not aware of any compiler that would take advantage of this when optimizing (Clang 15 included). It's good to avoid this anyway since compilers might some day infer that pointer arithmetic implies that the pointer is not NULL. That is, the following foo() would then unconditionally return 0, even for foo(NULL, 0): void bar(char *a, char *b); int foo(char *a, size_t n) { bar(a, a + n); return a == NULL; } In contrast to C, C++ explicitly allows null pointer + 0. So if the above is compiled as C++ then there is no undefined behavior in the foo(NULL, 0) call. To me it seems that changing the C standard would be the sane thing to do (just add one sentence) as it would ensure that a huge amount of old code won't break in the future. Based on web searches it seems that a large number of codebases (where null pointer + 0 occurs) are being fixed instead to be future-proof in case compilers will some day optimize based on it (like making the above foo(NULL, 0) return 0) which in the worst case will cause security bugs. Some projects don't plan to change it. For example, gnulib and thus many GNU tools currently require that null pointer + 0 is defined: https://lists.gnu.org/archive/html/bug-gnulib/2021-11/msg00000.html https://www.gnu.org/software/gnulib/manual/html_node/Other-portability-assumptions.html In XZ Utils null pointer + 0 issue should be fixed after this commit. This adds a few if-statements and thus branches to avoid null pointer + 0. These check for size > 0 instead of ptr != NULL because this way bugs where size > 0 && ptr == NULL will likely get caught quickly. None of them are in hot spots so it shouldn't matter for performance. A little less readable version would be replacing ptr + offset with offset != 0 ? ptr + offset : ptr or creating a macro for it: #define my_ptr_add(ptr, offset) \ ((offset) != 0 ? ((ptr) + (offset)) : (ptr)) Checking for offset != 0 instead of ptr != NULL allows GCC >= 8.1, Clang >= 7, and Clang-based ICX to optimize it to the very same code as ptr + offset. That is, it won't create a branch. So for hot code this could be a good solution to avoid null pointer + 0. Unfortunately other compilers like ICC 2021 or MSVC 19.33 (VS2022) will create a branch from my_ptr_add(). Thanks to Marcin Kowalczyk for reporting the problem: https://github.com/tukaani-project/xz/issues/36
2023-02-23liblzma: Adjust container.h for consistency with filter.h.Jia Tan1-11/+9
2023-02-23liblzma: Fix small typos and reword a few things in filter.h.Jia Tan1-7/+6
2023-02-23liblzma: Convert list of flags in lzma_mt to bulleted list.Jia Tan1-3/+6
2023-02-23liblzma: Fix typo in documentation in container.hJia Tan1-1/+1
lzma_microlzma_decoder -> lzma_microlzma_encoder
2023-02-23liblzma: Improve documentation for container.hJia Tan1-53/+93
Standardizing each function to always specify parameters and return values. Also moved the parameters and return values to the end of each function description.
2023-02-22CMake: Add LZIP decoder test to list of tests.Jia Tan1-0/+1
2023-02-17Update THANKS.Lasse Collin1-0/+1
2023-02-17Build: Use only the generic symbol versioning on MicroBlaze.Lasse Collin1-2/+10
On MicroBlaze, GCC 12 is broken in sense that __has_attribute(__symver__) returns true but it still doesn't support the __symver__ attribute even though the platform is ELF and symbol versioning is supported if using the traditional __asm__(".symver ...") method. Avoiding the traditional method is good because it breaks LTO (-flto) builds with GCC. See also: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101766 For now the only extra symbols in liblzma_linux.map are the compatibility symbols with the patch that spread from RHEL/CentOS 7. These require the use of __symver__ attribute or __asm__(".symver ...") in the C code. Compatibility with the patch from CentOS 7 doesn't seem valuable on MicroBlaze so use liblzma_generic.map on MicroBlaze instead. It doesn't require anything special in the C code and thus no LTO issues either. An alternative would be to detect support for __symver__ attribute in configure.ac and CMakeLists.txt and fall back to __asm__(".symver ...") but then LTO would be silently broken on MicroBlaze. It sounds likely that MicroBlaze is a special case so let's treat it as a such because that is simpler. If a similar issue exists on some other platform too then hopefully someone will report it and this can be reconsidered. (This doesn't do the same fix in CMakeLists.txt. Perhaps it should but perhaps CMake build of liblzma doesn't matter much on MicroBlaze. The problem breaks the build so it's easy to notice and can be fixed later.) Thanks to Vincent Fazio for reporting the problem and proposing a patch (in the end that solution wasn't used): https://github.com/tukaani-project/xz/pull/32
2023-02-16liblzma: Very minor API doc tweaks.Lasse Collin4-14/+14
Use "member" to refer to struct members as that's the term used by the C standard. Use lzma_options_delta.dist and such in docs so that in Doxygen's HTML output they will link to the doc of the struct member. Clean up a few trailing white spaces too.
2023-02-17liblzma: Adjust spacing in doc headers in bcj.h.Jia Tan1-7/+7