aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)AuthorFilesLines
2023-09-24Docs: Change quoting style from `...' to '...'.Jia Tan3-12/+12
These days the ` and ' do not look symmetric. This quoting style has been changed in various apps over the years including the GNU tools.
2023-09-24lib: Silence -Wsign-conversion in getopt.c.Jia Tan1-3/+3
2023-09-24Build: Update getopt.m4 from Gnulib.Jia Tan1-40/+39
This file was modified from upstream since we do not need to replace getopt() and can avoid complexity and feature tests.
2023-09-26CMake: Add /lib to include path.Jia Tan1-0/+5
2023-09-24CMake: Update libgnu target with new header files.Jia Tan1-0/+5
2023-09-23lib: Update Makefile.am for new header files.Jia Tan1-1/+11
2023-09-24lib: Update getopt1.c from Gnulib.Jia Tan1-34/+22
The only difference was maintaining the conditional inclusion for config.h.
2023-09-23lib: Update getopt.in.h from Gnulib with modifications.Jia Tan1-199/+29
We can still avoid modifying the contents of this file during configuration to simplify the build systems. Gnulib added replacements for inclusions guards for Cygwin. Cygwin should not need getopt_long replacement so this feature can be omitted. <unistd.h> is conditionally included to avoid MSVC since it is not available. The definition for _GL_ARG_NONNULL was also copied into this file from Gnulib since this stage is usually done during gnulib-tool.
2023-09-23lib: Update getopt_int.h from Gnulib.Jia Tan1-61/+48
2023-09-23lib: Update getopt.c from Gnulib with modifications.Jia Tan1-757/+377
The code maintains the prior modifications of conditionally including config.h and disabling NLS support. _GL_UNUSED is repalced with the simple cast to void trick. _GL_UNUSED is only used for these two parameters so its simpler than having to define it.
2023-09-23lib: Add getopt-cdefs.h for getopt_long update.Jia Tan1-0/+70
This was modified slightly from Gnulib. In Gnulib, it expects the @HAVE_SYS_CDEFS_H@ to be replaced. Instead, we can set HAVE_SYS_CDEFS_H on systems that have it and avoid copying another file into the build directory. Since we are not using gnulib-tool, copying extra files requires extra build system updates (and special handling with CMake) so we should avoid when possible.
2023-09-23lib: Copy new header files from Gnulib without modification.Jia Tan4-0/+309
The getopt related files have changed from Gnulib by splitting up getopt.in.h into more modular header files. We could have kept everything in just getopt.in.h, but this will help us continue to update in the future.
2023-09-24Windows: Update the version requirement comments from Win95 to W2k.Lasse Collin2-9/+7
2023-09-24tuklib_physmem: Comment out support for Windows versions older than 2000.Lasse Collin1-11/+9
2023-09-24sysdefs.h: Update the comment about __USE_MINGW_ANSI_STDIO.Lasse Collin1-1/+9
2023-09-22xz: Windows: Don't (de)compress to special files like "con" or "nul".Lasse Collin1-7/+28
Before this commit, the following writes "foo" to the console and deletes the input file: echo foo | xz > con_xz xz --suffix=_xz --decompress con_xz It cannot happen without --suffix because names like con.xz are also special and so attempting to decompress con.xz (or compress con to con.xz) will already fail when opening the input file. Similar thing is possible when compressing. The following writes to "nul" and the input file "n" is deleted. echo foo | xz > n xz --suffix=ul n Now xz checks if the destination is a special file before continuing. DOS/DJGPP version had a check for this but Windows (and OS/2) didn't.
2023-09-22CMake: Wrap two overlong lines that are possible to wrap.Lasse Collin1-2/+4
2023-09-22CMake: Add a comment about threads on Cygwin.Lasse Collin1-0/+1
2023-09-22MSVC: Remove Visual Studio project files and update INSTALL-MSVC.txt.Lasse Collin13-2928/+12
CMake is now the preferred build file generator when building with MSVC.
2023-09-22CMake: Require VS2015 or later for building xzdec.Lasse Collin1-1/+1
xzdec might build with VS2013 but it hasn't been tested. It was never supported before and VS2013 is old anyway so for simplicity only liblzma is supported with VS2013.
2023-09-22CMake: Allow building xz with Visual Studio 2015 and later.Lasse Collin1-1/+1
Building the command line tools xz and xzdec with the combination of CMake + Visual Studio 2015/2017/2019/2022 works now. VS2013 update 2 should still be able to build liblzma. VS2013 cannot build the xz command line tool because xz needs snprintf() that roughly conforms to C99. VS2013 is old and no extra code will be added to support it. Thanks to Kelvin Lee and Jia Tan for testing.
2023-09-22MSVC: #define inline and restrict only when needed.Lasse Collin1-5/+8
This also drops the check for _WIN32 as that shouldn't be needed.
2023-09-22CMake: Add support for replacement getopt_long (lib/getopt*).Lasse Collin1-7/+47
Thanks to Jia Tan for the initial work. I added the libgnu target and made a few related minor edits.
2023-09-22CMake: Bump maximum policy version to 3.27.Lasse Collin1-1/+1
There are several new policies. CMP0149 may affect the Windows SDK version that CMake will choose by default. The new behavior is more predictable, always choosing the latest SDK version by default. The other new policies shouldn't affect this package.
2023-09-22lib/getopt*.c: Include <config.h> only HAVE_CONFIG_H is defined.Lasse Collin2-2/+6
The CMake-based build doesn't use config.h. Up-to-date getopt_long in Gnulib is LGPLv2 so at some point it could be included in XZ Utils too but for now this commit is enough to make CMake-based build possible.
2023-09-22Doxygen: Add more C macro names to PREDEFINED.Lasse Collin1-2/+5
2023-09-22liblzma: Move a few __attribute__ uses in function declarations.Lasse Collin3-7/+10
The API headers have many attributes but these were left as is for now.
2023-09-22xz, xzdec, lzmainfo: Use tuklib_attr_noreturn.Lasse Collin7-25/+37
For compatibility with C23's [[noreturn]], tuklib_attr_noreturn must be at the beginning of declaration (before "extern" or "static", and even before any GNU C's __attribute__). This commit also moves all other function attributes to the beginning of function declarations. "extern" is kept at the beginning of a line so the attributes are listed on separate lines before "extern" or "static".
2023-09-22Remove incorrect uses of __attribute__((__malloc__)).Lasse Collin3-6/+6
xrealloc() is obviously incorrect, modern GCC docs even mention realloc() as an example where this attribute cannot be used. liblzma's lzma_alloc() and lzma_alloc_zero() would be correct uses most of the time but custom allocators may use a memory pool or otherwise hold the pointer so aliasing issues could happen in theory. The xstrdup() case likely was correct but I removed it anyway. Now there are no __malloc__ attributes left in the code. The allocations aren't in hot paths so this should make no practical difference.
2023-09-22Build: Omit -Wc99-c11-compat since it warns about _Noreturn.Lasse Collin1-1/+0
2023-09-22tuklib: Update tuklib_attr_noreturn for C11/C17 and C23.Lasse Collin2-3/+23
This makes no difference for GCC or Clang as they support GNU C's __attribute__((__noreturn__)) but this helps with MSVC: - VS 2019 version 16.7 and later support _Noreturn if the options /std:c11 or /std:c17 are used. This gets handled with the check for __STDC_VERSION__ >= 201112. - When MSVC isn't in C11/C17 mode, __declspec(noreturn) is used. C23 will deprecate _Noreturn (and <stdnoreturn.h>) for [[noreturn]]. This commit anticipates that but the final __STDC_VERSION__ value isn't known yet.
2023-09-22Update THANKS.Lasse Collin1-0/+1
2023-09-22MSVC: xz: Make file_io.c and file_io.h compatible with MSVC.Lasse Collin2-0/+36
Thanks to Kelvin Lee for the original patches and testing the modifications I made.
2023-09-22MSVC: xz: Use GetTickCount64() to implement mytime_now().Lasse Collin1-2/+9
It's available since Windows Vista.
2023-09-22MSVC: xz: Use _stricmp() instead of strcasecmp() in suffix.c.Kelvin Lee1-2/+8
2023-09-22MSVC: xz: Use _isatty() from <io.h> to implement isatty().Kelvin Lee2-0/+10
2023-09-22MSVC: xz: Use _fileno() instead of fileno().Kelvin Lee1-0/+4
2023-09-22MSVC: xzdec: Use _fileno and _setmode.Kelvin Lee1-0/+4
2023-09-22MSVC: Don't #include <unistd.h>.Kelvin Lee4-4/+14
2023-09-19Update THANKS.Lasse Collin1-0/+1
2023-09-14CI: Enable CLMUL in address sanitization test.Jia Tan1-8/+4
The crc64_clmul() function should be ignored by the address sanitizer now so these builds should still pass.
2023-09-14Update THANKS.Lasse Collin1-0/+1
2023-09-14liblzma: Mark crc64_clmul() with __attribute__((__no_sanitize_address__)).Lasse Collin1-0/+8
Thanks to Agostino Sarubbo. Fixes: https://github.com/tukaani-project/xz/issues/62
2023-09-12CMake: Fix time.h checks not running on second CMake run.Jia Tan1-8/+10
If CMake was configured more than once, HAVE_CLOCK_GETTIME and HAVE_CLOCK_MONOTONIC would not be set as compile definitions. The check for librt being needed to provide HAVE_CLOCK_GETTIME was also simplified.
2023-09-12CMake: Fix unconditionally defining HAVE_CLOCK_MONOTONIC.Jia Tan1-5/+3
If HAVE_CLOCK_GETTIME was defined, then HAVE_CLOCK_MONOTONIC was always added as a compile definition even if the check for it failed.
2023-08-31xz: Refactor thousand separator detection and disable it on MSVC.Lasse Collin1-44/+45
Now the two variations of the format strings are created with a macro, and the whole detection code can be easily disabled on platforms where thousand separator formatting is known to not work (MSVC has no support, and on DJGPP 2.05 it can have problems in some cases).
2023-08-31xz: Fix a too relaxed assertion and remove uses of SSIZE_MAX.Lasse Collin2-5/+4
SSIZE_MAX isn't readily available on MSVC. Removing it means that there is one thing less to worry when porting to MSVC.
2023-08-28Tests: Improve invalid unpadded size check in test_lzma_index_append().Jia Tan1-3/+23
This check was extended to test the code added to fix a failing assert in ae5c07b22a6b3766b84f409f1b6b5c100469068a.
2023-08-28Tests: Improve comments in test_index.c.Jia Tan1-3/+3
2023-08-28Update THANKS.Jia Tan1-0/+1
2023-08-28liblzma: Update assert in vli_ceil4().Jia Tan1-1/+1
The argument to vli_ceil4() should always guarantee the return value is also a valid lzma_vli. Thus the highest three valid lzma_vli values are invalid arguments. All uses of the function ensure this so the assert is updated to match this.
2023-08-28liblzma: Add overflow check for Unpadded size in lzma_index_append().Jia Tan1-0/+6
This was not a security bug since there was no path to overflow UINT64_MAX in lzma_index_append() or when it calls index_file_size(). The bug was discovered by a failing assert() in vli_ceil4() when called from index_file_size() when unpadded_sum (the sum of the compressed size of current Stream and the unpadded_size parameter) exceeds LZMA_VLI_MAX. Previously, the unpadded_size parameter was checked to be not greater than UNPADDED_SIZE_MAX, but no check was done once compressed_base was added. This could not have caused an integer overflow in index_file_size() when called by lzma_index_append(). The calculation for file_size breaks down into the sum of: - Compressed base from all previous Streams - 2 * LZMA_STREAM_HEADER_SIZE (size of the current Streams header and footer) - stream_padding (can be set by lzma_index_stream_padding()) - Compressed base from the current Stream - Unpadded size (parameter to lzma_index_append()) The sum of everything except for Unpadded size must be less than LZMA_VLI_MAX. This is guarenteed by overflow checks in the functions that can set these values including lzma_index_stream_padding(), lzma_index_append(), and lzma_index_cat(). The maximum value for Unpadded size is enforced by lzma_index_append() to be less than or equal UNPADDED_SIZE_MAX. Thus, the sum cannot exceed UINT64_MAX since LZMA_VLI_MAX is half of UINT64_MAX. Thanks to Joona Kannisto for reporting this.
2023-08-28Translations: Update the Esperanto translation.Jia Tan1-2/+2
2023-08-26Translations: Update the Esperanto translation.Jia Tan1-18/+29
2023-08-14Docs: Update INSTALL for --enable-threads method win95.Jia Tan1-2/+4
The Autotools build allows win95 threads and --enable-small together now if the compiler supports __attribute__((__constructor__)).
2023-08-14CMake: Conditionally allow win95 threads and --enable-small.Jia Tan1-8/+20
2023-08-09Build: Conditionally allow win95 threads and --enable-small.Jia Tan1-8/+13
When the compiler supports __attribute__((__constructor__)) mythread_once() is never used, even with --enable-small. A configuration with win95 threads and --enable-small will compile and be thread safe so it can be allowed. This isn't a very common configuration since MSVC does not support __attribute__((__constructor__)), but MINGW32 and CLANG32 environments for MSYS2 can use win95 threads and have __attribute__((__constructor__)) support.
2023-08-08mythread.h: Fix typo error in Vista threads mythread_once().Jamaika11-1/+1
The "once_" variable was accidentally referred to as just "once". This prevented building with Vista threads when HAVE_FUNC_ATTRIBUTE_CONSTRUCTOR was not defined.
2023-08-04codespell: Add .codespellrc to set default options.Jia Tan1-0/+24
The .codespellrc allows setting default options to avoid false positive matches, set additional dictionaries, etc. For now, codespell can be used locally before committing doc and comment changes. It should help prevent silly errors and fix up commits in the future.
2023-08-03Tests: Style fixes to test_lzip_decoder.c.Jia Tan1-12/+24
2023-08-03Translations: Update the Chinese (simplified) translation.Jia Tan1-1/+1
2023-08-02xz: Omit an empty paragraph on the man page.Lasse Collin1-1/+0
2023-08-02Add NEWS for 5.4.4.Jia Tan1-0/+43
2023-08-02build-aux/manconv.sh: Fix US-ASCII and UTF-8 output.Lasse Collin1-2/+2
groff defaults to SGR escapes. Using -P-c passes -c to grotty which restores the old behavior. Perhaps there is a better way to get pure plain text output but this works for now.
2023-08-01Update THANKS.Lasse Collin1-0/+1
2023-08-01Update THANKS.Lasse Collin1-0/+1
2023-08-01mythread.h: Disable signal functions in builds targeting Wasm + WASI.ChanTsune1-1/+1
signal.h in WASI SDK doesn't currently provide sigprocmask() or sigset_t. liblzma doesn't need them so this change makes liblzma and xzdec build against WASI SDK. xz doesn't build yet and the tests don't either as tuktest needs setjmp() which isn't (yet?) implemented in WASI SDK. Closes: https://github.com/tukaani-project/xz/pull/57 See also: https://github.com/tukaani-project/xz/pull/56 (The original commit was edited a little by Lasse Collin.)
2023-08-01Add newline to end of .gitignore.Jia Tan1-1/+1
Newline was accidentally removed in commit 01cbb7f023ee7fda8ddde04bd17cf7d3c2418706.
2023-07-31Docs: Fix typos found by codespellDimitri Papadopoulos Orfanos18-29/+29
2023-07-28Update .gitignore.Jia Tan1-0/+4
2023-07-28CMake: Conditionally allow the creation of broken symlinks.Jia Tan1-7/+75
The CMake build will try to create broken symlinks on Unix and Unix-like platforms. Cygwin and MSYS2 are Unix-like, but may not be able to create broken symlinks. The value of the CYGWIN or MSYS environment variables determine if broken symlinks are valid. The default for MSYS2 does not allow for broken symlinks, so the CMake build has been broken for MSYS2 since commit 80a1a8bb838842a2be343bd88ad1462c21c5e2c9.
2023-07-28CI: Fix windows-ci dependency installation.Jia Tan1-3/+3
All of the MSYS2 environments need make, and it does not come with the toolchain package. The toolchain package will install the needed compiler toolchains since without this package CMake cannot properly generate the Makefiles.
2023-07-28CI: Update ci_build.sh CMake to always make Unix Makefiles.Jia Tan1-2/+2
The default for many of the MSYS2 environments is for CMake to create Ninja build files. This would complicate the build script since we would need a different command to run the tests. Its simpler to always use Unix Makefiles so that "make test" is always a usable target for testing.
2023-07-25CI: Test CMake builds and test framework with MSYS2.Jia Tan1-12/+20
2023-07-25CI: Windows CI rename system matrix variable -> msys2_env.Jia Tan1-10/+5
Calling the MSYS2 environment "system" was a bit vague and should be more specific.
2023-07-24CI: Add Clang64 MSYS2 environment to Windows CI.Jia Tan1-0/+1
2023-07-24liblzma: Prevent an empty translation unit in Windows builds.Jia Tan1-1/+5
To workaround Automake lacking Windows resource compiler support, an empty source file is compiled to overwrite the resource files for static library builds. Translation units without an external declaration are not allowed by the C standard and result in a warning when used with -Wempty-translation-unit (Clang) or -pedantic (GCC).
2023-07-22Translations: Update the Vietnamese translation.Jia Tan1-17/+28
2023-07-22CI: Add Windows runner for Autotools builds with MSYS2.Jia Tan1-0/+119
Only a subset of the tests run by the Linux and MacOS Autotools builds are run. The most interesting tests are the ones that disable threads, encoders, and decoders. The Windows runner will only be run manually since these tests will likely take much longer than the Linux and MacOS runners. This runner should be used before merging any large features and before releases. Currently the clang64 environment fails to due to a warning and -Werror is enabled for the CI tests. This is still an early version since the CMake build can be done for MSVC and optionally each of the MSYS2 environments. GitHub does not allow manually running the CI tests unless the workflow is checked on the default branch so checking in a minimum version is a good idea. Thanks to Arthur S for the original proposing the original patch. Closes: https://github.com/tukaani-project/xz/pull/34
2023-07-22CI: Add argument to ci_build.sh to pass flags to autogen.sh.Jia Tan1-2/+7
2023-07-21Tests: Skip .lz files in test_files.sh if not configured.Jia Tan1-1/+1
Previously if the lzip decoder was not configured then test_files.sh would pass the lzip tests instead of skipping them.
2023-07-20Tests: Add ARM64 filter test to test_compress.sh.Jia Tan1-0/+1
2023-07-20Translations: Update the Croatian translation.Jia Tan1-19/+30
2023-07-20Translations: Update the Korean man page translations.Jia Tan1-626/+629
2023-07-20Translations: Update the Korean translation.Jia Tan1-17/+28
2023-07-20Translations: Update the Polish translation.Jia Tan1-18/+29
2023-07-20Translations: Update the German man page translations.Jia Tan1-626/+629
2023-07-20Translations: Update the German translation.Jia Tan1-18/+29
2023-07-20Translations: Update the Chinese (simplified) translation.Jia Tan1-18/+29
2023-07-20Translations: Update the Swedish translation.Jia Tan1-18/+29
2023-07-20Translations: Update the Ukrainian man page translations.Jia Tan1-625/+628
2023-07-20Translations: Update the Ukrainian translation.Jia Tan1-17/+28
2023-07-20Translations: Update the Spanish translation.Jia Tan1-18/+29
2023-07-20Translations: Update the Romanian translation.Jia Tan1-18/+30
2023-07-20Translations: Update the Romanian man page translations.Jia Tan1-625/+629
2023-07-19liblzma: Suppress -Wunused-function warning.Jia Tan1-0/+10
Clang 16.0.0 and earlier have a bug that the ifunc resolver function triggers the -Wunused-function warning. The resolver function is static and only "used" by the __attribute__((__ifunc()__)). At this time, the bug is still unresolved, but has been reported: https://github.com/llvm/llvm-project/issues/63957 This is not a problem in GCC.
2023-07-18liblzma: Reword lzma_str_list_filters() documentation.Jia Tan1-1/+1
This further improves the documentation from commit f36ca7982f6bd5e9827219ed4f3c5a1fbf5d7bdf. The previous wording of "supported options" was slightly misleading since the options that are printed are the ones that are relevant for encoding/decoding. It is not about which options can or must be specified.
2023-07-18liblzma: Improve comment in string_conversion.c.Jia Tan1-2/+2
The comment used "flag" when referring to decoder options. Just referring to them as options is more clear and consistent.
2023-07-18xz: Translate the second "%s: " in message.c since French needs "%s : ".Lasse Collin1-1/+1
This string is used to print a filename when using "xz -v" and stderr isn't a terminal.
2023-07-18xz: Make "%s: %s" translatable because French needs "%s : %s".Lasse Collin4-14/+18
2023-07-18liblzma: Tweak #if condition in memcmplen.h.Lasse Collin1-2/+2
Maybe ICC always #defines _MSC_VER on Windows but now it's very clear which code will get used.
2023-07-18liblzma: Omit unnecessary parenthesis in a preprocessor directive.Lasse Collin1-2/+2
2023-07-18xz: Update Authors list in a few files.Jia Tan5-5/+10
2023-07-18Docs: Add a new section to INSTALL for Tests.Jia Tan1-17/+64
The new Tests section describes basic information about the tests, how to run them, and important details when cross compiling. We have had a few questions about how to compile the tests without running them, so hopefully this information will help others with the same question in the future. Fixes: https://github.com/tukaani-project/xz/issues/54
2023-07-17Docs: Update README.Jia Tan1-0/+4
This adds an entry to "Other implementations of the .xz format" for XZ for Java.
2023-07-17xz: Fix typo in man page.Jia Tan1-1/+1
The Memory limit information section described three output columns when it actually has six. This was reworded to "multiple" to make it more future proof.
2023-07-17xz: Minor clean up for coder.cJia Tan1-32/+21
* Moved max_block_list_size from a global to local variable. * Reworded error message in validate_block_list_filter(). * Removed helper function filter_chain_error(). * Changed 1 << X to 1U << X in many places
2023-07-17xz: Update man page Authors and date.Jia Tan1-2/+3
2023-07-17xz: Add a section to man page for robot mode --filters-help.Jia Tan1-2/+30
2023-07-17xz: Slight reword in xz man page for consistency.Jia Tan1-1/+1
Changed will print => prints in xz --robot --version description to match --robot --info-memory description.
2023-07-17xz: Reorder robot mode subsections in the man page.Jia Tan1-96/+96
The order is now consistent with the order the command line arguments are documented earlier in the man page. The new order is: 1. --list 2. --info-memory 3. --version Instead of the previous order: 1. --version 2. --info-memory 3. --list
2023-07-17xz: Update man page for new --filters-help option.Jia Tan1-0/+10
2023-07-17xz: Add a new --filters-help option.Jia Tan3-0/+43
The --filters-help can be used to help create filter chains with the --filters and --filtersX options. The message in --long-help is too short to fully explain the syntax to construct complex filter chains. In --robot mode, xz will only print the output from liblzma function lzma_str_list_filters.
2023-07-17xz: Update the man page for --block-list and --filtersXJia Tan1-26/+80
The --block-list option description needed updating since the new --filtersX option changes how it can be used. The new entry for --filters1=FILTERS ... --filter9=FILTERS was created right after the --filters option.
2023-07-17xz: Update --long-help for the new --filtersX option.Jia Tan1-2/+10
2023-07-17xz: Ignore filter chains that are set but never used in --block-list.Jia Tan1-18/+48
If a filter chain is set but not used in --block-list, it introduced unexpected behavior such as requiring an unneeded amount of memory to compress, reducing the number of threads in multi-threaded encoding, and printing an incorrect amount of memory needed to decompress. This also renames filters_init_mask => filters_used_mask. A filter is assumed to be used if it is specified in --filtersX until coder_set_compression_settings() determines which filters are referenced in --block-list.
2023-07-17xz: Set the Block size for mt encoding correctly.Jia Tan1-1/+67
When opt_block_size is not used, the Block size for mt encoder is derived from the minimum of the largest Block specified by --block-list and the recommended Block size on all filter chains calculated by lzma_mt_block_size(). This avoids using unnecessary memory and ensures that all Blocks are large enough for the most memory needy filter chain.
2023-07-17xz: Validate --flush-timeout for all specified filter chains.Jia Tan1-8/+16
2023-07-17xz: Allows --block-list filters to scale down memory usage.Jia Tan1-55/+214
Previously, only the default filter chain could have its memory usage adjusted. The filter chains specified with --filtersX were not checked for memory usage. Now, all used filter chains will be adjusted if necessary.
2023-07-17xz: Do not include block splitting if encoders are disabled.Jia Tan1-9/+20
The block splitting logic and split_block() function are not needed if encoders are disabled. This will help slightly reduce the binary size when built without encoders and allow split_block() to use functions that require encoders being enabled.
2023-07-17xz: Free filters[] in debug mode.Jia Tan1-0/+10
This will only free filter chains created with --filters1-9 since the default filter chain may be set from a static function variable. The complexity to free the default filter chain is not worth the burden on code maintenance.
2023-07-17xz: Add a message if --block-list is used outside of xz compresssion.Jia Tan1-0/+11
--block-list is only supported with compression in xz format. This avoids silently ignoring when --block-list is unused.
2023-07-17xz: Create command line options for filters[1-9].Jia Tan3-60/+230
The new command line options are meant to be combined with --block-list. They work as an optional extension to --block-list to specify a custom filter chain for each block listed. The new options allow the creation of up to 9 reusable filter chains. For instance: xz --block-list=1:10MiB,3:5MiB,,2:5MiB,1:0 --filters1=delta--lzma2 \ --filters2=x86--lzma2 --filters3=arm64--lzma2 Will create the following blocks: 1. A block of size 10 MiB with filter chain delta, lzma2. 2. A block of size 5 MiB with filter chain arm64, lzma2. 3. A block of size 5 MiB with filter chain arm64, lzma2. 4. A block of size 5 MiB with filter chain x86, lzma2. 5. A block containing the rest of the file contents with filter chain delta, lzma2.
2023-07-17xz: Use lzma_filters_free() in forget_filter_chain().Jia Tan1-8/+10
This is a little cleaner than the previous implementation of forget_filter_chain(). It is also more consistent since lzma_str_to_filters() will always terminate the filter chain so there is no need to terminate it later in coder_set_compression_settings().
2023-07-17xz: Separate string to filter conversion into a helper function.Jia Tan1-13/+20
Converting from string to filter will also need to be done for block specific filter chains.
2023-07-17Tests: Use new --filters option in test_compress.shJia Tan1-10/+10
2023-07-17xz: Update --long-help and man page for new --filters option.Jia Tan2-5/+42
2023-07-17xz: Add --filters option to CLI.Jia Tan3-4/+58
The --filters option uses the new lzma_str_to_filters() function to convert a string into a full filter chain. Using this option will reset all previous filters set by --preset, --[filter], or --filters.
2023-07-14Tests: Improve feature testing for skipping.Jia Tan2-3/+3
Fixed a bug where test_compress_* would all fail if arm64 or armthumb filters were enabled for compression but arm was disabled. Since the grep tests only checked for "define HAVE_ENCODER_ARM", this would match on HAVE_ENCODER_ARM64 or HAVE_ENCODER_ARMTHUMB. Now the config.h feature test requires " 1" at the end to prevent the prefix problem. have_feature() was also updated for this even though there were known current bugs affecting it. This is just in case future features have a similar prefix problem.
2023-07-10Translations: Update the Chinese (traditional) translation.Jia Tan1-282/+377
2023-07-08liblzma: Remove non-portable empty initializer.Jia Tan1-1/+1
Commit 78704f36e74205857c898a351c757719a6c8b666 added an empty initializer {} to prevent a warning. The empty initializer is a GNU extension and results in a build failure on MSVC. The -wpedantic flag warns about empty initializers.
2023-07-08Translations: Update the Vietnamese translation.Jia Tan1-271/+349
2023-06-29Tests: Fix memory leaks in test_index.Jia Tan1-0/+11
Several tests were missing calls to lzma_index_end() to clean up the lzma_index structs. The memory leaks were discovered by using -fsanitize=address with GCC.
2023-06-29Tests: Fix memory leaks in test_block_header.Jia Tan1-16/+22
test_block_header was not properly freeing the filter options between calls to lzma_block_header_decode(). The memory leaks were discovered by using -fsanitize=address with GCC.
2023-06-29liblzma: Prevent uninitialzed warning in mt stream encoder.Jia Tan1-1/+1
This change only impacts the compiler warning since it was impossible for the wait_abs struct in stream_encode_mt() to be used before it was initialized since mythread_condtime_set() will always be called before mythread_cond_timedwait(). Since the mythread.h code is different between the POSIX and Windows versions, this warning was only present on Windows builds. Thanks to Arthur S for reporting the warning and providing an initial patch.
2023-06-28liblzma: Prevent warning for MSYS2 Windows build.Jia Tan1-2/+4
In lzma_memcmplen(), the <intrin.h> header file is only included if _MSC_VER and _M_X64 are both defined but _BitScanForward64() was previously used if _M_X64 was defined. GCC for MSYS2 defines _M_X64 but not _MSC_VER so _BitScanForward64() was used without including <intrin.h>. Now, lzma_memcmplen() will use __builtin_ctzll() for MSYS2 GCC builds as expected.
2023-06-28CI: Add test with -fsanitize=address,undefined.Jia Tan2-5/+26
ci_build.sh was updated to accept disabling of __attribute__ ifunc and CLMUL. This will allow -fsanitize=address to pass because ifunc is incompatible with -fsanitize=address. The CLMUL implementation has optimizations that potentially read past the buffer and mask out the unwanted bytes. This test will only run on Autotools Linux.
2023-06-28CI: Upgrade checkout action from v2 to v3.Jia Tan1-1/+1
2023-06-27Update THANKS.Jia Tan1-0/+1
2023-06-27Docs: Document the configure option --disable-ifunc in INSTALL.Jia Tan1-0/+8
2023-06-27Minor tweaks to style and comments.Lasse Collin2-8/+9
2023-06-27CMake: Rename CHECK_ATTR_IFUNC to ALLOW_ATTR_IFUNC.Lasse Collin1-3/+3
It's so that there's a clear difference in wording compared to liblzma's integrity check types.
2023-06-27liblzma: Add ifunc implementation to crc64_fast.c.Lasse Collin1-9/+26
The ifunc method avoids indirection via the function pointer crc64_func. This works on GNU/Linux and probably on FreeBSD too. The previous __attribute((__constructor__)) method is kept for compatibility with ELF platforms which do support ifunc. The ifunc method has some limitations, for example, building liblzma with -fsanitize=address will result in segfaults. The configure option --disable-ifunc must be used for such builds. Thanks to Hans Jansen for the original patch. Closes: https://github.com/tukaani-project/xz/pull/53
2023-06-27Add ifunc check to CMakeLists.txtHans Jansen1-0/+19
CMake build system will now verify if __attribute__((__ifunc__())) can be used in the build system. If so, HAVE_FUNC_ATTRIBUTE_IFUNC will be defined to 1.
2023-06-27Add ifunc check to configure.acHans Jansen1-0/+28
configure.ac will now verify if __attribute__((__ifunc__())) can be used in the build system. If so, HAVE_FUNC_ATTRIBUTE_IFUNC will be defined to 1.
2023-06-07CI: Add apt update command before installing dependencies.Jia Tan1-2/+6
Without the extra command, all of the CI tests were automatically failing because the Ubuntu servers could not be reached properly.
2023-06-07Update THANKS.Jia Tan1-0/+1
2023-06-06CMake: Protects against double find_packageBenjamin Buch1-7/+9
Boost iostream uses `find_package` in quiet mode and then again uses `find_package` with required. This second call triggers a `add_library cannot create imported target "ZLIB::ZLIB" because another target with the same name already exists.` This can simply be fixed by skipping the alias part on secondary `find_package` runs.
2023-05-31Translations: Update the Esperanto translation.Jia Tan1-93/+92
2023-05-31Translations: Update the Croatian translation.Jia Tan1-1/+1
2023-05-31Translations: Update the Chinese (simplified) translation.Jia Tan1-160/+157
2023-05-17Translations: Update German translation of man pages.Jia Tan1-40/+12
2023-05-17Translations: Update the German translation.Jia Tan1-95/+94
2023-05-17Translations: Update the Croatian translation.Jia Tan1-94/+93
2023-05-17Translations: Update Korean translation of man pages.Jia Tan1-2446/+567
2023-05-17Translations: Update the Korean translation.Jia Tan1-161/+158
2023-05-16Translations: Update the Spanish translation.Jia Tan1-161/+158
2023-05-16Translations: Update the Romanian translation.Jia Tan1-97/+98
2023-05-16Translations: Update Romanian translation of man pages.Jia Tan1-9/+10
2023-05-16Translations: Update Ukrainian translation of man pages.Jia Tan1-6/+6
2023-05-16Translations: Update the Ukrainian translation.Jia Tan1-162/+159
2023-05-16Translations: Update the Polish translation.Jia Tan1-161/+155
2023-05-16Translations: Update the Swedish translation.Jia Tan1-161/+158
2023-05-16Translations: Update the Esperanto translation.Jia Tan1-17/+17
2023-05-13liblzma: Slightly rewords lzma_str_list_filters() documentation.Jia Tan1-1/+1
Reword "options required" to "supported options". The previous may have suggested that the options listed were all required anytime a filter is used for encoding or decoding. The reword makes this more clear that adjusting the options is optional.
2023-05-12liblzma: Adds lzma_nothrow to MicroLZMA API functions.Jia Tan1-2/+3
None of the liblzma functions may throw an exception, so this attribute should be applied to all liblzma API functions.
2023-05-11liblzma: Exports lzma_mt_block_size() as an API function.Jia Tan7-22/+61
The lzma_mt_block_size() was previously just an internal function for the multithreaded .xz encoder. It is used to provide a recommended Block size for a given filter chain. This function is helpful to determine the maximum Block size for the multithreaded .xz encoder when one wants to change the filters between blocks. Then, this determined Block size can be provided to lzma_stream_encoder_mt() in the lzma_mt options parameter when intializing the coder. This requires one to know all the filter chains they are using before starting to encode (or at least the filter chain that will need the largest Block size), but that isn't a bad limitation.
2023-05-11liblzma: Creates IS_ENC_DICT_SIZE_VALID() macro.Jia Tan2-3/+9
This creates an internal liblzma macro to test if the dictionary size is valid for encoding.
2023-05-04Add NEWS for 5.4.3.Jia Tan1-0/+10
2023-05-04Add NEWS for 5.2.12.Jia Tan1-0/+14
2023-05-04Translations: Update the Croatian translation.Jia Tan1-3/+3
2023-05-04tuklib_integer.h: Reverts previous commit.Jia Tan1-2/+2
Previous commit 6be460dde07113fe3f08f814b61ddc3264125a96 would cause an error if the integer size was 32 bit.
2023-05-04tuklib_integer.h: Changes two other UINT_MAX == UINT32_MAX to >=.Jia Tan1-2/+2
2023-05-03tuklib_integer.h: Fix a recent copypaste error in Clang detection.Lasse Collin1-2/+2
Wrong line was changed in 7062348bf35c1e4cbfee00ad9fffb4a21aa6eff7. Also, this has >= instead of == since ints larger than 32 bits would work too even if not relevant in practice.
2023-04-25CI: Adds a build and test for small configuration.Jia Tan1-0/+5
2023-04-25CI: ci_build.sh allows configuring small build.Jia Tan1-1/+6
2023-04-20Update THANKS.Jia Tan1-0/+1
2023-04-19Windows: Include <intrin.h> when needed.Jia Tan2-0/+16
Legacy Windows did not need to #include <intrin.h> to use the MSVC intrinsics. Newer versions likely just issue a warning, but the MSVC documentation says to include the header file for the intrinsics we use. GCC and Clang can "pretend" to be MSVC on Windows, so extra checks are needed in tuklib_integer.h to only include <intrin.h> when it will is actually needed.
2023-04-19tuklib_integer: Use __builtin_clz() with Clang.Jia Tan1-3/+3
Clang has support for __builtin_clz(), but previously Clang would fallback to either the MSVC intrinsic or the regular C code. This was discovered due to a bug where a new version of Clang required the <intrin.h> header file in order to use the MSVC intrinsics. Thanks to Anton Kochkov for notifying us about the bug.
2023-04-14liblzma: Update project maintainers in lzma.h.Lasse Collin1-1/+1
AUTHORS was updated earlier, lzma.h was simply forgotten.
2023-04-13liblzma: Cleans up old commented out code.Jia Tan1-11/+0
2023-04-07Docs: Add missing word to SECURITY.md.Jia Tan1-1/+1
2023-04-07Update THANKS.Jia Tan1-0/+1
2023-04-07Docs: Minor edits to SECURITY.md.Jia Tan1-5/+20
2023-04-07Docs: Create SECURITY.mdGabriela Gutierrez1-0/+14
Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>
2023-03-29CI: Tests for disabling threading on CMake builds.Jia Tan2-5/+2
2023-03-29CI: Removes CMakeCache.txt between builds.Jia Tan1-0/+2
If the cache file is not removed, CMake will not reset configurations back to their default values. In order to make the tests independent, it is simplest to purge the cache. Unfortunatly, this will slow down the tests a little and repeat some checks.
2023-03-29CMake: Update liblzma-config.cmake generation.Jia Tan1-11/+22
Now that the threading is configurable, the liblzma CMake package only needs the threading library when using POSIX threads.
2023-03-29CMake: Allows setting thread method.Jia Tan1-40/+104
The thread method is now configurable for the CMake build. It matches the Autotools build by allowing ON (pick the best threading method), OFF (no threading), posix, win95, and vista. If both Windows and posix threading are both available, then ON will choose Windows threading. Windows threading will also not use: target_link_libraries(liblzma Threads::Threads) since on systems like MinGW-w64 it would link the posix threads without purpose.
2023-03-24CI: Runs CMake feature tests.Jia Tan1-114/+55
Now, CMake will run similar feature disable tests that the Autotools version did before. In order to do this without repeating lines in ci.yml, it now makes sense to use the GitHub Workflow matrix to create a loop.
2023-03-24CI: ci_build.sh allows CMake features to be configured.Jia Tan1-90/+143
Also included various clean ups for style and helper functions for repeated work.
2023-03-24CI: Change ci_build.sh to use bash instead of sh.Jia Tan1-1/+1
This script is only meant to be run as part of the CI build/test process on machines that are known to have bash (Ubuntu and MacOS). If this assumption changes in the future, then the bash specific commands will need to be replaced with a more portable option. For now, it is convenient to use bash commands.
2023-03-24CMake: Only build xzdec if decoders are enabled.Jia Tan1-1/+1
2023-03-23Build: Removes redundant check for LZMA1 filter support.Jia Tan1-4/+1
2023-03-23CMake: Bump maximum policy version to 3.26.Lasse Collin1-1/+1
It adds only one new policy related to FOLDERS which we don't use. This makes it clear that the code is compatible with the policies up to 3.26.
2023-03-23CMake: Conditionally build xz list.* files if decoders are enabled.Jia Tan1-2/+7
2023-03-23CMake: Allow configuring features as cache variables.Jia Tan1-137/+391
This allows users to change the features they build either in CMakeCache.txt or by using a CMake GUI. The sources built for liblzma are affected by this too, so only the necessary files will be compiled.
2023-03-21Build: Add a comment that AC_PROG_CC_C99 is needed for Autoconf 2.69.Lasse Collin1-0/+3
It's obsolete in Autoconf >= 2.70 and just an alias for AC_PROG_CC but Autoconf 2.69 requires AC_PROG_CC_C99 to get a C99 compiler.
2023-03-21Build: configure.ac: Use AS_IF and AS_CASE where required.Lasse Collin1-15/+15
This makes no functional difference in the generated configure (at least with the Autotools versions I have installed) but this change might prevent future bugs like the one that was just fixed in the commit 5a5bd7f871818029d5ccbe189f087f591258c294.
2023-03-21Update THANKS.Lasse Collin1-0/+1
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-14 20:29:06 +0000