| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
|
|
We need this for when we're passing sanitizer flags or -gdwarf-4 for Clang
with Valgrind. Just always start with -O2 if CFLAGS isn't set in the
environment and append what was passed on the command line.
|
|
This is a lot easier to work with than the save-logs thing the action
tries to do...
|
|
Unfortunately, UBSAN doesn't do this by default. See also the change
I made in Meson for this in October [0].
[0] https://github.com/mesonbuild/meson/commit/7b7d2e060b447de9c2642848847370a58711ac1c
|
|
Using `--trace-children=yes` has a trade-off here, as it makes
`test_scripts.sh` pretty slow when calling various non-xz utilities.
But I also feel like it's not useless to have Valgrind used there and it's
not easy to exclude Valgrind just for that one test...
I did consider using AX_VALGRIND_CHECK [0][1] but I couldn't get it working
immediately with some conditionally-built tests and I wondered if it was
worth spending time on at least while we're debating xz's future build
system situation.
[0] https://www.gnu.org/software/autoconf-archive/ax_valgrind_check.html
[1] https://tecnocode.co.uk/2014/12/23/automatically-valgrinding-code-with-ax_valgrind_check/
|
|
A macro is useful to prevent a single #if directive from
getting too ugly but only one macro is needed for all archs.
|
|
The macro name had an odd typo so the table wasn't omitted
when it should have.
Fixes: 1940f0ec28f08c0ac72c1413d9706fb82eabe6ad
|
|
This can speed up configure a tiny bit.
Fixes: c5f6d79cc9515a7f22d7ea4860c6cc394b295732
|
|
I didn't test this but it shouldn't change any functionality.
Fixes: 761f5b69a4c778c8bcb09279b845b07c28790575
|
|
Also add parenthesis to the return statement.
I didn't test this.
Fixes: 761f5b69a4c778c8bcb09279b845b07c28790575
|
|
Subtracting from 0 is negation, this just keeps warnings away.
Fixes: 761f5b69a4c778c8bcb09279b845b07c28790575
|
|
|
|
|
|
|
|
This is *NOT* done for security reasons even though the backdoor
relied on the ifunc code. Instead, the reason is that in this
project ifunc provides little benefits but it's quite a bit of
extra code to support it. The only case where ifunc *might* matter
for performance is if the CRC functions are used directly by an
application. In normal compression use it's completely irrelevant.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The other maintainer suddenly disappeared.
|
|
This only reverts the XZ URL changes.
|
|
The XZ projects were moved back to their original URLs.
|
|
|
|
|
|
|
|
While the backdoor was inactive (and thus harmless) without inserting
a small trigger code into the build system when the source package was
created, it's good to remove this anyway:
- The executable payloads were embedded as binary blobs in
the test files. This was a blatant violation of the
Debian Free Software Guidelines.
- On machines that see lots bots poking at the SSH port, the backdoor
noticeably increased CPU load, resulting in degraded user experience
and thus overwhelmingly negative user feedback.
- The maintainer who added the backdoor has disappeared.
- Backdoors are bad for security.
This reverts the following without making any other changes:
6e636819 Tests: Update two test files.
a3a29bbd Tests: Test --single-stream can decompress bad-3-corrupt_lzma2.xz.
0b4ccc91 Tests: Update RISC-V test files.
8c9b8b20 liblzma: Fix typos in crc32_fast.c and crc64_fast.c.
82ecc538 liblzma: Fix false Valgrind error report with GCC.
cf44e4b7 Tests: Add a few test files.
3060e107 Tests: Use smaller dictionary size in RISC-V test files.
e2870db5 Tests: Add two RISC-V Filter test files.
The RISC-V test files also have real content that tests the filter
but the real content would fit into much smaller files. A generator
program would need to be available as well.
Thanks to Andres Freund for finding and reporting it and making
it public quickly so others could act without a delay.
See: https://www.openwall.com/lists/oss-security/2024/03/29/4
|
|
It never enabled it.
|
|
|
|
|
|
|
|
This does the previous commit with CMake.
AC_EGREP_CPP uses AC_REQUIRE so the outermost if-commands must
be changed to AS_IF to ensure that things wont break some day.
See 5a5bd7f871818029d5ccbe189f087f591258c294.
|
|
It doesn't support the __symver__ attribute or __asm__(".symver ...").
The generic symbol versioning can still be used since it only needs
linker support.
|
|
|
|
|
|
NVHPC compiler has several issues that make it impossible to
build liblzma:
- the compiler cannot handle unions that contain pointers that
are not the first members;
- the compiler cannot handle the assembler code in range_decoder.h
(LZMA_RANGE_DECODER_CONFIG has to be set to zero);
- the compiler fails to produce valid code for delta_decode if the
vectorization is enabled, which results in failed tests.
This introduces NVHPC-specific workarounds that address the issues.
|
|
This better matches what configure.ac does. For example, musl has
only basic symbol versioning support:
https://wiki.musl-libc.org/functional-differences-from-glibc.html#Symbol_versioning
configure.ac tries to enable symbol versioning only with glibc
so now CMake does the same.
|
|
|
|
The AC_MSG_ERROR line is overlong anyway as are a few other
AC_MSG_ERROR lines already.
|
|
There are cases when the users want to decide themselves whether
they want to have the generic (even on GNU/Linux) or the linux
(even if we do not recommend that) symbol versioning variant.
The former might be needed to circumvent compiler issues (i.e.
the compiler does not support all features that are required
for the linux versioning), the latter might help in overriding
the assumptions made in the configure script.
|
|
|
|
|
|
The original files were generated with random local to my machine.
To better reproduce these files in the future, a constant seed was used
to recreate these files.
|
|
The first stream in this file is valid, so this tests that xz properly
stops after decompressing it.
|
|
This increases code coverage and tests for possible shifting bugs.
|
|
|
|
This reverts commit adaacafde6661496ca2814b1e94a3ba5186428cb.
|
|
With GCC and a certain combination of flags, Valgrind will falsely
trigger an invalid write. This appears to be due to the omission of
instructions to properly save, set up, and restore the frame pointer.
The IFUNC resolver is a leaf function since it only calls a function
that is inlined. So sometimes GCC omits the frame pointer instructions
in the resolver unless this optimization is explictly disabled.
This fixes https://bugzilla.redhat.com/show_bug.cgi?id=2267598.
|
|
|
|
Thanks to Sam James for determining this was the attribute needed to
workaround the GCC bug and for his version of the patch in Gentoo.
|
|
Using __attribute__((__no_profile_instrument_function__)) on the ifunc
resolver works around a bug in GCC -fprofile-generate:
it adds profiling code even to ifunc resolvers which can make
the ifunc resolver crash at program startup. This attribute
was not introduced until GCC 7 and Clang 13, so ifunc won't
be used with prior versions of these compilers.
This bug was brought to our attention by:
https://bugs.gentoo.org/925415
And was reported to upstream GCC by:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=11411
|
|
|
|
|
|
It's only done with CMake >= 3.20 and if library support
for translation was already found.
Sort of fixes: https://github.com/tukaani-project/xz/issues/82
|
|
|
|
Now that multi threaded encoding is the default, users do not need to
see a warning message everytime the number of threads is reduced. On
some machines, this could happen very often. It is not unreasonable for
users to need to set double verbose mode to see this kind of
information.
To see these warning messages -vv or --verbose --verbose must be passed
to set xz into the highest possible verbosity mode.
These warnings had caused automated testing frameworks to fail when they
expected no output to stderr.
Thanks to Sebastian Andrzej Siewior for reporting this and for the
initial version of the patch.
|
|
|
|
|
|
Signed-off-by: Chien Wong <m@xv97.com>
|
|
The previous Linux Landlock feature test assumed that having the
linux/landlock.h header file was enough. The new feature tests also
requires that prctl() and the required Landlock system calls are
supported.
|
|
|
|
|
|
|
|
Like 5.5.0alpha, 5.7.0alpha won't be released, it's just to mark that
the branch is not stable.
Once again there is no API/ABI stability for new features in devel
versions. The major soname won't be bumped even if API/ABI of new
features breaks between devel releases.
|
|
|
|
The French and Brazilian Portuguese man page translations have not been
updated since the switch from public domain to 0BSD. The old GPLv2
strings have now been removed from these files.
|
|
This will be fixed in the next round of translations, but this avoids
having a fuzzy match or not fixing the English version.
|
|
Thanks to Yuri Chornoivan.
|
|
|
|
|
|
|
|
|
|
|
|
user_abort_pipe[] was still being used instead of the parameters.
|
|
Old versions of Clang reported the unsupported function attribute and
__crc32d() function as warnings instead of errors, so the feature test
passed when it shouldn't have, causing a compile error at build time.
-Werror was added to this feature test to fix this. The change is not
needed for CMake because check_c_source_compiles() also performs
linking and the error is caught then.
Thanks to Sebastian Andrzej Siewior for reporting this.
|
|
LOCALEDIR may contain spaces like in "C:\Program Files".
|
|
If xz is given a directory, it should look like this:
$ xz /usr/bin
xz: /usr/bin: Is a directory, skipping
The Landlock rules didn't allow opening directories for reading:
$ xz /usr/bin
xz: /usr/bin: Permission denied
The simplest fix was to allow opening directories for reading.
While it's a bit silly to allow it solely for the error message,
it shouldn't make the sandbox significantly weaker.
The single-file use case (like when called from GNU tar) is
still as strict as possible: all Landlock restrictions are
enabled before (de)compression starts.
|
|
Thanks to Sebastian Andrzej Siewior and Sam James for
benchmarking on various systems.
|
|
|
|
|
|
|
|
|
|
It's for binary packages built with windows/build.bash.
|
|
Support for the old MinGW was dropped. Only MinGW-w64 with GCC
is supported now.
The script now supports also cross-compilation from GNU/Linux
(tests are not run). MSYS2 and also the old MSYS 1.0.11 work
for building on Windows. The i686 and x86_64 toolchains must
be in PATH to build both 32-bit and 64-bit versions.
Parallel builds are done if "nproc" from GNU coreutils is available.
MinGW-w64 runtime copyright information file was renamed from
COPYING-Windows.txt to COPYING.MinGW-w64-runtime.txt which
is the filename used by MinGW-w64 itself. Its existence
is now mandatory, it's checked at the beginning of the script.
The file TODO is no longer copied to the package.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The original code was good enough for supporting GNU/Linux
and a few others but it wasn't very portable.
CMake doesn't support Solaris Studio's -xldscope=hidden.
If it ever does, things should still work with this commit
as Solaris Studio supports not only its own __global but also
the GNU C __attribute__((visibility("default"))). Support for the
attribute was added in 2007 to Sun Studio 12 compiler version 5.9.
|
|
|
|
|
|
-O3 doesn't seem useful for speed but it makes the code bigger.
CMake makes is difficult for users to simply override the
optimization level: CFLAGS / CMAKE_C_FLAGS aren't helpful because
they go before CMAKE_C_FLAGS_RELEASE. Of course, users can override
CMAKE_C_FLAGS_RELEASE directly but then they have to remember to
add also -DNDEBUG to disable assertions.
This commit changes -O3 to -O2 in CMAKE_C_FLAGS_RELEASE if and only if
CMAKE_C_FLAGS_RELEASE cache variable doesn't already exist. So if
a custom value is passed on the command line (or reconfiguring an
already-configured build), the cache variable won't be modified.
|
|
This is to match configure.ac.
|
|
|
|
|
|
CMP0154 doesn't affect us since we don't use FILE_SET.
|
|
|
|
|
|
In contrast to Automake, skipping of this test when decoders
are disabled is handled at CMake side instead of test_scripts.sh
because CMake-build doesn't create config.h.
|
|
Compared to the Autotools-based build, this has simpler handling
for the shell (@POSIX_SHELL@) and extra PATH entry for the scripts
(configure has --enable-path-for-scripts=PREFIX). The simpler
metho should be enough for non-ancient systems and Solaris.
|
|
PACKAGE_VERSION was already used in liblzma.pc.in.
This way only one version @foo@ is used.
|
|
It helps that cmake_install.cmake doesn't parallelize installation
so symlinks can be created so that the target is always known to
exist (a requirement on Windows in some cases).
This bumps the minimum CMake version from 3.13 to 3.14 to use
file(CREATE_LINK ...). It could be made to work on 3.13 by
calling "cmake -E create_symlink" but it's uglier code and
slower in "make install". 3.14 should be a reasonable version
to require nowadays, especially since the Autotools build
is still the primary build system for most OSes.
|
|
If gettext tools are available, the .po files listed in po/LINGUAS
are converted using msgfmt. This allows building with translations
directly from xz.git without Autotools.
If gettext tools aren't available, the Autotools-created .gmo files
in the "po" directory will be used. This allows CMake-based build
to use translations from Autotools-generated tarball.
If translation support is found (Intl_FOUND) but both the
gettext tools and the pre-generated .gmo files are missing,
then "make" will fail.
|
|
|
|
|
|
This makes these sandboxing methods stricter when no files are
created or deleted. That is, it's a middle ground between the
initial sandbox and the strictest single-file-to-stdout sandbox:
this allows opening files for reading but output has to go to stdout.
|
|
Linux 6.7 added support for ABI version 4 which restricts
TCP connections which xz won't need and thus those can be
forbidden now. Since the ABI version is handled at runtime,
supporting version 4 won't cause any compatibility issues.
Note that new enough kernel headers are required to get
version 4 support enabled at build time.
|
|
Landlock is now always used just like pledge(2) is: first in more
permissive mode and later (under certain common conditions) in
a strict mode that doesn't allow opening more files.
I put pledge(2) first in sandbox.c because it's the simplest API
to use and still somewhat fine-grained for basic applications.
So it's the simplest thing to understand for anyone reading sandbox.c.
|
|
|
|
Also explicitly initialize progress_automatic to make it clear
that it can be read before message_init() sets it. Static variable
was initialized to false by default already so this is only for
clarity.
|
|
Dirs first, then files in case-sensitive ASCII order.
|
|
|
|
|
|
|
|
All other translated man pages were being installed but
lzmainfo had been forgotten.
|
|
GCC docs promise that it works and a few other compilers do
too. Clang/LLVM is documented source code only but unsurprisingly
it behaves the same as others on x86-64 at least. But the
certainly-portable way is good enough here so use that.
|
|
|
|
The x32 port has a x86-64 ABI in term of all registers but uses only
32bit pointer like x86-32. The assembly optimisation fails to compile on
x32. Given the state of x32 I suggest to exclude it from the
optimisation rather than trying to fix it.
Signed-off-by: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Don't assume that Clang defines __GNUC__ as the extensions
are available in clang-cl as well (and possibly in some other
Clang variants?).
|
|
|
|
|
|
|
|
Adding the SPDX license identifier changed the line numbers.
|
|
|
|
|
|
|
|
|
|
|
|
It was added in 2017 in c2e29f06a7d1e3ba242ac2fafc69f5d6e92f62cd
but it never got into any release tarballs because it was
forgotten to be added to Makefile.am.
|
|
|
|
|
|
This kind of fixes the problem reported here:
https://bugs.launchpad.net/ubuntu/+source/xz-utils/+bug/1291020
|
|
|
|
|
|
|
|
|
|
It's compatible with GCC and Clang.
|
|
It's used only for basic bittrees and fixed-size reverse bittree
because those showed a clear benefit on x86-64 with GCC and Clang.
The other methods were more mixed and thus are commented out but
they should be tested on other archs.
|
|
|
|
But now it needs one more local variable.
|
|
|
|
|
|
|
|
Now extra buffer space is reserved so that repeating bytes for
any single match will never need to copy from two places (both
the beginning and the end of the buffer). This simplifies
dict_repeat() and helps a little with speed.
This seems to reduce .lzma decompression time about 2 %, so
with .xz and CRC it could be slightly less. The small things
add up still.
|
|
It's not completely obvious if this is better in the decoder.
It should be good if compiler can avoid creating a branch
(like using CMOV on x86).
This also makes lzma_encoder.c use the new macros.
|
|
This adds macros for bittree decoding which prepares the code
for alternative C versions and inline assembly.
|
|
The new decoder resumes the first decoder loop in the Resumable mode.
Then, the code executes in Non-resumable mode until it detects that it
cannot guarantee to have enough input/output to decode another symbol.
The Resumable mode is how the decoder has always worked. Before decoding
every input bit, it checks if there is enough space and will save its
location to be resumed later. When the decoder has more input/output,
it jumps back to the correct sequence in the Resumable mode code.
When the input/output buffers are large, the Resumable mode is much
slower than the Non-resumable because it has more branches and is harder
for the compiler to optimize since it is in a large switch block.
Early benchmarking shows significant time improvement (8-10% on gcc and
clang x86) by using the Non-resumable code as much as possible.
|
|
The new "safe" range decoder mode is the same as old range decoder, but
now the default behavior of the range decoder will not check if there is
enough input or output to complete the operation. When the buffers are
close to fully consumed, the "safe" operations must be used instead. This
will improve speed because it will reduce the number of branches needed
for most of the range decoder operations.
|
|
The footer template from Doxygen has the closing </body> </html>
as Doxygen doesn't add them otherwise.
target="_blank" was omitted as it's not useful here but
it can be slightly annoying as one cannot just go back
in the browser history.
Since the footer links to the license file in the same
directory and not to CC website, the rel attributes
can be omitted.
|
|
|
|
It looked odd to only have the original English authors listed
in the header comments of the translated files.
|
|
lzmainfo has had translation support since 2009 at least but
it was never added to po/POTFILES.in so the messages weren't
translated. It's a very rarely needed tool so it's not too bad.
This also adds src/xz/mytime.c to po/POTFILES.in although there
are no translatable strings. It's simpler this way so that it
won't be forgotten if strings were ever added to that file.
|
|
The same is used for both po/xz.pot and po4a/xz-man.pot.
|
|
All man pages are under 0BSD now so this is simple now.
|
|
|
|
The main reason is a kind of silly one:
xz-man.pot contains strings from all man pages in XZ Utils.
The man pages of xzdiff, xzgrep, and xzmore were under GPLv2
and the rest under 0BSD. Thus xz-man.pot contained strings
under two licences. po4a creates the translated man pages
from the combined 0BSD+GPLv2 xz-man.pot.
I haven't liked this mixing in xz-man.pot but the
Translation Project requires that all man pages must be
in the same .pot file. So a separate xz-man-gpl.pot
wasn't an option.
Since these man pages are short, rewriting them was quick enough.
Now xz-man.pot is entirely under 0BSD and marking the per-file
licenses is simpler.
As a bonus, some wording hopefully is now slightly better
although it's perhaps a matter of taste.
NOTE: In xzgrep.1, the EXIT STATUS section was written by me
in the commit d796b6d7fdb8b7238b277056cf9146cce25db604 so that's
why that section could be taken as is from the old xzgrep.1.
|
|
The xz tool can decompress three file formats and xzless
has always supported uncompressed files too.
|
|
|
|
Also add SPDX license identifier now that there is a known license.
|
|
Perhaps the generated files aren't even copyrightable but
using the same license for them as for the rest of the liblzma
keeps things more consistent for tools that look for license info.
|
|
It is built and run only manually so this didn't matter
unless one wanted to regenerate the price_table.c.
|
|
|
|
|
|
|
|
Translations and doc/xz-file-format.txt and doc/lzma-file-format.txt
were not touched.
COPYING.0BSD was added.
|
|
The initial commit 5d018dc03549c1ee4958364712fb0c94e1bf2741
in 2007 had a comment in sha256.c that the code is based on
Crypto++ Library 5.5.1. In 2009 the Authors list in sha256.c
and the AUTHORS file was updated with information that the
code had come from Crypto++ but via 7-Zip. I know I had viewed
7-Zip's SHA-256 code but back then the C code has been identical
enough with Crypto++, so I don't why I thought the author info
would need that extra step via 7-Zip for this single file.
Another error is that I had mixed sha.* and shacal2.* files
when checking for author info in Crypto++. The shacal2.* files
aren't related to liblzma's sha256.c and thus Kevin Springle's
code in Crypto++ isn't either.
|
|
It was last updated in 2013.
|
|
It was good to keep these around in parallel with the newer examples
but I think it's OK to remove the old ones at this point.
|
|
|
|
If any other BCJ filter was enabled for encoding or decoding, then this
was not a problem.
|
|
|
|
|
|
|
|
|
|
This makes "less" show a warning if a decompression error occurred.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
