| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
|
|
DJGPP 2.05 added support for thousands separators but it's
broken at least under WinXP with Finnish locale that uses
a non-breaking space as the thousands separator. Workaround
by disabling thousands separators for DJGPP builds.
|
|
It doesn't need -fgnu89-inline like 2.04beta did.
|
|
|
|
The added defines assume GCC >= 4.8.
|
|
I made a few white space changes to these without getting them
approved by the translation teams. (I tried to contact the hu and
zh_TW teams but didn't succeed. I didn't contact the zh_CN team.)
|
|
The translated strings haven't been updated but word wrapping
is different.
|
|
The German translation isn't identical to the file in
the Translation Project but the changes (white space changes
only) were approved by the translator Mario Blättermann.
|
|
|
|
This version matches CMake files in the master branch (commit
265daa873c0d871f5f23f9b56e133a6f20045a0a) except that this omits
two source files that aren't in v5.2 and in the beginning of
CMakeLists.txt the first paragraph in the comment is slightly
different to point out possible issues in building shared liblzma.
|
|
Normally, if po4a isn't available, autogen.sh will return
with non-zero exit status. The option --no-po4a can be useful
when one knows that po4a isn't available but wants autogen.sh
to still return with zero exit status.
|
|
|
|
This file only generates fastpos_table.c.
It isn't built as a part of liblzma.
|
|
This should silence the equivalent of -Wundef in compilers that
don't define __GNUC__.
|
|
This gives a tiny encoder speed improvement. This could have been done
in 2014 after the commit 544aaa3d13554e8640f9caf7db717a96360ec0f6 but
it was forgotten.
|
|
Appears that this file used to get included as a side effect of
gettext. After the change to gettext version requirements this file
no longer got copied to the package and so the build was broken.
|
|
It can be true at least on z/OS.
|
|
|
|
The __builtin byteswapping is the preferred one so check for it first.
|
|
strerror() needs <string.h> which happened to be included via
tuklib_common.h -> tuklib_config.h -> sysdefs.h if HAVE_CONFIG_H
was defined. This wasn't tested without config.h before so it
had worked fine.
|
|
string.h is used unconditionally elsewhere in the project and
configure has always stopped if limits.h is missing, so these
headers must have been always available even on the weirdest
systems.
|
|
There is no specific reason for this other than blocking
the most ancient versions. These are still old:
Autoconf 2.69 (2012)
Automake 1.12 (2012)
gettext 0.19.6 (2015)
Libtool 2.4 (2010)
|
|
This bumps the version requirement from 0.19 (from 2014) to
0.19.6 (2015).
Using only the old AM_GNU_GETTEXT_VERSION results in old
gettext infrastructure being placed in the package. By using
both macros we get the latest gettext files while the other
programs in the Autotools family can still see the old macro.
|
|
Thanks to Mario Blättermann.
|
|
The dependency on po4a is optional. It's never required to install
the translated man pages when xz is built from a release tarball.
If po4a is missing when building from xz.git, the translated man
pages won't be generated but otherwise the build will work normally.
The translations are only updated automatically by autogen.sh and
by "make mydist". This makes it easy to keep po4a as an optional
dependency and ensures that I won't forget to put updated
translations to a release tarball.
The translated man pages aren't installed if --disable-nls is used.
The installation of translated man pages abuses Automake internals
by calling "install-man" with redefined dist_man_MANS and man_MANS.
This makes the hairy script code slightly less hairy. If it breaks
some day, this code needs to be fixed; don't blame Automake developers.
Also, this adds more quotes to the existing shell script code in
the Makefile.am "-hook"s.
|
|
|
|
|
|
|
|
|
|
|
|
I don't know if the problem is in gnulib's gl_POSIX_SHELL macro
or if xzgrep does something that isn't in POSIX. The workaround
adds a special case for Solaris: if /usr/xpg4/bin/sh exists and
gl_cv_posix_shell wasn't overriden on the configure command line,
use that shell for xzgrep and other scripts. That shell is known
to work and exists on most Solaris systems.
|
|
|
|
See the code comment for reasoning. It's far from perfect but
hopefully good enough for certain cases while hopefully doing
nothing bad in other situations.
At presets -5 ... -9, 4020 MiB vs. 4096 MiB makes no difference
on how xz scales down the number of threads.
The limit has to be a few MiB below 4096 MiB because otherwise
things like "xz --lzma2=dict=500MiB" won't scale down the dict
size enough and xz cannot allocate enough memory. With
"ulimit -v $((4096 * 1024))" on x86-64, the limit in xz had
to be no more than 4085 MiB. Some safety margin is good though.
This is hack but it should be useful when running 32-bit xz on
a 64-bit kernel that gives full 4 GiB address space to xz.
Hopefully this is enough to solve this:
https://bugzilla.redhat.com/show_bug.cgi?id=1196786
FreeBSD has a patch that limits the result in tuklib_physmem()
to SIZE_MAX on 32-bit systems. While I think it's not the way
to do it, the results on --memlimit-compress have been good. This
commit should achieve practically identical results for compression
while leaving decompression and tuklib_physmem() and thus
lzma_physmem() unaffected.
|
|
xz --flush-timeout=2000, old version:
1. xz is started. The next flush will happen after two seconds.
2. No input for one second.
3. A burst of a few kilobytes of input.
4. No input for one second.
5. Two seconds have passed and flushing starts.
The first second counted towards the flush-timeout even though
there was no pending data. This can cause flushing to occur more
often than needed.
xz --flush-timeout=2000, after this commit:
1. xz is started.
2. No input for one second.
3. A burst of a few kilobytes of input. The next flush will
happen after two seconds counted from the time when the
first bytes of the burst were read.
4. No input for one second.
5. No input for another second.
6. Two seconds have passed and flushing starts.
|
|
|
|
The same code sequence repeats so it's nicer as a separate function.
Note that in one case there was no test for opt_mode != MODE_TEST,
but that was only because that condition would always be true, so
this commit doesn't change the behavior there.
|
|
When input blocked, xz --flush-timeout=1 would wake up every
millisecond and initiate flushing which would have nothing to
flush and thus would just waste CPU time. The fix disables the
timeout when no input has been seen since the previous flush.
|
|
|
|
|
|
|
|
It has been enabled in FreeBSD for a while and reported to work fine.
Thanks to Xin Li.
|
|
|
|
Using the aligned methods requires more care to ensure that
the address really is aligned, so it's nicer if the aligned
methods are prefixed. The next commit will remove the unaligned_
prefix from the unaligned methods which in liblzma are used in
more places than the aligned ones.
|
|
Add a configure option --enable-unsafe-type-punning to get the
old non-conforming memory access methods. It can be useful with
old compilers or in some other less typical situations but
shouldn't normally be used.
Omit the packed struct trick for unaligned access. While it's
best in some cases, this is simpler. If the memcpy trick doesn't
work, one can request unsafe type punning from configure.
Because CRC32/CRC64 code needs fast aligned reads, if no very
safe way to do it is found, type punning is used as a fallback.
This sucks but since it currently works in practice, it seems to
be the least bad option. It's never needed with GCC >= 4.7 or
Clang >= 3.6 since these support __builtin_assume_aligned and
thus fast aligned access can be done with the memcpy trick.
Other things:
- Support GCC/Clang __builtin_bswapXX
- Cleaner bswap fallback macros
- Minor cleanups
|
|
Thanks to Daniel Richard G.
|
|
This adds a configure option --enable-path-for-scripts=PREFIX
which defaults to empty except on Solaris it is /usr/xpg4/bin
to make POSIX grep and others available. The Solaris case had
been documented in INSTALL with a manual fix but it's better
to do this automatically since it is needed on most Solaris
systems anyway.
Thanks to Daniel Richard G.
|
|
|
|
It didn't matter in XZ Utils because sysdefs.h
includes string.h anyway.
|
|
|
|
|
|
Also changed 999 to 99 so it fits even if lzma_check happened
to be 8 bits wide.
|
|
|
|
|
|
|
|
Caught by clang -Wused-but-marked-unused.
|
|
|
|
Or any off_t which isn't very big (like signed 64 bit integer
that most system have). A small off_t could overflow if the
file being decompressed had long enough run of zero bytes,
which would result in corrupt output.
|
|
The duplication was introduced about eleven years ago and
should have been cleaned up back then already.
This was caught by -Wduplicated-branches.
|
|
|
|
|
|
|
|
|
|
Also, more parentheses were added to the literal_subcoder
macro in lzma_comon.h (better style but no functional change
in the current usage).
|
|
|
|
Use a temporary variable instead of e.g.
conv32le(unaligned_read32ne(buf)) because the macro can
evaluate its argument multiple times.
|
|
|
|
Thanks to Bruce Stark.
|
|
|
|
|
|
The same compiler-specific #ifdefs are already in tuklib_integer.h
|
|
|
|
Now gcc -fsanitize=undefined should be clean.
Thanks to Jeffrey Walton.
|
|
Now memcpy() or GNU C packed structs for unaligned access instead
of type punning. See the comment in this commit for details.
Avoiding type punning with unaligned access is needed to
silence gcc -fsanitize=undefined.
New functions: unaliged_readXXne and unaligned_writeXXne where
XX is 16, 32, or 64.
|
|
I should have always known this but I didn't. Here is an example
as a reminder to myself:
int mycopy(void *dest, void *src, size_t n)
{
memcpy(dest, src, n);
return dest == NULL;
}
In the example, a compiler may assume that dest != NULL because
passing NULL to memcpy() would be undefined behavior. Testing
with GCC 8.2.1, mycopy(NULL, NULL, 0) returns 1 with -O0 and -O1.
With -O2 the return value is 0 because the compiler infers that
dest cannot be NULL because it was already used with memcpy()
and thus the test for NULL gets optimized out.
In liblzma, if a null-pointer was passed to memcpy(), there were
no checks for NULL *after* the memcpy() call, so I cautiously
suspect that it shouldn't have caused bad behavior in practice,
but it's hard to be sure, and the problematic cases had to be
fixed anyway.
Thanks to Jeffrey Walton.
|
|
|
|
|
|
|
|
XZ Utils is now part of the Translation Project
<https://translationproject.org/>.
|
|
|
|
"xz -dcfv not_an_xz_file" crashed (all four options are
required to trigger it). It caused xz to call
lzma_get_progress(&strm, ...) when no coder was initialized
in strm. In this situation strm.internal is NULL which leads
to a crash in lzma_get_progress().
The bug was introduced when xz started using lzma_get_progress()
to get progress info for multi-threaded compression, so the
bug is present in versions 5.1.3alpha and higher.
Thanks to Filip Palian <Filip.Palian@pjwstk.edu.pl> for
the bug report.
|
|
|
|
|
|
In this particular case I don't see this affecting readability
of the code.
Thanks to Pavel Raiskup.
|
|
This should help static analysis tools to see that newg
isn't leaked.
Thanks to Pavel Raiskup.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
I understood that if a WTPV is specified, it's often wrong
because different VS installations have different SDK version
installed. Omitting the WTPV tag makes VS2017 default to
Windows SDK 8.1 which often is also missing, so in any case
people may need to specify the WTPV before building. But some
day in the future a missing WTPV tag will start to default to
the latest installed SDK which sounds reasonable:
https://developercommunity.visualstudio.com/content/problem/140294/windowstargetplatformversion-makes-it-impossible-t.html
Thanks to "dom".
|
|
|
|
|
|
|
|
|
|
In C++11, the `throw()` specifier is deprecated and `noexcept` is
preffered instead.
|
|
In most cases it was harmless but it could affect some
custom build systems.
Thanks to Pippijn van Steenhoven.
|
|
|
|
Thanks to Melanie Blower (Intel) for the patch.
|
|
|
|
Some paths use slashes instead of backslashes as directory
separators... now it should work (I tested VS2013 version).
|
|
These files match the v5.2 branch (no file info decoder).
|
|
|
|
|
|
|
|
It ended up printing an uninitialized char-array when trying to
print the check names (column 7) on the "totals" line.
This also changes the column 12 (minimum xz version) to
50000002 (xz 5.0.0) instead of 0 when there are no valid
input files.
Thanks to kidmin for the bug report.
|
|
It makes ChangeLog significantly smaller.
|
|
|
|
|
|
xz --list is random access so POSIX_FADV_SEQUENTIAL was clearly
wrong.
|
|
The 0 got treated specially in a buggy way and as a result
the function did nothing. The API doc said that 0 was supposed
to return LZMA_PROG_ERROR but it didn't.
Now 0 is treated as if 1 had been specified. This is done because
0 is already used to indicate an error from lzma_memlimit_get()
and lzma_memusage().
In addition, lzma_memlimit_set() no longer checks that the new
limit is at least LZMA_MEMUSAGE_BASE. It's counter-productive
for the Index decoder and was actually needed only by the
auto decoder. Auto decoder has now been modified to check for
LZMA_MEMUSAGE_BASE.
|
|
|
|
It returned LZMA_PROG_ERROR, which was done to avoid zero as
the limit (because it's a special value elsewhere), but using
LZMA_PROG_ERROR is simply inconvenient and can cause bugs.
The fix/workaround is to treat 0 as if it were 1 byte. It's
effectively the same thing. The only weird consequence is
that then lzma_memlimit_get() will return 1 even when 0 was
specified as the limit.
This fixes a very rare corner case in xz --list where a specific
memory usage limit and a multi-stream file could print the
error message "Internal error (bug)" instead of saying that
the memory usage limit is too low.
|
|
|
|
|
|
|
|
|
|
Only one definition was visible in a translation unit.
It avoided a few casts and temp variables but seems that
this hack doesn't work with link-time optimizations in compilers
as it's not C99/C11 compliant.
Fixes:
http://www.mail-archive.com/xz-devel@tukaani.org/msg00279.html
|
|
|
|
In the v5.2 branch this feature is considered experimental
and thus disabled by default.
The sandboxing is used conditionally as described in main.c.
This isn't optimal but it was much easier to implement than
a full sandboxing solution and it still covers the most common
use cases where xz is writing to standard output. This should
have practically no effect on performance even with small files
as fork() isn't needed.
C and locale libraries can open files as needed. This has been
fine in the past, but it's a problem with things like Capsicum.
io_sandbox_enter() tries to ensure that various locale-related
files have been loaded before cap_enter() is called, but it's
possible that there are other similar problems which haven't
been seen yet.
Currently Capsicum is available on FreeBSD 10 and later
and there is a port to Linux too.
Thanks to Loganaden Velvindron for help.
|
|
AU_ALIAS was removed because the new version is incompatible
with the old version.
It no longer checks for <sys/capability.h> separately.
It's enough to test for it as part of AC_CHECK_DECL.
The defines HAVE_CAPSICUM_SYS_CAPSICUM_H and
HAVE_CAPSICUM_SYS_CAPABILITY_H were removed as unneeded.
HAVE_SYS_CAPSICUM_H from AC_CHECK_HEADERS is enough.
It no longer does a useless search for the Capsicum library
if the header wasn't found.
Fixed a bug in ACTION-IF-FOUND (the first argument). Specifying
the argument omitted the default action but the given action
wasn't used instead.
AC_DEFINE([HAVE_CAPSICUM]) is now always called when Capsicum
support is found. Previously it was part of the default
ACTION-IF-FOUND which a custom action would override. Now
the default action only prepends ${CAPSICUM_LIB} to LIBS.
The documentation was updated.
Since there as no serial number, "#serial 2" was added.
|
|
The file was loaded from this web page:
https://github.com/google/capsicum-test/blob/dev/autoconf/m4/ax_check_capsicum.m4
Thanks to Loganaden Velvindron for pointing it out for me.
|
|
lzma_index_dup() calls index_dup_stream() which, in case of
an error, calls index_stream_end() to free memory allocated
by index_stream_init(). However, it illogically didn't
actually free the memory. To make it logical, the tree
handling code was modified a bit in addition to changing
index_stream_end().
Thanks to Evan Nemerson for the bug report.
|
|
|
|
It's available in glibc (GNU/Linux, GNU/kFreeBSD). It's better
than sysconf(_SC_NPROCESSORS_ONLN) because sched_getaffinity()
gives the number of cores available to the process instead of
the total number of cores online.
As a side effect, this commit fixes a bug on GNU/kFreeBSD where
configure would detect the FreeBSD-specific cpuset_getaffinity()
but it wouldn't actually work because on GNU/kFreeBSD it requires
using -lfreebsd-glue when linking. Now the glibc-specific function
will be used instead.
Thanks to Sebastian Andrzej Siewior for the original patch
and testing.
|
|
xz used to call utime() on Windows, but its result gets lost
on close(). Using _futime() seems to work.
Thanks to Martok for reporting the bug:
http://www.mail-archive.com/xz-devel@tukaani.org/msg00261.html
|
|
Thanks to Evan Nemerson.
|
|
Thanks to Christian Kujau.
|
|
It silences a few warnings and most people probably have
0.19 even on stable distributions.
Thanks to Christian Kujau.
|
|
This is the sane thing to do. The conflict with OpenSSL
on some OSes and especially that the OS-provided versions
can be significantly slower makes it clear that it was
a mistake to have the external SHA-256 support enabled by
default.
Those who want it can now pass --enable-external-sha256 to
configure. INSTALL was updated with notes about OSes where
this can be a bad idea.
The SHA-256 detection code in configure.ac had some bugs that
could lead to a build failure in some situations. These were
fixed, although it doesn't matter that much now that the
external SHA-256 is disabled by default.
MINIX >= 3.2.0 uses NetBSD's libc and thus has SHA256_Init
in libc instead of libutil. Support for the libutil version
was removed.
|
|
|
|
On FreeBSD 10 and older, SHA256_Init from libmd conflicts
with libcrypto from OpenSSL. The OpenSSL version has
different sizeof(SHA256_CTX) and it can cause weird
problems if wrong SHA256_Init gets used.
Looking at the source, MINIX 3 seems to have a similar issue but
I'm not sure. To be safe, I disabled SHA256_Init on MINIX 3 too.
NetBSD has SHA256_Init in libc and they had a similar problem,
but they already fixed it in 2009.
Thanks to Jim Wilcoxson for the bug report that helped
in finding the problem.
|
|
|
|
|
|
When optimizing, GCC can reorder code so that an uninitialized
value gets used in a comparison, which makes Valgrind unhappy.
It doesn't happen when compiled with -O0, which I tend to use
when running Valgrind.
Thanks to Rich Prohaska. I remember this being mentioned long
ago by someone else but nothing was done back then.
|
|
It would be too annoying to update other build systems
just because of this.
|
|
They all need decoder support and if that isn't available,
there's no point trying to build them.
|
|
|
|
|
|
|
|
The patch is quite long but it's mostly about adding new #ifdefs
to omit code when encoders or decoders have been disabled.
This adds two new #defines to config.h: HAVE_ENCODERS and
HAVE_DECODERS.
|
|
People shouldn't rely on the presets when decoding raw streams,
but xz uses the presets as the starting point for raw decoder
options anyway.
lzma_encocder_presets.c was renamed to lzma_presets.c to
make it clear it's not used solely by the encoder code.
|
|
Now it gives an error if LZMA1 encoder/decoder is missing
when LZMA2 encoder/decoder was requested. Even better would
be LZMA2 implicitly enabling LZMA1 but it would need more code.
|
|
Previously it was omitted if encoders were disabled
with --disable-encoders. It didn't make sense and
it also broke the build.
|
|
If an appropriate header and structure were found by configure,
but a library with a usable SHA-256 functions wasn't, the build
failed.
|
|
unlink() can return EBUSY in errno for open files on some
operating systems and file systems.
|
|
|
|
|
|
Stream Flags and Stream Padding weren't copied from
empty Streams.
|
|
|
|
|
|
|
|
As pointed out by Robert Pollak, there's a typo in the German
translation of the compression preset option (-0 ... -9) help text.
"The compressor" translates to "der Komprimierer", and the genitive
form is "des Komprimierers". The old word makes no sense at all.
|
|
Provide an update of the German translation.
* A lot of compound words were previously written with spaces, while
German orthography is relatively clear in that the components
should not be separated.
* When referring to the actual process of (de)compression rather than the
concept, replace “(De-)Kompression” with “(De-)Komprimierung”.
Previously, both forms were used in this context and are now used in a
manner consistent with “Komprimierung” being more likely to refer to
a process.
* Consistently translate “standard input”/“output”
* Use “Zeichen” instead of false friend “Charakter” for “character”
* Insert commas around relative clauses (as required in German)
* Some other minor corrections
* Capitalize “ß” as “ẞ”
* Consistently start option descriptions in --help with capital letters
Acked-By: Andre Noll <maan@tuebingen.mpg.de>
* Update after msgmerge
|
|
Some tests used "cygwin*" and some used "cygwin". I changed
them all to use "cygwin". Shouldn't affect anything in practice.
|
|
|
|
src/liblzma/common/common.h uses it to set __declspec(dllexport)
for the API symbols.
Thanks to Adam Walling.
|
|
|
|
|
|
|
|
|
|
Thanks to Adam Walling for creating these files.
|
|
Thanks to Cristian Rodríguez.
|
|
Thanks to Andy Hochhaus.
|
|
|
|
|
|
This reverts commit 7a11c4a8e5e15f13d5fa59233b3172e65428efdd.
It is a problem when libc has pipe2() but the kernel is too
old to have pipe2() and thus pipe2() fails. In xz it's pointless
to have a fallback for non-functioning pipe2(); it's better to
avoid pipe2() completely.
Thanks to Michael Fox for the bug report.
|
|
|
|
The earlier version compiled but didn't actually work
since sysconf(_SC_PHYS_PAGES) always fails (or so I was told).
Thanks to Ole André Vadla Ravnås for the patch and testing.
|
|
It tried to use sysctl() on QNX but
- it broke the build because sysctl() needs -lsocket on QNX;
- sysctl() doesn't work for detecting the core count on QNX
even if it compiled.
sysconf() works. An alternative would have been to use
QNX-specific SYSPAGE_ENTRY(num_cpu) from <sys/syspage.h>.
Thanks to Ole André Vadla Ravnås.
|
|
|
|
|
|
|
|
Actually the value of arg_count cannot exceed INT_MAX
but it's nicer as an unsigned int.
|
|
|
|
|
|
|
|
|
|
|
|
The bug was added in the commit
f48fce093b07aeda95c18850f5e086d9f2383380 and thus
affected 5.1.4beta and 5.2.0. Luckily the bug cannot
cause data corruption or other nasty things.
|
|
Now it reads the old flags instead of blindly setting O_NONBLOCK.
The old code may have worked correctly, but this is better.
|
|
|
|
In FreeBSD, cpuset_getaffinity() is the preferred way to get
the number of available cores.
Thanks to Rui Paulo for the patch. I edited it slightly, but
hopefully I didn't break anything.
|
|
Thanks to Rui Paulo for the fix.
|
|
I heard that Visual Studio 2013 gave warnings without the casts.
Thanks to Gabi Davar.
|
|
|
|
|
|
Thanks to Torsten Rupp for reporting this. I had
forgotten to run Valgrind before the 5.2.0 release.
|
|
|
|
|
|
This is similar to the case with stdin.
Thanks to Brad Smith for the bug report and testing
on OpenBSD.
|
|
|
|
It's a problem at least on OpenBSD which doesn't support
O_NONBLOCK on e.g. /dev/null. I'm not surprised if it's
a problem on other OSes too since this behavior is allowed
in POSIX-1.2008.
The code relying on this behavior was committed in June 2013
and included in 5.1.3alpha released on 2013-10-26. Clearly
the development releases only get limited testing.
|
|
Hiding them makes no sense since normally there's no error
when testing the "good" files. With "bad" files errors are
expected and then it makes sense to keep the messages hidden.
|
|
Mention the possible "make check" failure on Solaris in the
Solaris-specific section of INSTALL. It was already in
section 4.5 but it is better mention it in the OS-specific
section too.
|
|
|
|
|
|
|
|
|
|
|
