aboutsummaryrefslogtreecommitdiff
path: root/src/xz (follow)
AgeCommit message (Collapse)AuthorFilesLines
2022-11-08xz: Initialize the pledge(2) sandbox at the very beginning of main().Lasse Collin1-13/+14
It feels better that the initializations are sandboxed too. They don't do anything that the pledge() call wouldn't allow.
2022-11-07xz: Extend --robot --info-memory output.Lasse Collin2-15/+56
Now it includes everything that the human-readable --info-memory shows.
2022-11-07xz: Avoid a compiler warning in progress_speed() in message.c.Jia Tan1-6/+3
This should be smaller too since it avoids the string constants.
2022-10-25xz: Fix --single-stream with an empty .xz Stream.Lasse Collin1-0/+9
Example: $ xz -dc --single-stream good-0-empty.xz xz: good-0-empty.xz: Internal error (bug) The code, that is tries to catch some input file issues early, didn't anticipate LZMA_STREAM_END which is possible in that code only when --single-stream is used.
2022-10-25xz: Add support for OpenBSD's pledge() sandbox.Lasse Collin3-1/+25
2022-10-25xz: Fix decompressor behavior if input uses an unsupported check type.Lasse Collin1-4/+15
Now files with unsupported check will make xz display a warning, set the exit status to 2 (unless --no-warn is used), and then decompress the file normally. This is how it was supposed to work since the beginning but this was broken by the commit 231c3c7098f1099a56abb8afece76fc9b8699f05, that is, a little before 5.0.0 was released. The buggy behavior displayed a message, set exit status 1 (error), and xz didn't attempt to to decompress the file. This doesn't matter today except for special builds that disable CRC64 or SHA-256 at build time (but such builds should be used in special situations only). The bug matters if new check type is added in the future and an old xz version is used to decompress such a file; however, it's likely that such files would use a new filter too and an old xz wouldn't be able to decompress the file anyway. The first hunk in the commit is the actual fix. The second hunk is a cleanup since LZMA_TELL_ANY_CHECK isn't used in xz. There is a test file for unsupported check type but it wasn't used by test_files.sh, perhaps due to different behavior between xz and the simpler xzdec.
2022-10-25xz: Clarify the man page: input file isn't removed if an error occurs.Lasse Collin1-2/+3
2022-10-25xz: Refactor to remove is_empty_filename().Lasse Collin3-17/+3
Long ago it was used in list.c too but nowadays it's needed only in io_open_src() so it's nicer to avoid a separate function.
2022-10-25xz: If input file cannot be removed, treat it as a warning, not error.Lasse Collin1-2/+2
Treating it as a warning (message + exit status 2) matches gzip and it seems more logical as at that point the output file has already been successfully closed. When it's a warning it is possible to suppress it with --no-warn.
2022-09-19xz: Add --experimental-arm64[=width=WIDTH].Lasse Collin4-0/+60
It will be renamed to --arm64 once it is stable. Man page or --long-help weren't updated yet.
2022-08-22xz: Try to clarify --memlimit-mt-decompress vs. --memlimit-compress.Lasse Collin1-12/+19
2022-08-19xz: Revise --info-memory output.Lasse Collin2-6/+27
The strings could be more descriptive but it's good to have some version of this committed now. --robot mode wasn't changed yet.
2022-08-19xz: Update the man page for threaded decompression and memlimits.Lasse Collin1-27/+121
This documents the changes made in commits 6c6da57ae2aa962aabde6892442227063d87e88c, cad299008cf73ec566f0662a9cf2b94f86a99659, and 898faa97287a756231c663a3ed5165672b417207. The --info-memory bit hasn't been finished yet even though it's already mentioned in this commit under --memlimit-mt-decompress and --threads.
2022-07-24xz: Update the man page that change to --keep will be in 5.2.6.Lasse Collin1-2/+2
2022-07-12xz: Document the special memlimit case of 2000 MiB on MIPS32.Lasse Collin1-2/+6
See commit fc3d3a7296ef58bb799a73943636b8bfd95339f7.
2022-04-14xz: Fix build with --disable-threads.Lasse Collin1-0/+4
2022-04-14xz: Change the cap of the default -T0 memlimit for 32-bit xz.Lasse Collin1-1/+3
The SIZE_MAX / 3 was 1365 MiB. 1400 MiB gives little more room and it looks like a round (artificial) number in --info-memory once --info-memory is made to display it. Also, using #if avoids useless code on 64-bit builds.
2022-04-14xz: Add a default soft memory usage limit for --threads=0.Lasse Collin3-11/+82
This is a soft limit in sense that it only affects the number of threads. It never makes xz fail and it never makes xz change settings that would affect the compressed output. The idea is to make -T0 have more reasonable behavior when the system has very many cores or when a memory-hungry compression options are used. This also helps with 32-bit xz, preventing it from running out of address space. The downside of this commit is that now the number of threads might become too low compared to what the user expected. I hope this to be an acceptable compromise as the old behavior has been a source of well-argued complaints for a long time.
2022-04-14xz: Make -T0 use multithreaded mode on single-core systems.Lasse Collin3-9/+27
The main problem withi the old behavior is that the compressed output is different on single-core systems vs. multicore systems. This commit fixes it by making -T0 one thread in multithreaded mode on single-core systems. The downside of this is that it uses more memory. However, if --memlimit-compress is used, xz can (thanks to the previous commit) drop to the single-threaded mode still.
2022-04-14xz: Changes to --memlimit-compress and --no-adjust.Lasse Collin1-20/+43
In single-threaded mode, --memlimit-compress can make xz scale down the LZMA2 dictionary size to meet the memory usage limit. This obviously affects the compressed output. However, if xz was in threaded mode, --memlimit-compress could make xz reduce the number of threads but it wouldn't make xz switch from multithreaded mode to single-threaded mode or scale down the LZMA2 dictionary size. This seemed illogical and there was even a "FIXME?" about it. Now --memlimit-compress can make xz switch to single-threaded mode if one thread in multithreaded mode uses too much memory. If memory usage is still too high, then the LZMA2 dictionary size can be scaled down too. The option --no-adjust was also changed so that it no longer prevents xz from scaling down the number of threads as that doesn't affect compressed output (only performance). After this commit --no-adjust only prevents adjustments that affect compressed output, that is, with --no-adjust xz won't switch from multithreaded mode to single-threaded mode and won't scale down the LZMA2 dictionary size. The man page wasn't updated yet.
2022-04-12xz: Add --memlimit-mt-decompress along with a default limit value.Lasse Collin5-42/+97
--memlimit-mt-decompress allows specifying the limit for multithreaded decompression. This matches memlimit_threading in liblzma. This limit can only affect the number of threads being used; it will never prevent xz from decompressing a file. The old --memlimit-decompress option is still used at the same time. If the value of --memlimit-decompress (the default value or one specified by the user) is less than the value of --memlimit-mt-decompress , then --memlimit-mt-decompress is reduced to match --memlimit-decompress. Man page wasn't updated yet.
2022-03-07xz: Add initial support for threaded decompression.Lasse Collin1-1/+35
If threading support is enabled at build time, this will use lzma_stream_decoder_mt() even for single-threaded mode. With memlimit_threading=0 the behavior should be identical. This needs some work like adding --memlimit-threading=LIMIT. The original patch from Sebastian Andrzej Siewior included a method to get currently available RAM on Linux. It might be one way to go but as it is Linux-only, the available-RAM approach needs work for portability or using a fallback method on other OSes. The man page wasn't updated yet.
2021-10-27xz: Change the coding style of the previous commit.Lasse Collin1-5/+6
It isn't any better now but it's consistent with the rest of the code base.
2021-10-27xz: Avoid fchown(2) failure.Alexander Bluhm1-1/+7
OpenBSD does not allow to change the group of a file if the user does not belong to this group. In contrast to Linux, OpenBSD also fails if the new group is the same as the old one. Do not call fchown(2) in this case, it would change nothing anyway. This fixes an issue with Perl Alien::Build module. https://github.com/PerlAlien/Alien-Build/issues/62
2021-04-11Reduce maximum possible memory limit on MIPS32Ivan A. Melnikov1-0/+6
Due to architectural limitations, address space available to a single userspace process on MIPS32 is limited to 2 GiB, not 4, even on systems that have more physical RAM -- e.g. 64-bit systems with 32-bit userspace, or systems that use XPA (an extension similar to x86's PAE). So, for MIPS32, we have to impose stronger memory limits. I've chosen 2000MiB to give the process some headroom.
2021-01-11xz: Make --keep accept symlinks, hardlinks, and setuid/setgid/sticky.Lasse Collin2-5/+20
Previously this required using --force but that has other effects too which might be undesirable. Changing the behavior of --keep has a small risk of breaking existing scripts but since this is a fairly special corner case I expect the likehood of breakage to be low enough. I think the new behavior is more logical. The only reason for the old behavior was to be consistent with gzip and bzip2. Thanks to Vincent Lefevre and Sebastian Andrzej Siewior.
2020-11-01xz: Avoid unneeded \f escapes on the man page.Lasse Collin1-9/+22
I don't want to use \c in macro arguments but groff_man(7) suggests that \f has better portability. \f would be needed for the .TP strings for portability reasons anyway. Thanks to Bjarni Ingi Gislason.
2020-11-01xz: Use non-breaking spaces when intentionally using more than one space.Lasse Collin1-1/+1
This silences some style checker warnings. Seems that spaces in the beginning of a line don't need this treatment. Thanks to Bjarni Ingi Gislason.
2020-11-01xz: Protect the ellipsis (...) on the man page with \&.Lasse Collin1-2/+2
This does it only when ... appears outside macro calls. Thanks to Bjarni Ingi Gislason.
2020-11-01xz: Avoid the abbreviation "e.g." on the man page.Lasse Collin1-33/+33
A few are simply omitted, most are converted to "for example" and surrounded with commas. Sounds like that this is better style, for example, man-pages(7) recommends avoiding such abbreviations except in parenthesis. Thanks to Bjarni Ingi Gislason.
2020-07-12xz man page: Change \- (minus) to \(en (en-dash) for a numeric range.Lasse Collin1-8/+8
Docs of ancient troff/nroff mention \(em (em-dash) but not \(en and \- was used for both minus and en-dash. I don't know how portable \(en is nowadays but it can be changed back if someone complains. At least GNU groff and OpenBSD's mandoc support it. Thanks to Bjarni Ingi Gislason for the patch.
2020-04-06src/xz/xz.1: Correct misused two-fonts macrosBjarni Ingi Gislason1-5/+5
Output is from: test-groff -b -e -mandoc -T utf8 -rF0 -t -w w -z [ "test-groff" is a developmental version of "groff" ] Input file is ./src/xz/xz.1 <src/xz/xz.1>:408 (macro BR): only 1 argument, but more are expected <src/xz/xz.1>:1009 (macro BR): only 1 argument, but more are expected <src/xz/xz.1>:1743 (macro BR): only 1 argument, but more are expected <src/xz/xz.1>:1920 (macro BR): only 1 argument, but more are expected <src/xz/xz.1>:2213 (macro BR): only 1 argument, but more are expected Output from nroff and troff is unchanged, except for a font change of a full stop (.). Signed-off-by: Bjarni Ingi Gislason <bjarniig@rhi.hi.is>
2020-03-23Typo fixes from fossies.org.Lasse Collin1-2/+2
https://fossies.org/linux/misc/xz-5.2.5.tar.xz/codespell.html
2020-03-11xz: Never use thousand separators in DJGPP builds.Lasse Collin1-2/+12
DJGPP 2.05 added support for thousands separators but it's broken at least under WinXP with Finnish locale that uses a non-breaking space as the thousands separator. Workaround by disabling thousands separators for DJGPP builds.
2020-02-21xz: Silence a warning when sig_atomic_t is long int.Lasse Collin1-1/+1
It can be true at least on z/OS.
2020-02-21xz: Avoid unneeded access of a volatile variable.Lasse Collin1-1/+1
2020-02-07Build: Add support for translated man pages using po4a.Lasse Collin1-14/+36
The dependency on po4a is optional. It's never required to install the translated man pages when xz is built from a release tarball. If po4a is missing when building from xz.git, the translated man pages won't be generated but otherwise the build will work normally. The translations are only updated automatically by autogen.sh and by "make mydist". This makes it easy to keep po4a as an optional dependency and ensures that I won't forget to put updated translations to a release tarball. The translated man pages aren't installed if --disable-nls is used. The installation of translated man pages abuses Automake internals by calling "install-man" with redefined dist_man_MANS and man_MANS. This makes the hairy script code slightly less hairy. If it breaks some day, this code needs to be fixed; don't blame Automake developers. Also, this adds more quotes to the existing shell script code in the Makefile.am "-hook"s.
2020-02-05xz: Make it a fatal error if enabling the sandbox fails.Lasse Collin1-1/+1
Perhaps it's too drastic but on the other hand it will let me learn about possible problems if people report the errors. This won't be backported to the v5.2 branch.
2020-02-05xz: Comment out annoying sandboxing messages.Lasse Collin1-3/+7
2020-02-01xz: Limit --memlimit-compress to at most 4020 MiB for 32-bit xz.Lasse Collin2-2/+51
See the code comment for reasoning. It's far from perfect but hopefully good enough for certain cases while hopefully doing nothing bad in other situations. At presets -5 ... -9, 4020 MiB vs. 4096 MiB makes no difference on how xz scales down the number of threads. The limit has to be a few MiB below 4096 MiB because otherwise things like "xz --lzma2=dict=500MiB" won't scale down the dict size enough and xz cannot allocate enough memory. With "ulimit -v $((4096 * 1024))" on x86-64, the limit in xz had to be no more than 4085 MiB. Some safety margin is good though. This is hack but it should be useful when running 32-bit xz on a 64-bit kernel that gives full 4 GiB address space to xz. Hopefully this is enough to solve this: https://bugzilla.redhat.com/show_bug.cgi?id=1196786 FreeBSD has a patch that limits the result in tuklib_physmem() to SIZE_MAX on 32-bit systems. While I think it's not the way to do it, the results on --memlimit-compress have been good. This commit should achieve practically identical results for compression while leaving decompression and tuklib_physmem() and thus lzma_physmem() unaffected.
2020-01-26xz: Set the --flush-timeout deadline when the first input byte arrives.Lasse Collin3-7/+6
xz --flush-timeout=2000, old version: 1. xz is started. The next flush will happen after two seconds. 2. No input for one second. 3. A burst of a few kilobytes of input. 4. No input for one second. 5. Two seconds have passed and flushing starts. The first second counted towards the flush-timeout even though there was no pending data. This can cause flushing to occur more often than needed. xz --flush-timeout=2000, after this commit: 1. xz is started. 2. No input for one second. 3. A burst of a few kilobytes of input. The next flush will happen after two seconds counted from the time when the first bytes of the burst were read. 4. No input for one second. 5. No input for another second. 6. Two seconds have passed and flushing starts.
2020-01-26xz: Move flush_needed from mytime.h to file_pair struct in file_io.h.Lasse Collin5-9/+7
2020-01-26xz: coder.c: Make writing output a separate function.Lasse Collin1-13/+17
The same code sequence repeats so it's nicer as a separate function. Note that in one case there was no test for opt_mode != MODE_TEST, but that was only because that condition would always be true, so this commit doesn't change the behavior there.
2020-01-26xz: Fix semi-busy-waiting in xz --flush-timeout.Lasse Collin3-4/+19
When input blocked, xz --flush-timeout=1 would wake up every millisecond and initiate flushing which would have nothing to flush and thus would just waste CPU time. The fix disables the timeout when no input has been seen since the previous flush.
2020-01-26xz: Refactor io_read() a bit.Lasse Collin1-9/+8
2020-01-26xz: Update a comment in file_io.h.Lasse Collin1-1/+4
2020-01-26xz: Move the setting of flush_needed in file_io.c to a nicer location.Lasse Collin1-4/+2
2019-06-28xz: Automatically align the strings in --info-memory.Lasse Collin1-11/+34
This makes it easier to translate the strings. Also, the string for amount of RAM was shortened.
2019-06-24Add LZMA_RET_INTERNAL1..8 to lzma_ret and use one for LZMA_TIMED_OUT.Lasse Collin1-0/+8
LZMA_TIMED_OUT is *internally* used as a value for lzma_ret enumeration. Previously it was #defined to 32 and cast to lzma_ret. That way it wasn't visible in the public API, but this was hackish. Now the public API has eight LZMA_RET_INTERNALx members and LZMA_TIMED_OUT is #defined to LZMA_RET_INTERNAL1. This way the code is cleaner overall although the public API has a few extra mysterious enum members.
2019-06-24xz: Silence a warning from clang -Wsign-conversion in main.c.Lasse Collin1-1/+1
2019-06-24xz: Make "headings" static in list.c.Lasse Collin1-1/+1
Caught by clang -Wmissing-variable-declarations.
2019-06-24xz: Fix an integer overflow with 32-bit off_t.Lasse Collin1-2/+9
Or any off_t which isn't very big (like signed 64 bit integer that most system have). A small off_t could overflow if the file being decompressed had long enough run of zero bytes, which would result in corrupt output.
2019-06-24xz: Cleanup io_seek_src() a bit.Lasse Collin1-3/+1
lseek() returns -1 on error and checking for -1 is nicer.
2019-06-24xz: Change io_seek_src and io_pread arguments from off_t to uint64_t.Lasse Collin3-11/+18
This helps fixing warnings from -Wsign-conversion and makes the code look better too.
2019-06-24xz: list.c: Fix some warnings from -Wsign-conversion.Lasse Collin1-3/+4
2019-06-23xz: Fix some of the warnings from -Wsign-conversion.Lasse Collin7-13/+14
2019-05-11xz: Update xz man page date.Lasse Collin1-1/+1
2019-05-11spellingAntoine Cœur7-9/+9
2019-05-01xz: In xz -lvv look at the widths of the check names too.Lasse Collin1-6/+26
Now the widths of the check names is used to adjust the width of the Check column. This way there no longer is a need to restrict the widths of the check names to be at most ten terminal-columns.
2019-05-01xz: Fix xz -lvv column alignment to look at the translated strings.Lasse Collin1-2/+2
2019-03-04xz: Automatically align column headings in xz -lvv.Lasse Collin1-51/+212
2019-03-04xz: Automatically align strings ending in a colon in --list output.Lasse Collin1-12/+102
This should avoid alignment errors in translations with these strings.
2018-12-20xz: Fix a crash in progress indicator when in passthru mode.Lasse Collin3-7/+25
"xz -dcfv not_an_xz_file" crashed (all four options are required to trigger it). It caused xz to call lzma_get_progress(&strm, ...) when no coder was initialized in strm. In this situation strm.internal is NULL which leads to a crash in lzma_get_progress(). The bug was introduced when xz started using lzma_get_progress() to get progress info for multi-threaded compression, so the bug is present in versions 5.1.3alpha and higher. Thanks to Filip Palian <Filip.Palian@pjwstk.edu.pl> for the bug report.
2018-11-22xz: Update man page timestamp.Lasse Collin1-1/+1
2018-11-22'have have' typosPavel Raiskup2-2/+2
2017-08-14Fix or hide warnings from GCC 7's -Wimplicit-fallthrough.Lasse Collin1-0/+2
2017-05-23xz: Fix "xz --list --robot missing_or_bad_file.xz".Lasse Collin1-2/+6
It ended up printing an uninitialized char-array when trying to print the check names (column 7) on the "totals" line. This also changes the column 12 (minimum xz version) to 50000002 (xz 5.0.0) instead of 0 when there are no valid input files. Thanks to kidmin for the bug report.
2017-04-24xz: Use lzma_file_info_decoder() for --list.Lasse Collin1-210/+44
2017-04-21liblzma: Rename LZMA_SEEK to LZMA_SEEK_NEEDED and seek_in to seek_pos.Lasse Collin1-1/+1
2017-04-19Update the home page URLs to HTTPS.Lasse Collin1-3/+3
2017-04-05xz: Add io_seek_src().Lasse Collin2-3/+30
2017-03-30xz: Use POSIX_FADV_RANDOM for in "xz --list" mode.Lasse Collin1-2/+8
xz --list is random access so POSIX_FADV_SEQUENTIAL was clearly wrong.
2017-03-30liblzma: Add generic support for input seeking (LZMA_SEEK).Lasse Collin1-0/+1
Also mention LZMA_SEEK in xz/message.c to silence a warning.
2016-06-30xz: Fix copying of timestamps on Windows.Lasse Collin1-0/+18
xz used to call utime() on Windows, but its result gets lost on close(). Using _futime() seems to work. Thanks to Martok for reporting the bug: http://www.mail-archive.com/xz-devel@tukaani.org/msg00261.html
2016-06-16xz: Silence warnings from -Wlogical-op.Lasse Collin1-2/+10
Thanks to Evan Nemerson.
2016-04-10Build: Fix = to += for xz_SOURCES in src/xz/Makefile.am.Lasse Collin1-1/+1
Thanks to Christian Kujau.
2015-11-03xz: Make xz buildable even when encoders or decoders are disabled.Lasse Collin5-13/+58
The patch is quite long but it's mostly about adding new #ifdefs to omit code when encoders or decoders have been disabled. This adds two new #defines to config.h: HAVE_ENCODERS and HAVE_DECODERS.
2015-11-02xz: Always close the file before trying to delete it.Lasse Collin1-13/+12
unlink() can return EBUSY in errno for open files on some operating systems and file systems.
2015-05-11xz: Document that threaded decompression hasn't been implemented yet.Lasse Collin1-1/+9
2015-04-20Revert "xz: Use pipe2() if available."Lasse Collin1-8/+1
This reverts commit 7a11c4a8e5e15f13d5fa59233b3172e65428efdd. It is a problem when libc has pipe2() but the kernel is too old to have pipe2() and thus pipe2() fails. In xz it's pointless to have a fallback for non-functioning pipe2(); it's better to avoid pipe2() completely. Thanks to Michael Fox for the bug report.
2015-04-01xz: Fix the Capsicum rights on user_abort_pipe.Lasse Collin1-1/+5
2015-03-31xz: Add support for sandboxing with Capsicum.Lasse Collin5-1/+110
The sandboxing is used conditionally as described in main.c. This isn't optimal but it was much easier to implement than a full sandboxing solution and it still covers the most common use cases where xz is writing to standard output. This should have practically no effect on performance even with small files as fork() isn't needed. C and locale libraries can open files as needed. This has been fine in the past, but it's a problem with things like Capsicum. io_sandbox_enter() tries to ensure that various locale-related files have been loaded before cap_enter() is called, but it's possible that there are other similar problems which haven't been seen yet. Currently Capsicum is available on FreeBSD 10 and later and there is a port to Linux too. Thanks to Loganaden Velvindron for help.
2015-03-07xz: size_t/uint32_t cleanup in options.c.Lasse Collin1-6/+6
2015-03-07xz: Fix a comment and silence a warning in message.c.Lasse Collin1-2/+3
2015-03-07xz: Make arg_count an unsigned int to silence a warning.Lasse Collin2-2/+2
Actually the value of arg_count cannot exceed INT_MAX but it's nicer as an unsigned int.
2015-02-22xz: Use pipe2() if available.Lasse Collin1-1/+8
2015-02-21xz: Fix the fcntl() usage when creating a pipe for the self-pipe trick.Lasse Collin1-5/+11
Now it reads the old flags instead of blindly setting O_NONBLOCK. The old code may have worked correctly, but this is better.
2015-01-09xz: Fix comments.Lasse Collin1-4/+8
2015-01-09xz: Don't fail if stdout doesn't support O_NONBLOCK.Lasse Collin1-21/+15
This is similar to the case with stdin. Thanks to Brad Smith for the bug report and testing on OpenBSD.
2015-01-07xz: Fix a memory leak in DOS-specific code.Lasse Collin1-0/+2
2015-01-07xz: Don't fail if stdin doesn't support O_NONBLOCK.Lasse Collin1-11/+7
It's a problem at least on OpenBSD which doesn't support O_NONBLOCK on e.g. /dev/null. I'm not surprised if it's a problem on other OSes too since this behavior is allowed in POSIX-1.2008. The code relying on this behavior was committed in June 2013 and included in 5.1.3alpha released on 2013-10-26. Clearly the development releases only get limited testing.
2014-12-21xz: Fix a comment.Lasse Collin1-2/+2
2014-12-16xz: Update the man page about --threads.Lasse Collin1-5/+0
2014-12-16xz: Update the man page about --block-size.Lasse Collin1-8/+33
2014-11-26Remove LZMA_UNSTABLE macro.Lasse Collin1-1/+0
2014-10-29Build: Prepare to support Automake's subdir-objects.Lasse Collin1-5/+5
Due to a bug in Automake, subdir-objects won't be enabled for now. http://debbugs.gnu.org/cgi/bugreport.cgi?bug=17354 Thanks to Daniel Richard G. for the original patches.
2014-08-05xz: Add --ignore-check.Lasse Collin5-1/+38
2014-06-29xz: Update the help message of a few options.Lasse Collin1-7/+11
Updated: --threads, --block-size, and --block-list Added: --flush-timeout
2014-06-18xz: Use lzma_cputhreads() instead of own copy of tuklib_cpucores().Lasse Collin2-4/+9
2014-06-18xz: Check for filter chain compatibility for --flush-timeout.Lasse Collin1-9/+21
This avoids LZMA_PROG_ERROR from lzma_code() with filter chains that don't support LZMA_SYNC_FLUSH.
2014-06-09xz: Force single-threaded mode when --flush-timeout is used.Lasse Collin1-0/+11
2014-05-08xz: Fix uint64_t vs. size_t which broke 32-bit build.Lasse Collin1-1/+1
Thanks to Christian Hesse.
2014-04-09xz: Rename a variable to avoid a namespace collision on Solaris.Lasse Collin1-5/+7
I don't know the details but I have an impression that there's no problem in practice if using GCC since people have built xz with GCC (without patching xz), but renaming the variable cannot hurt either. Thanks to Mark Ashley.
2014-01-12xz: Fix a comment.Lasse Collin1-2/+2
2014-01-12xz: Fix use of wrong variable.Lasse Collin1-1/+1
Since the only call to suffix_set() uses optarg as the argument, fixing this bug doesn't change the behavior of the program.
2013-11-12xz: Update the man page about --block-size and --block-list.Lasse Collin1-9/+15
2013-11-12xz: Make --block-list and --block-size work together in single-threaded.Lasse Collin1-15/+75
Previously, --block-list and --block-size only worked together in threaded mode. Boundaries are specified by --block-list, but --block-size specifies the maximum size for a Block. Now this works in single-threaded mode too. Thanks to James M Leddy for the original patch.
2013-10-25xz: Document behavior of --block-list with threads.Lasse Collin1-3/+21
This needs to be updated before 5.2.0.
2013-10-22xz: Document --flush-timeout=TIMEOUT on the man page.Lasse Collin1-1/+36
2013-10-22xz: Take advantage of LZMA_FULL_BARRIER with --block-list.Lasse Collin1-17/+15
Now if --block-list is used in threaded mode, the encoder won't need to flush at each Block boundary specified via --block-list. This improves performance a lot, making threading helpful with --block-list. The flush timer was reset after LZMA_FULL_FLUSH but since LZMA_FULL_BARRIER doesn't flush, resetting the timer is no longer done.
2013-09-17xz: Add a missing test for TUKLIB_DOSLIKE.Lasse Collin1-0/+2
2013-09-17Add native threading support on Windows.Lasse Collin1-4/+4
Now liblzma only uses "mythread" functions and types which are defined in mythread.h matching the desired threading method. Before Windows Vista, there is no direct equivalent to pthread condition variables. Since this package doesn't use pthread_cond_broadcast(), pre-Vista threading can still be kept quite simple. The pre-Vista code doesn't use anything that wasn't already available in Windows 95, so the binaries should run even on Windows 95 if someone happens to care.
2013-07-04xz: Add preliminary support for --flush-timeout=TIMEOUT.Lasse Collin3-21/+78
When --flush-timeout=TIMEOUT is used, xz will use LZMA_SYNC_FLUSH if read() would block and at least TIMEOUT milliseconds has elapsed since the previous flush. This can be useful in realtime-like use cases where the data is simultanously decompressed by another process (possibly on a different computer). If new uncompressed input data is produced slowly, without this option xz could buffer the data for a long time until it would become decompressible from the output. If TIMEOUT is 0, the feature is disabled. This is the default. This commit affects the compression side. Using xz for the decompression side for the above purpose doesn't work yet so well because there is quite a bit of input and output buffering when decompressing. The --long-help or man page were not updated yet. The details of this feature may change.
2013-07-04xz: Don't set src_eof=true after an I/O error because it's useless.Lasse Collin1-3/+0
2013-07-04xz: Fix the test when to read more input.Lasse Collin1-3/+3
Testing for end of file was no longer correct after full flushing became possible with --block-size=SIZE and --block-list=SIZES. There was no bug in practice though because xz just made a few unneeded zero-byte reads.
2013-07-04xz: Move some of the timing code into mytime.[hc].Lasse Collin6-40/+158
This switches units from microseconds to milliseconds. New clock_gettime(CLOCK_MONOTONIC) will be used if available. There is still a fallback to gettimeofday().
2013-07-01xz: Silence a warning seen with _FORTIFY_SOURCE=2.Lasse Collin1-1/+7
Thanks to Christian Hesse.
2013-06-29xz: Use non-blocking I/O for the output file.Lasse Collin1-8/+49
Now both reading and writing should be without race conditions with signals. They might still be signal handling issues left. Signals are blocked during many operations to avoid EINTR but it may cause problems e.g. if writing to stderr blocks when trying to display an error message.
2013-06-28xz: Fix return value type in io_write_buf().Lasse Collin1-1/+1
It didn't affect the behavior of the code since -1 becomes true anyway.
2013-06-28xz: Use the self-pipe trick to avoid a race condition with signals.Lasse Collin3-12/+57
It is possible that a signal to set user_abort arrives right before a blocking system call is made. In this case the call may block until another signal arrives, while the wanted behavior is to make xz clean up and exit as soon as possible. After this commit, the race condition is avoided with the input side which already uses non-blocking I/O. The output side still uses blocking I/O and thus has the race condition.
2013-06-28xz: Use non-blocking I/O for the input file.Lasse Collin1-45/+111
2013-06-28xz: Remove an outdated NetBSD-specific comment.Lasse Collin1-4/+0
Nowadays errno == EFTYPE is documented in open(2).
2013-06-28xz: Fix error detection of fcntl(fd, F_SETFL, flags) calls.Lasse Collin1-4/+4
POSIX says that fcntl(fd, F_SETFL, flags) returns -1 on error and "other than -1" on success. This is how it is documented e.g. on OpenBSD too. On Linux, success with F_SETFL is always 0 (at least accorinding to fcntl(2) from man-pages 3.51).
2013-06-28xz: Fix use of wrong variable in a fcntl() call.Lasse Collin1-11/+13
Due to a wrong variable name, when writing a sparse file to standard output, *all* file status flags were cleared (to the extent the operating system allowed it) instead of only clearing the O_APPEND flag. In practice this worked fine in the common situations on GNU/Linux, but I didn't check how it behaved elsewhere. The original flags were still restored correctly. I still changed the code to use a separate boolean variable to indicate when the flags should be restored instead of relying on a special value in stdout_flags.
2013-06-28xz: Fix assertion related to posix_fadvise().Lasse Collin1-8/+2
Input file can be a FIFO or something else that doesn't support posix_fadvise() so don't check the return value even with an assertion. Nothing bad happens if the call to posix_fadvise() fails.
2013-06-26xz: Check the value of lzma_stream_flags.version in --list.Lasse Collin1-0/+14
It is a no-op for now, but if an old xz version is used together with a newer liblzma that supports something new, then this check becomes important and will stop the old xz from trying to parse files that it won't understand.
2013-06-23xz: Validate Uncompressed Size from Block Header in list.c.Lasse Collin1-1/+13
This affects only "xz -lvv". Normal decompression with xz already detected if Block Header and Index had mismatched Uncompressed Size fields. So this just makes "xz -lvv" show such files as corrupt instead of showing the Uncompressed Size from Index.
2013-06-21xz: Make the man page more friendly to doclifter.Lasse Collin1-3/+4
Thanks to Eric S. Raymond.
2013-06-21xz: A couple of man page fixes.Lasse Collin1-12/+23
Now the interaction of presets and custom filter chains is described correctly. Earlier it contradicted itself. Thanks to DevHC who reported these issues on IRC to me on 2012-12-14.
2013-06-21xz: Fix interaction between preset and custom filter chains.Lasse Collin1-14/+21
There was somewhat illogical behavior when --extreme was specified and mixed with custom filter chains. Before this commit, "xz -9 --lzma2 -e" was equivalent to "xz --lzma2". After it is equivalent to "xz -6e" (all earlier preset options get forgotten when a custom filter chain is specified and the default preset is 6 to which -e is applied). I find this less illogical. This also affects the meaning of "xz -9e --lzma2 -7". Earlier it was equivalent to "xz -7e" (the -e specified before a custom filter chain wasn't forgotten). Now it is "xz -7". Note that "xz -7e" still is the same as "xz -e7". Hopefully very few cared about this in the first place, so pretty much no one should even notice this change. Thanks to Conley Moorhous.
2012-12-14Make the progress indicator smooth in threaded mode.Lasse Collin1-7/+13
This adds lzma_get_progress() to liblzma and takes advantage of it in xz. lzma_get_progress() collects progress information from the thread-specific structures so that fairly accurate progress information is available to applications. Adding a new function seemed to be a better way than making the information directly available in lzma_stream (like total_in and total_out are) because collecting the information requires locking mutexes. It's waste of time to do it more often than the up to date information is actually needed by an application.
2012-12-13xz: Mention --threads in --help.Lasse Collin1-0/+4
Thanks to Olivier Delhomme for pointing out that this was still missing.
2012-10-03xz: Fix the note about --rsyncable on the man page.Lasse Collin1-8/+9
2012-09-28xz: Improve handling of failed realloc in xrealloc.Lasse Collin1-2/+12
Thanks to Jim Meyering.
2012-08-24A few typo fixes to comments and the xz man page.Lasse Collin1-2/+2
Thanks to Jim Meyering.
2012-08-13xz: Add a warning to --help about alpha and beta versions.Lasse Collin1-0/+5
2012-07-04xz: Document --block-list better.Lasse Collin1-1/+7
Thanks to Jonathan Nieder.
2012-07-04xz: Fix the version number printed by xz -lvv.Lasse Collin1-3/+3
The decoder bug was fixed in 5.0.2 instead of 5.0.3.
2012-07-03xz: Add incomplete support for --block-list.Lasse Collin7-10/+151
It's broken with threads and when also --block-size is used.
2012-07-01xz: Update the man page about the new field in --robot -lvv.Lasse Collin1-1/+17
2012-06-22xz: Update man page date to match the latest update.Lasse Collin1-1/+1
2012-05-28xz: Don't show a huge number in -vv when memory limit is disabled.Lasse Collin1-1/+11
2012-05-27xz: Document the "summary" lines of --robot -lvv.Lasse Collin1-0/+19
This documents only the columns that are in v5.0. The new columns added in the master branch aren't necessarily stable yet.
2012-05-27xz: Fix output of verbose --robot --list modes.Lasse Collin1-1/+1
It printed the filename in "filename (x/y)" format which it obviously shouldn't do in robot mode.
2011-11-07xz: Show minimum required XZ Utils version in xz -lvv.Lasse Collin1-6/+57
Man page wasn't updated yet.
2011-11-04xz: Fix a typo in a comment.Lasse Collin1-1/+1
Thanks to Bela Lubkin.
2011-11-03xz: Fix xz on EBCDIC systems.Lasse Collin1-1/+4
Thanks to Chris Donawa.
2011-05-27xz: Fix error handling in xz -lvv.Lasse Collin1-15/+6
It could do an invalid free() and read past the end of the uninitialized filters array.
2011-05-21xz: Translate also the string used to print the program name.Lasse Collin1-1/+5
French needs a space before a colon, e.g. "xz : foo error".
2011-05-17Add underscores to attributes (__attribute((__foo__))).Lasse Collin7-19/+19
2011-05-01xz: Fix input file position when --single-stream is used.Lasse Collin3-0/+29
Now the following works as you would expect: echo foo | xz > foo.xz echo bar | xz >> foo.xz ( xz -dc --single-stream ; xz -dc --single-stream ) < foo.xz Note that it doesn't work if the input is not seekable or if there is Stream Padding between the concatenated .xz Streams.
2011-05-01xz: Print the maximum number of worker threads in xz -vv.Lasse Collin1-0/+4
2011-04-12Put the unstable APIs behind #ifdef LZMA_UNSTABLE.Lasse Collin1-0/+2
This way people hopefully won't complain if these APIs change and break code that used an older API.
2011-04-12Remove doubled words from documentation and comments.Lasse Collin2-2/+2
Spot candidates by running these commands: git ls-files |xargs perl -0777 -n \ -e 'while (/\b(then?|[iao]n|i[fst]|but|f?or|at|and|[dt]o)\s+\1\b/gims)' \ -e '{$n=($` =~ tr/\n/\n/ + 1); ($v=$&)=~s/\n/\\n/g; print "$ARGV:$n:$v\n"}' Thanks to Jim Meyering for the original patch.
2011-04-12xz: Update the man page about threading.Lasse Collin1-14/+20
2011-04-11xz: Add support for threaded compression.Lasse Collin2-80/+125
2011-04-11xz: Move the description of --block-size in --long-help.Lasse Collin1-4/+4
2011-04-11Docs: Document --single-stream and --block-size.Lasse Collin1-2/+36
2011-04-10xz/DOS: Add experimental 8.3 filename support.Lasse Collin1-9/+167
This is incompatible with the 8.3 support patch made by Juan Manuel Guerrero. I think this one is nicer, but I need to get feedback from DOS users before saying that this is the final version of 8.3 filename support.
2011-04-10xz/DOS: Be more careful with the destination file.Lasse Collin1-2/+33
Try to avoid overwriting the source file if --force is used and the generated destination filename refers to the source file. This can happen with 8.3 filenames where extra characters are ignored. If the generated output file refers to a special file like "con" or "prn", refuse to write to it even if --force is used.
2011-04-09xz: Avoid unneeded fstat() on DOS-like systems.Lasse Collin1-6/+8
2011-04-09xz: Minor internal changes to handling of --threads.Lasse Collin3-19/+18
Now it always defaults to one thread. Maybe this will change again if a threading method is added that doesn't affect memory usage.
2011-04-08xz: Change size_t to uint32_t in a few places.Lasse Collin2-4/+4
2011-04-08xz: Fix a typo in a comment.Lasse Collin1-1/+1
2011-04-05xz: Use posix_fadvise() if it is available.Lasse Collin1-0/+15
2011-04-05xz: Call lzma_end(&strm) before exiting if debugging is enabled.Lasse Collin3-0/+19
2011-03-18xz: Add --block-size=SIZE.Lasse Collin4-10/+54
This uses LZMA_FULL_FLUSH every SIZE bytes of input. Man page wasn't updated yet.
2011-03-18xz: Add --single-stream.Lasse Collin4-3/+23
This can be useful when there is garbage after the compressed stream (.xz, .lzma, or raw stream). Man page wasn't updated yet.
2011-02-06xz: Clean up suffix.c.Lasse Collin1-24/+20
struct suffix_pair isn't needed in compresed_name() so get rid of it there.
2011-02-06xz: Check if the file already has custom suffix when compressing.Lasse Collin1-0/+9
Now "xz -S .test foo.test" refuses to compress the file because it already has the suffix .test. The man page had it documented this way already.
2011-01-26xz: Fix --force on setuid/setgid/sticky and multi-hardlink files.Lasse Collin1-8/+7
xz didn't compress setuid/setgid/sticky files and files with multiple hard links even with --force. This bug was introduced in 23ac2c44c3ac76994825adb7f9a8f719f78b5ee4. Thanks to Charles Wilson.
2011-01-18Add alloc_size and malloc attributes to a few functions.Lasse Collin1-2/+3
Thanks to Cristian Rodríguez for the original patch.
2010-12-12DOS-like: Treat \ and : as directory separators in addition to /.Lasse Collin1-4/+29
Juan Manuel Guerrero had fixed this in his XZ Utils port to DOS/DJGPP. The bug affects also Windows and OS/2.
2010-10-21xz: Make sure that message_strm() can never return NULL.Lasse Collin1-2/+5
2010-10-19Clean up a few FIXMEs and TODOs.Lasse Collin1-1/+1
lzma_chunk_size() was commented out because it is currently useless.
2010-10-12xz: Avoid raise() also on OpenVMS.Lasse Collin1-1/+1
This is similar to DOS/DJGPP that killing the program with a signal will print a backtrace or a similar message.
2010-10-11xz: Avoid SA_RESTART for portability reasons.Lasse Collin3-23/+25
SA_RESTART is not as portable as I had hoped. It's missing at least from OpenVMS, QNX, and DJGPP). Luckily we can do fine without SA_RESTART.
2010-10-10xz: Use "%"PRIu32 instead of "%d" in a format string.Lasse Collin1-1/+1
2010-10-09OS/2 and DOS: Be less verbose on signals.Lasse Collin1-0/+7
Calling raise() to kill xz when user has pressed C-c is a bit verbose on OS/2 and DOS/DJGPP. Instead of calling raise(), set only the exit status to 1.
2010-10-08Build: Add options to disable individual command line tools.Lasse Collin1-1/+5
2010-10-05Build: Remove the static/dynamic tricks.Lasse Collin1-3/+1
Most distros want xz linked against shared liblzma, so it doesn't help much to require --enable-dynamic for that. Those who want to avoid PIC on x86-32 to get better performance, can still do it e.g. by using --disable-shared to compile xz and then another pass to compile shared liblzma. Part of these static/dynamic tricks were needed for Windows in the past. Nowadays we rely on GCC and binutils to do the right thing with auto-import. If the Autotooled build system needs to support some other toolchain on Windows in the future, this may need some rethinking.
2010-10-04A few more languages files to the xz man page.Lasse Collin1-21/+24
Thanks to Jonathan Nieder.
2010-09-28Fix accomodate -> accommodate on the xz man page.Lasse Collin1-1/+1
2010-09-27Major man page updates.Lasse Collin1-598/+1366
Lots of content was updated on the xz man page. Technical improvements: - Start a new sentence on a new line. - Use fairly short lines. - Use constant-width font for examples (where supported). - Some minor cleanups. Thanks to Jonathan Nieder for some language fixes.
2010-09-15xz: Edit a translators comment.Lasse Collin1-1/+1
2010-09-10xz: Add a comment to translators about "literal context bits".Lasse Collin1-0/+4
2010-09-10xz: Multiple fixes.Lasse Collin7-187/+232
The code assumed that printing numbers with thousand separators and decimal points would always produce only US-ASCII characters. This was used for buffer sizes (with snprintf(), no overflows) and aligning columns of the progress indicator and --list. That assumption was wrong (e.g. LC_ALL=fi_FI.UTF-8 with glibc), so multibyte character support was added in this commit. The old way is used if the operating system doesn't have enough multibyte support (e.g. lacks wcwidth()). The sizes of buffers were increased to accomodate multibyte characters. I don't know how big they should be exactly, but they aren't used for anything critical, so it's not too bad. If they still aren't big enough, I hopefully get a bug report. snprintf() takes care of avoiding buffer overflows. Some static buffers were replaced with buffers allocated on stack. double_to_str() was removed. uint64_to_str() and uint64_to_nicestr() now share the static buffer and test for thousand separator support. Integrity check names "None" and "Unknown-N" (2 <= N <= 15) were marked to be translated. I had forgot these, plus they wouldn't have worked correctly anyway before this commit, because printing tables with multibyte strings didn't work. Thanks to Marek Černocký for reporting the bug about misaligned table columns in --list output.
2010-09-07xz: Add a note to translators.Lasse Collin1-0/+2
2010-09-07Fix use of N_() and ngettext().Lasse Collin1-2/+2
I had somehow thought that N_() is usually used as shorthand for ngettext(). This also fixes a missing \n from a call to ngettext().
2010-09-06xz: Improve a comment.Lasse Collin1-3/+4
2010-09-05xz: Update the comment about NetBSD in file_io.c.Lasse Collin1-4/+4
Thanks to Joerg Sonnenberger.
2010-09-05xz: Use an array instead of pointer for stdin_filename.Lasse Collin2-2/+2
Thanks Joerg Sonnenberger.
2010-09-05xz: Hopefully ease translating the messages in list.c.Lasse Collin1-10/+23
2010-09-04xz: Fix grammar.Lasse Collin1-2/+1
2010-09-04xz: Use lzma_lzma_preset() to initialize the options structure.Lasse Collin1-12/+2
2010-09-04xz: Minor improvements to --help and --long-help.Lasse Collin1-12/+12
2010-09-03xz: Make -vv show also decompressor memory usage.Lasse Collin1-0/+7
2010-09-03xz: Make --help two lines shorter.Lasse Collin1-2/+5
At least for now, the --help option doesn't list any options that take arguments, so "Mandatory arguments to..." can be omitted.
2010-09-02xz: Make setting a preset override a custom filter chain.Lasse Collin1-0/+9
This is more logical behavior than ignoring preset level options once a custom filter chain has been specified.
2010-09-02xz: Always warn if adjusting dictionary size due to memlimit.Lasse Collin1-19/+9
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-19 01:13:18 +0000