aboutsummaryrefslogtreecommitdiff
path: root/src/xz/file_io.c (follow)
AgeCommit message (Collapse)AuthorFilesLines
2024-02-14Add SPDX license identifier into 0BSD source code files.Lasse Collin1-0/+2
2024-02-14Change most public domain parts to 0BSD.Lasse Collin1-3/+0
Translations and doc/xz-file-format.txt and doc/lzma-file-format.txt were not touched. COPYING.0BSD was added.
2023-12-21xz: Add a comment to Capsicum sandbox setup.Jia Tan1-0/+1
This comment is repeated in xzdec.c to help remind us why all the capabilities are removed from stdin in certain situations.
2023-11-30xz: Fix typoKian-Meng Ang1-1/+1
2023-10-22xz: Support basic sandboxing with Linux Landlock (ABI versions 1-3).Lasse Collin1-0/+58
It is enabled only when decompressing one file to stdout, similar to how Capsicum is used. Landlock was added in Linux 5.13.
2023-10-22Simplify detection of Capsicum support.Lasse Collin1-9/+5
This removes support for FreeBSD 10.0 and 10.1 which used <sys/capability.h> instead of <sys/capsicum.h>. Support for FreeBSD 10.1 ended on 2016-12-31. So now FreeBSD >= 10.2 is required to enable Capsicum support. This also removes support for Capsicum on Linux (libcaprights) which seems to have been unmaintained since 2017 and Linux 4.11: https://github.com/google/capsicum-linux
2023-09-24xz: Change quoting style from `...' to '...'.Jia Tan1-1/+1
2023-09-22xz: Windows: Don't (de)compress to special files like "con" or "nul".Lasse Collin1-7/+28
Before this commit, the following writes "foo" to the console and deletes the input file: echo foo | xz > con_xz xz --suffix=_xz --decompress con_xz It cannot happen without --suffix because names like con.xz are also special and so attempting to decompress con.xz (or compress con to con.xz) will already fail when opening the input file. Similar thing is possible when compressing. The following writes to "nul" and the input file "n" is deleted. echo foo | xz > n xz --suffix=ul n Now xz checks if the destination is a special file before continuing. DOS/DJGPP version had a check for this but Windows (and OS/2) didn't.
2023-09-22MSVC: xz: Make file_io.c and file_io.h compatible with MSVC.Lasse Collin1-0/+26
Thanks to Kelvin Lee for the original patches and testing the modifications I made.
2023-08-31xz: Fix a too relaxed assertion and remove uses of SSIZE_MAX.Lasse Collin1-3/+2
SSIZE_MAX isn't readily available on MSVC. Removing it means that there is one thing less to worry when porting to MSVC.
2023-07-18xz: Make "%s: %s" translatable because French needs "%s : %s".Lasse Collin1-4/+4
2023-03-11xz: Simplify the error-label in Capsicum sandbox code.Lasse Collin1-15/+12
Also remove unneeded "sandbox_allowed = false;" as this code will never be run more than once (making it work with multiple input files isn't trivial).
2023-03-08xz: Make Capsicum sandbox more strict with stdin and stdout.Lasse Collin1-0/+8
2023-03-08Revert: "Add warning if Capsicum sandbox system calls are unsupported."Jia Tan1-6/+4
The warning causes the exit status to be 2, so this will cause problems for many scripted use cases for xz. The sandbox usage is already very limited already, so silently disabling this allows it to be more usable.
2023-03-07xz: Fix -Wunused-label in io_sandbox_enter().Jia Tan1-2/+2
Thanks to Xin Li for recommending the fix.
2023-03-06xz: Add warning if Capsicum sandbox system calls are unsupported.Jia Tan1-0/+2
The warning is only used when errno == ENOSYS. Otherwise, xz still issues a fatal error.
2023-03-06xz: Skip Capsicum sandbox system calls when they are unsupported.Jia Tan1-5/+17
If a system has the Capsicum header files but does not actually implement the system calls, then this would render xz unusable. Instead, we can check if errno == ENOSYS and not issue a fatal error.
2023-03-06xz: Reorder cap_enter() to beginning of capsicum sandbox code.Jia Tan1-3/+3
cap_enter() puts the process into the sandbox. If later calls to cap_rights_limit() fail, then the process can still have some extra protections.
2023-01-12xz: Use ssize_t for the to-be-ignored return value from write(fd, ptr, 1).Lasse Collin1-1/+1
It makes no difference here as the return value fits into an int too and it then gets ignored but this looks better.
2022-11-09xz: Add comments about stdin and src_st.st_size.Lasse Collin1-0/+4
"xz -v < regular_file > out.xz" doesn't display the percentage and estimated remaining time because it doesn't even try to check the input file size when input is read from stdin. This could be improved but for now there's just a comment to remind about it.
2022-10-25xz: Add support for OpenBSD's pledge() sandbox.Lasse Collin1-0/+11
2022-10-25xz: Refactor to remove is_empty_filename().Lasse Collin1-1/+3
Long ago it was used in list.c too but nowadays it's needed only in io_open_src() so it's nicer to avoid a separate function.
2022-10-25xz: If input file cannot be removed, treat it as a warning, not error.Lasse Collin1-2/+2
Treating it as a warning (message + exit status 2) matches gzip and it seems more logical as at that point the output file has already been successfully closed. When it's a warning it is possible to suppress it with --no-warn.
2021-10-27xz: Change the coding style of the previous commit.Lasse Collin1-5/+6
It isn't any better now but it's consistent with the rest of the code base.
2021-10-27xz: Avoid fchown(2) failure.Alexander Bluhm1-1/+7
OpenBSD does not allow to change the group of a file if the user does not belong to this group. In contrast to Linux, OpenBSD also fails if the new group is the same as the old one. Do not call fchown(2) in this case, it would change nothing anyway. This fixes an issue with Perl Alien::Build module. https://github.com/PerlAlien/Alien-Build/issues/62
2021-01-11xz: Make --keep accept symlinks, hardlinks, and setuid/setgid/sticky.Lasse Collin1-4/+5
Previously this required using --force but that has other effects too which might be undesirable. Changing the behavior of --keep has a small risk of breaking existing scripts but since this is a fairly special corner case I expect the likehood of breakage to be low enough. I think the new behavior is more logical. The only reason for the old behavior was to be consistent with gzip and bzip2. Thanks to Vincent Lefevre and Sebastian Andrzej Siewior.
2020-02-05xz: Make it a fatal error if enabling the sandbox fails.Lasse Collin1-1/+1
Perhaps it's too drastic but on the other hand it will let me learn about possible problems if people report the errors. This won't be backported to the v5.2 branch.
2020-02-05xz: Comment out annoying sandboxing messages.Lasse Collin1-3/+7
2020-01-26xz: Set the --flush-timeout deadline when the first input byte arrives.Lasse Collin1-1/+5
xz --flush-timeout=2000, old version: 1. xz is started. The next flush will happen after two seconds. 2. No input for one second. 3. A burst of a few kilobytes of input. 4. No input for one second. 5. Two seconds have passed and flushing starts. The first second counted towards the flush-timeout even though there was no pending data. This can cause flushing to occur more often than needed. xz --flush-timeout=2000, after this commit: 1. xz is started. 2. No input for one second. 3. A burst of a few kilobytes of input. The next flush will happen after two seconds counted from the time when the first bytes of the burst were read. 4. No input for one second. 5. No input for another second. 6. Two seconds have passed and flushing starts.
2020-01-26xz: Move flush_needed from mytime.h to file_pair struct in file_io.h.Lasse Collin1-1/+2
2020-01-26xz: Fix semi-busy-waiting in xz --flush-timeout.Lasse Collin1-4/+11
When input blocked, xz --flush-timeout=1 would wake up every millisecond and initiate flushing which would have nothing to flush and thus would just waste CPU time. The fix disables the timeout when no input has been seen since the previous flush.
2020-01-26xz: Refactor io_read() a bit.Lasse Collin1-9/+8
2020-01-26xz: Move the setting of flush_needed in file_io.c to a nicer location.Lasse Collin1-4/+2
2019-06-24xz: Fix an integer overflow with 32-bit off_t.Lasse Collin1-2/+9
Or any off_t which isn't very big (like signed 64 bit integer that most system have). A small off_t could overflow if the file being decompressed had long enough run of zero bytes, which would result in corrupt output.
2019-06-24xz: Cleanup io_seek_src() a bit.Lasse Collin1-3/+1
lseek() returns -1 on error and checking for -1 is nicer.
2019-06-24xz: Change io_seek_src and io_pread arguments from off_t to uint64_t.Lasse Collin1-4/+12
This helps fixing warnings from -Wsign-conversion and makes the code look better too.
2019-06-23xz: Fix some of the warnings from -Wsign-conversion.Lasse Collin1-2/+3
2017-04-05xz: Add io_seek_src().Lasse Collin1-3/+17
2017-03-30xz: Use POSIX_FADV_RANDOM for in "xz --list" mode.Lasse Collin1-2/+8
xz --list is random access so POSIX_FADV_SEQUENTIAL was clearly wrong.
2016-06-30xz: Fix copying of timestamps on Windows.Lasse Collin1-0/+18
xz used to call utime() on Windows, but its result gets lost on close(). Using _futime() seems to work. Thanks to Martok for reporting the bug: http://www.mail-archive.com/xz-devel@tukaani.org/msg00261.html
2016-06-16xz: Silence warnings from -Wlogical-op.Lasse Collin1-2/+10
Thanks to Evan Nemerson.
2015-11-02xz: Always close the file before trying to delete it.Lasse Collin1-13/+12
unlink() can return EBUSY in errno for open files on some operating systems and file systems.
2015-04-20Revert "xz: Use pipe2() if available."Lasse Collin1-8/+1
This reverts commit 7a11c4a8e5e15f13d5fa59233b3172e65428efdd. It is a problem when libc has pipe2() but the kernel is too old to have pipe2() and thus pipe2() fails. In xz it's pointless to have a fallback for non-functioning pipe2(); it's better to avoid pipe2() completely. Thanks to Michael Fox for the bug report.
2015-04-01xz: Fix the Capsicum rights on user_abort_pipe.Lasse Collin1-1/+5
2015-03-31xz: Add support for sandboxing with Capsicum.Lasse Collin1-0/+81
The sandboxing is used conditionally as described in main.c. This isn't optimal but it was much easier to implement than a full sandboxing solution and it still covers the most common use cases where xz is writing to standard output. This should have practically no effect on performance even with small files as fork() isn't needed. C and locale libraries can open files as needed. This has been fine in the past, but it's a problem with things like Capsicum. io_sandbox_enter() tries to ensure that various locale-related files have been loaded before cap_enter() is called, but it's possible that there are other similar problems which haven't been seen yet. Currently Capsicum is available on FreeBSD 10 and later and there is a port to Linux too. Thanks to Loganaden Velvindron for help.
2015-02-22xz: Use pipe2() if available.Lasse Collin1-1/+8
2015-02-21xz: Fix the fcntl() usage when creating a pipe for the self-pipe trick.Lasse Collin1-5/+11
Now it reads the old flags instead of blindly setting O_NONBLOCK. The old code may have worked correctly, but this is better.
2015-01-09xz: Fix comments.Lasse Collin1-4/+8
2015-01-09xz: Don't fail if stdout doesn't support O_NONBLOCK.Lasse Collin1-21/+15
This is similar to the case with stdin. Thanks to Brad Smith for the bug report and testing on OpenBSD.
2015-01-07xz: Fix a memory leak in DOS-specific code.Lasse Collin1-0/+2
2015-01-07xz: Don't fail if stdin doesn't support O_NONBLOCK.Lasse Collin1-11/+7
It's a problem at least on OpenBSD which doesn't support O_NONBLOCK on e.g. /dev/null. I'm not surprised if it's a problem on other OSes too since this behavior is allowed in POSIX-1.2008. The code relying on this behavior was committed in June 2013 and included in 5.1.3alpha released on 2013-10-26. Clearly the development releases only get limited testing.
2013-09-17xz: Add a missing test for TUKLIB_DOSLIKE.Lasse Collin1-0/+2
2013-07-04xz: Add preliminary support for --flush-timeout=TIMEOUT.Lasse Collin1-10/+36
When --flush-timeout=TIMEOUT is used, xz will use LZMA_SYNC_FLUSH if read() would block and at least TIMEOUT milliseconds has elapsed since the previous flush. This can be useful in realtime-like use cases where the data is simultanously decompressed by another process (possibly on a different computer). If new uncompressed input data is produced slowly, without this option xz could buffer the data for a long time until it would become decompressible from the output. If TIMEOUT is 0, the feature is disabled. This is the default. This commit affects the compression side. Using xz for the decompression side for the above purpose doesn't work yet so well because there is quite a bit of input and output buffering when decompressing. The --long-help or man page were not updated yet. The details of this feature may change.
2013-07-04xz: Don't set src_eof=true after an I/O error because it's useless.Lasse Collin1-3/+0
2013-07-01xz: Silence a warning seen with _FORTIFY_SOURCE=2.Lasse Collin1-1/+7
Thanks to Christian Hesse.
2013-06-29xz: Use non-blocking I/O for the output file.Lasse Collin1-8/+49
Now both reading and writing should be without race conditions with signals. They might still be signal handling issues left. Signals are blocked during many operations to avoid EINTR but it may cause problems e.g. if writing to stderr blocks when trying to display an error message.
2013-06-28xz: Fix return value type in io_write_buf().Lasse Collin1-1/+1
It didn't affect the behavior of the code since -1 becomes true anyway.
2013-06-28xz: Use the self-pipe trick to avoid a race condition with signals.Lasse Collin1-12/+44
It is possible that a signal to set user_abort arrives right before a blocking system call is made. In this case the call may block until another signal arrives, while the wanted behavior is to make xz clean up and exit as soon as possible. After this commit, the race condition is avoided with the input side which already uses non-blocking I/O. The output side still uses blocking I/O and thus has the race condition.
2013-06-28xz: Use non-blocking I/O for the input file.Lasse Collin1-45/+111
2013-06-28xz: Remove an outdated NetBSD-specific comment.Lasse Collin1-4/+0
Nowadays errno == EFTYPE is documented in open(2).
2013-06-28xz: Fix error detection of fcntl(fd, F_SETFL, flags) calls.Lasse Collin1-4/+4
POSIX says that fcntl(fd, F_SETFL, flags) returns -1 on error and "other than -1" on success. This is how it is documented e.g. on OpenBSD too. On Linux, success with F_SETFL is always 0 (at least accorinding to fcntl(2) from man-pages 3.51).
2013-06-28xz: Fix use of wrong variable in a fcntl() call.Lasse Collin1-11/+13
Due to a wrong variable name, when writing a sparse file to standard output, *all* file status flags were cleared (to the extent the operating system allowed it) instead of only clearing the O_APPEND flag. In practice this worked fine in the common situations on GNU/Linux, but I didn't check how it behaved elsewhere. The original flags were still restored correctly. I still changed the code to use a separate boolean variable to indicate when the flags should be restored instead of relying on a special value in stdout_flags.
2013-06-28xz: Fix assertion related to posix_fadvise().Lasse Collin1-8/+2
Input file can be a FIFO or something else that doesn't support posix_fadvise() so don't check the return value even with an assertion. Nothing bad happens if the call to posix_fadvise() fails.
2011-05-01xz: Fix input file position when --single-stream is used.Lasse Collin1-0/+15
Now the following works as you would expect: echo foo | xz > foo.xz echo bar | xz >> foo.xz ( xz -dc --single-stream ; xz -dc --single-stream ) < foo.xz Note that it doesn't work if the input is not seekable or if there is Stream Padding between the concatenated .xz Streams.
2011-04-12Remove doubled words from documentation and comments.Lasse Collin1-1/+1
Spot candidates by running these commands: git ls-files |xargs perl -0777 -n \ -e 'while (/\b(then?|[iao]n|i[fst]|but|f?or|at|and|[dt]o)\s+\1\b/gims)' \ -e '{$n=($` =~ tr/\n/\n/ + 1); ($v=$&)=~s/\n/\\n/g; print "$ARGV:$n:$v\n"}' Thanks to Jim Meyering for the original patch.
2011-04-10xz/DOS: Be more careful with the destination file.Lasse Collin1-2/+33
Try to avoid overwriting the source file if --force is used and the generated destination filename refers to the source file. This can happen with 8.3 filenames where extra characters are ignored. If the generated output file refers to a special file like "con" or "prn", refuse to write to it even if --force is used.
2011-04-09xz: Avoid unneeded fstat() on DOS-like systems.Lasse Collin1-6/+8
2011-04-05xz: Use posix_fadvise() if it is available.Lasse Collin1-0/+15
2011-01-26xz: Fix --force on setuid/setgid/sticky and multi-hardlink files.Lasse Collin1-8/+7
xz didn't compress setuid/setgid/sticky files and files with multiple hard links even with --force. This bug was introduced in 23ac2c44c3ac76994825adb7f9a8f719f78b5ee4. Thanks to Charles Wilson.
2010-09-06xz: Improve a comment.Lasse Collin1-3/+4
2010-09-05xz: Update the comment about NetBSD in file_io.c.Lasse Collin1-4/+4
Thanks to Joerg Sonnenberger.
2010-02-12Collection of language fixes to comments and docs.Lasse Collin1-1/+1
Thanks to Jonathan Nieder.
2010-02-01Fix compression of symlinks with --force.Lasse Collin1-1/+13
xz --force accepted symlinks, but didn't remove them after successful compression. Instead, an error message was displayed.
2010-01-31Delay opening the destionation file and other fixes.Lasse Collin1-55/+52
The opening of the destination file is now delayed a little. The coder is initialized, and if decompressing, the memory usage of the first Block compared against the memory usage limit before the destination file is opened. This means that if --force was used, the old "target" file won't be deleted so easily when something goes wrong very early. Thanks to Mark K for the bug report. The above fix required some changes to progress message handling. Now there is a separate function for setting and printing the filename. It is used also in list.c. list_file() now handles stdin correctly (gives an error). A useless check for user_abort was removed from file_io.c.
2010-01-27Silence two compiler warnings on DOS-like systems.Lasse Collin1-0/+3
2010-01-26Use past tense in error message in io_unlink().Lasse Collin1-2/+12
Added a note to translators too. Thanks to Robert Readman.
2010-01-24Add io_pread().Lasse Collin1-0/+25
It will be used by --list.
2010-01-13Don't compress or decompress special files unless writingLasse Collin1-5/+10
to stdout even if --force is used. --force will still enable compression of symlinks, but only in case they point to a regular file. The new way simply seems more reasonable. It matches gzip's behavior while the old one matched bzip2's behavior.
2009-12-07Fix file_io.c on DOS-like systems.Lasse Collin1-0/+4
The problem was introduced when adding sparse file support in 465d1b0d6518c5d980f2db4c2d769f9905bdd902. Thanks to Charles Wilson.
2009-11-28Remove duplicate code in io_open_dest().Lasse Collin1-8/+1
Fix a missing _() in the error message too.
2009-11-25Create sparse files by default when decompressing intoLasse Collin1-34/+209
a regular file. Sparse file creation can be disabled with --no-sparse. I don't promise yet that the name of this option won't change before 5.0.0. It's possible that the code, that checks when it is safe to use sparse output on stdout, is not good enough, and a more flexible command line option is needed to configure sparse file handling.
2009-09-24Fix an error in OpenVMS-specific code.Lasse Collin1-1/+1
Thanks to Jouk Jansen.
2009-09-22Better fixes for OpenVMS support.Lasse Collin1-3/+16
Thanks to Jouk Jansen.
2009-09-19Various changes.Lasse Collin1-21/+22
Separate a few reusable components from XZ Utils specific code. The reusable code is now in "tuklib" modules. A few more could be separated still, e.g. bswap.h. Fix some bugs in lzmainfo. Fix physmem and cpucores code on OS/2. Thanks to Elbert Pol for help. Add OpenVMS support into physmem. Add a few #ifdefs to ease building XZ Utils on OpenVMS. Thanks to Jouk Jansen for the original patch.
2009-06-27Silence a compiler warning on DOS-like systems.Lasse Collin1-1/+4
2009-06-26Updated comments to match renamed files.Lasse Collin1-1/+1
2009-06-26Rename process.[hc] to coder.[hc] and io.[hc] to file_io.[hc]Lasse Collin1-0/+716
to avoid problems on systems with system headers with those names.