aboutsummaryrefslogtreecommitdiff
path: root/src/scripts (follow)
AgeCommit message (Collapse)AuthorFilesLines
2024-02-14xzdiff, xzgrep, and xzmore: Rewrite the man pages.Lasse Collin3-116/+173
The main reason is a kind of silly one: xz-man.pot contains strings from all man pages in XZ Utils. The man pages of xzdiff, xzgrep, and xzmore were under GPLv2 and the rest under 0BSD. Thus xz-man.pot contained strings under two licences. po4a creates the translated man pages from the combined 0BSD+GPLv2 xz-man.pot. I haven't liked this mixing in xz-man.pot but the Translation Project requires that all man pages must be in the same .pot file. So a separate xz-man-gpl.pot wasn't an option. Since these man pages are short, rewriting them was quick enough. Now xz-man.pot is entirely under 0BSD and marking the per-file licenses is simpler. As a bonus, some wording hopefully is now slightly better although it's perhaps a matter of taste. NOTE: In xzgrep.1, the EXIT STATUS section was written by me in the commit d796b6d7fdb8b7238b277056cf9146cce25db604 so that's why that section could be taken as is from the old xzgrep.1.
2024-02-14xzless: Update man page slightly.Lasse Collin1-4/+4
The xz tool can decompress three file formats and xzless has always supported uncompressed files too.
2024-02-14Add SPDX license identifiers to GPL, LGPL, and FSFULLR files.Lasse Collin7-6/+7
2024-02-14Add SPDX license identifier into 0BSD source code files.Lasse Collin1-2/+1
2024-02-14Change most public domain parts to 0BSD.Lasse Collin2-6/+1
Translations and doc/xz-file-format.txt and doc/lzma-file-format.txt were not touched. COPYING.0BSD was added.
2024-02-09xzless: Use ||- in LESSOPEN with with "less" 451 and newer.Lasse Collin1-1/+8
2024-02-09xzless: Use --show-preproc-errors with "less" 632 and newer.Lasse Collin1-2/+9
This makes "less" show a warning if a decompression error occurred.
2023-09-24Scripts: Change quoting style from `...' to '...'.Jia Tan2-2/+2
2022-11-11xzdiff: Add support for .lz files.Lasse Collin1-5/+5
The other scripts don't need changes for .lz support because in those scripts it is enough that xz supports .lz.
2022-11-11Scripts: Ignore warnings from xz.Lasse Collin4-7/+8
In practice this means making the scripts work when the input files have an unsupported check type which isn't a problem in practice unless support for some check types has been disabled at build time.
2022-09-16xzgrep: Fix compatibility with old shells.Lasse Collin1-3/+3
Running the current xzgrep on Slackware 10.1 with GNU bash 3.00.15: xzgrep: line 231: syntax error near unexpected token `;;' On SCO OpenServer 5.0.7 with Korn Shell 93r: syntax error at line 231 : `;;' unexpected Turns out that some old shells don't like apostrophes (') inside command substitutions. For example, the following fails: x=$(echo foo # asdf'zxcv echo bar) printf '%s\n' "$x" The problem was introduced by commits 69d1b3fc29677af8ade8dc15dba83f0589cb63d6 (2022-03-29), bd7b290f3fe4faeceb7d3497ed9bf2e6ed5e7dc5 (2022-07-18), and a648978b20495b7aa4a8b029c5a810b5ad9d08ff (2022-07-19). 5.2.6 is the only stable release that included this problem. Thanks to Kevin R. Bulgrien for reporting the problem on SCO OpenServer 5.0.7 and for providing the fix.
2022-07-19xzgrep man page: Document exit statuses.Lasse Collin1-1/+14
2022-07-19xzgrep: Improve error handling, especially signals.Lasse Collin1-19/+53
xzgrep wouldn't exit on SIGPIPE or SIGQUIT when it clearly should have. It's quite possible that it's not perfect still but at least it's much better. If multiple exit statuses compete, now it tries to pick the largest of value. Some comments were added. The exit status handling of signals is still broken if the shell uses values larger than 255 in $? to indicate that a process died due to a signal ***and*** their "exit" command doesn't take this into account. This seems to work well with the ksh and yash versions I tried. However, there is a report in gzip/zgrep that OpenSolaris 5.11 (not 5.10) has a problem with "exit" truncating the argument to 8 bits: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=22900#25 Such a bug would break xzgrep but I didn't add a workaround at least for now. 5.11 is old and I don't know if the problem exists in modern descendants, or if the problem exists in other ksh implementations in use.
2022-07-19xzgrep: Make the fix for ZDI-CAN-16587 more robust.Lasse Collin1-1/+4
I don't know if this can make a difference in the real world but it looked kind of suspicious (what happens with sed implementations that cannot process very long lines?). At least this commit shouldn't make it worse.
2022-07-18xzgrep: Use grep -H --label when available (GNU, *BSDs).Lasse Collin1-0/+21
It avoids the use of sed for prefixing filenames to output lines. Using sed for that is slower and prone to security bugs so now the sed method is only used as a fallback. This also fixes an actual bug: When grepping a binary file, GNU grep nowadays prints its diagnostics to stderr instead of stdout and thus the sed-method for prefixing the filename doesn't work. So with this commit grepping binary files gives reasonable output with GNU grep now. This was inspired by zgrep but the implementation is different.
2022-07-18xzgrep: Use -e to specify the pattern to grep.Lasse Collin1-8/+4
Now we don't need the separate test for adding the -q option as it can be added directly in the two places where it's needed.
2022-07-18Scripts: Use printf instead of echo in a few places.Lasse Collin4-11/+11
It's a good habbit as echo has some portability corner cases when the string contents can be anything.
2022-07-17xzgrep: Add more LC_ALL=C to avoid bugs with multibyte characters.Lasse Collin1-6/+8
Also replace one use of expr with printf. The rationale for LC_ALL=C was already mentioned in 69d1b3fc29677af8ade8dc15dba83f0589cb63d6 that fixed a security issue. However, unrelated uses weren't changed in that commit yet. POSIX says that with sed and such tools one should use LC_ALL=C to ensure predictable behavior when strings contain byte sequences that aren't valid multibyte characters in the current locale. See under "Application usage" in here: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/sed.html With GNU sed invalid multibyte strings would work without this; it's documented in its Texinfo manual. Some other implementations aren't so forgiving.
2022-07-17xzgrep: Fix parsing of certain options.Lasse Collin1-2/+17
Fix handling of "xzgrep -25 foo" (in GNU grep "grep -25 foo" is an alias for "grep -C25 foo"). xzgrep would treat "foo" as filename instead of as a pattern. This bug was fixed in zgrep in gzip in 2012. Add -E, -F, -G, and -P to the "no argument required" list. Add -X to "argument required" list. It is an intentionally-undocumented GNU grep option so this isn't an important option for xzgrep but it seems that other grep implementations (well, those that I checked) don't support -X so I hope this change is an improvement still. grep -d (grep --directories=ACTION) requires an argument. In contrast to zgrep, I kept -d in the "no argument required" list because it's not supported in xzgrep (or zgrep). This way "xzgrep -d" gives an error about option being unsupported instead of telling that it requires an argument. Both zgrep and xzgrep tell that it's unsupported if an argument is specified. Add comments.
2022-03-29xzgrep: Fix escaping of malicious filenames (ZDI-CAN-16587).Lasse Collin1-8/+12
Malicious filenames can make xzgrep to write to arbitrary files or (with a GNU sed extension) lead to arbitrary code execution. xzgrep from XZ Utils versions up to and including 5.2.5 are affected. 5.3.1alpha and 5.3.2alpha are affected as well. This patch works for all of them. This bug was inherited from gzip's zgrep. gzip 1.12 includes a fix for zgrep. The issue with the old sed script is that with multiple newlines, the N-command will read the second line of input, then the s-commands will be skipped because it's not the end of the file yet, then a new sed cycle starts and the pattern space is printed and emptied. So only the last line or two get escaped. One way to fix this would be to read all lines into the pattern space first. However, the included fix is even simpler: All lines except the last line get a backslash appended at the end. To ensure that shell command substitution doesn't eat a possible trailing newline, a colon is appended to the filename before escaping. The colon is later used to separate the filename from the grep output so it is fine to add it here instead of a few lines later. The old code also wasn't POSIX compliant as it used \n in the replacement section of the s-command. Using \<newline> is the POSIX compatible method. LC_ALL=C was added to the two critical sed commands. POSIX sed manual recommends it when using sed to manipulate pathnames because in other locales invalid multibyte sequences might cause issues with some sed implementations. In case of GNU sed, these particular sed scripts wouldn't have such problems but some other scripts could have, see: info '(sed)Locale Considerations' This vulnerability was discovered by: cleemy desu wayo working with Trend Micro Zero Day Initiative Thanks to Jim Meyering and Paul Eggert discussing the different ways to fix this and for coordinating the patch release schedule with gzip.
2021-11-13xzgrep: Update man page timestamp.Lasse Collin1-1/+1
2021-11-13xzgrep: use `grep -E/-F` instead of `egrep` and `fgrep`Ville Skyttä2-6/+6
`egrep` and `fgrep` have been deprecated in GNU grep since 2007, and in current post 3.7 Git they have been made to emit obsolescence warnings: https://git.savannah.gnu.org/cgit/grep.git/commit/?id=a9515624709865d480e3142fd959bccd1c9372d1
2021-06-04xzdiff: Update the man page about the exit status.Lasse Collin1-2/+2
This was forgotten from 194029ffaf74282a81f0c299c07f73caca3232ca.
2021-06-04xzless: Fix less(1) version detection when it contains a dot.Lasse Collin1-1/+1
Sometimes the version number from "less -V" contains a dot, sometimes not. xzless failed detect the version number when it does contain a dot. This fixes it. Thanks to nick87720z for reporting this. Apparently it had been reported here <https://bugs.gentoo.org/489362> in 2013.
2021-01-11Scripts: Add zstd support to xzdiff.Lasse Collin2-7/+15
2021-01-11Scripts: Fix exit status of xzgrep.Lasse Collin1-7/+13
Omit the -q option from xz, gzip, and bzip2. With xz this shouldn't matter. With gzip it's important because -q makes gzip replace SIGPIPE with exit status 2. With bzip2 it's important because with -q bzip2 is completely silent if input is corrupt while other decompressors still give an error message. Avoiding exit status 2 from gzip is important because bzip2 uses exit status 2 to indicate corrupt input. Before this commit xzgrep didn't recognize corrupt .bz2 files because xzgrep was treating exit status 2 as SIGPIPE for gzip compatibility. zstd still needs -q because otherwise it is noisy in normal operation. The code to detect real SIGPIPE didn't check if the exit status was due to a signal (>= 128) and so could ignore some other exit status too.
2021-01-11Scripts: Fix exit status of xzdiff/xzcmp.Lasse Collin1-14/+21
This is a minor fix since this affects only the situation when the files differ and the exit status is something else than 0. In such case there could be SIGPIPE from a decompression tool and that would result in exit status of 2 from xzdiff/xzcmp while the correct behavior would be to return 1 or whatever else diff or cmp may have returned. This commit omits the -q option from xz/gzip/bzip2/lzop arguments. I'm not sure why the -q was used in the first place, perhaps it hides warnings in some situation that I cannot see at the moment. Hopefully the removal won't introduce a new bug. With gzip the -q option was harmful because it made gzip return 2 instead of >= 128 with SIGPIPE. Ignoring exit status 2 (warning from gzip) isn't practical because bzip2 uses exit status 2 to indicate corrupt input file. It's better if SIGPIPE results in exit status >= 128. With bzip2 the removal of -q seems to be good because with -q it prints nothing if input is corrupt. The other tools aren't silent in this situation even with -q. On the other hand, if zstd support is added, it will need -q since otherwise it's noisy in normal situations. Thanks to Étienne Mollier and Sebastian Andrzej Siewior.
2020-12-05Scripts: Add zstd support to xzgrep.Adam Borowski2-3/+7
Thanks to Adam Borowski.
2020-04-06src/scripts/xzgrep.1: Filenames to xzgrep are optional.Lasse Collin1-1/+1
xzgrep --help was correct already.
2020-04-06src/script/xzgrep.1: Remove superfluous '.RB'Bjarni Ingi Gislason1-6/+6
Output is from: test-groff -b -e -mandoc -T utf8 -rF0 -t -w w -z [ "test-groff" is a developmental version of "groff" ] Input file is ./src/scripts/xzgrep.1 <src/scripts/xzgrep.1>:20 (macro RB): only 1 argument, but more are expected <src/scripts/xzgrep.1>:23 (macro RB): only 1 argument, but more are expected <src/scripts/xzgrep.1>:26 (macro RB): only 1 argument, but more are expected <src/scripts/xzgrep.1>:29 (macro RB): only 1 argument, but more are expected <src/scripts/xzgrep.1>:32 (macro RB): only 1 argument, but more are expected "abc..." does not mean the same as "abc ...". The output from nroff and troff is unchanged except for the space between "file" and "...". Signed-off-by: Bjarni Ingi Gislason <bjarniig@rhi.hi.is>
2020-04-06xzgrep.1: Delete superfluous '.PP'Bjarni Ingi Gislason1-1/+0
Summary: mandoc -T lint xzgrep.1 : mandoc: xzgrep.1:79:2: WARNING: skipping paragraph macro: PP empty There is no change in the output of "nroff" and "troff". Signed-off-by: Bjarni Ingi Gislason <bjarniig@rhi.hi.is>
2020-02-07Build: Add support for translated man pages using po4a.Lasse Collin1-15/+49
The dependency on po4a is optional. It's never required to install the translated man pages when xz is built from a release tarball. If po4a is missing when building from xz.git, the translated man pages won't be generated but otherwise the build will work normally. The translations are only updated automatically by autogen.sh and by "make mydist". This makes it easy to keep po4a as an optional dependency and ensures that I won't forget to put updated translations to a release tarball. The translated man pages aren't installed if --disable-nls is used. The installation of translated man pages abuses Automake internals by calling "install-man" with redefined dist_man_MANS and man_MANS. This makes the hairy script code slightly less hairy. If it breaks some day, this code needs to be fixed; don't blame Automake developers. Also, this adds more quotes to the existing shell script code in the Makefile.am "-hook"s.
2019-09-24Scripts: Put /usr/xpg4/bin to the beginning of PATH on Solaris.Lasse Collin4-0/+4
This adds a configure option --enable-path-for-scripts=PREFIX which defaults to empty except on Solaris it is /usr/xpg4/bin to make POSIX grep and others available. The Solaris case had been documented in INSTALL with a manual fix but it's better to do this automatically since it is needed on most Solaris systems anyway. Thanks to Daniel Richard G.
2019-05-11spellingAntoine Cœur1-1/+1
2018-07-27xzless: Rename unused variables to silence static analysers.Lasse Collin1-1/+1
In this particular case I don't see this affecting readability of the code. Thanks to Pavel Raiskup.
2015-02-09xzdiff: Make the mktemp usage compatible with FreeBSD's mktemp.Lasse Collin1-1/+6
Thanks to Rui Paulo for the fix.
2014-11-10xzdiff: Use mkdir if mktemp isn't available.Lasse Collin1-1/+16
2014-11-10xzdiff: Create a temporary directory to hold a temporary file.Lasse Collin1-5/+5
This avoids the possibility of "File name too long" when creating a temp file when the input file name is very long. This also means that other users on the system can no longer see the input file names in /tmp (or whatever $TMPDIR is) since the temporary directory will have a generic name. This usually doesn't matter since on many systems one can see the arguments given to all processes anyway. The number X chars to mktemp where increased from 6 to 10. Note that with some shells temp files or dirs won't be used at all.
2014-10-09xzgrep: Avoid passing both -q and -l to grep.Lasse Collin1-2/+4
The behavior of grep -ql varies: - GNU grep behaves like grep -q. - OpenBSD grep behaves like grep -l. POSIX doesn't make it 100 % clear what behavior is expected. Anyway, using both -q and -l at the same time makes no sense so both options simply should never be used at the same time. Thanks to Christian Weisgerber.
2014-06-11xzgrep: exit 0 when at least one file matches.Lasse Collin1-2/+13
Mimic the original grep behavior and return exit_success when at least one xz compressed file matches given pattern. Original bugreport: https://bugzilla.redhat.com/show_bug.cgi?id=1108085 Thanks to Pavel Raiskup for the patch.
2013-06-30Man pages: Use similar syntax for synopsis as in xz.Lasse Collin1-3/+3
The man pages of lzmainfo, xzmore, and xzdec had similar constructs as the man page of xz had before the commit eb6ca9854b8eb9fbf72497c1cf608d6b19d2d494. Eric S. Raymond didn't mention these man pages in his bug report, but it's nice to be consistent.
2013-04-05xzgrep: make the '-h' option to be --no-filename equivalentJeff Bastian1-1/+1
* src/scripts/xzgrep.in: Accept the '-h' option in argument parsing.
2013-03-05Avoid unneeded use of awk in xzless.Lasse Collin1-2/+1
Use "read" instead of "awk" in xzless to get the version number of "less". The need for awk was introduced in the commit db5c1817fabf7cbb9e4087b1576eb26f0747338e. Thanks to Ariel P for the patch.
2012-11-21xzless: Make "less -V" parsing more robustJonathan Nieder1-1/+2
In v4.999.9beta~30 (xzless: Support compressed standard input, 2009-08-09), xzless learned to parse ‘less -V’ output to figure out whether less is new enough to handle $LESSOPEN settings starting with “|-”. That worked well for a while, but the version string from ‘less’ versions 448 (June, 2012) is misparsed, producing a warning: $ xzless /tmp/test.xz; echo $? /usr/bin/xzless: line 49: test: 456 (GNU regular expressions): \ integer expression expected 0 More precisely, modern ‘less’ lists the regexp implementation along with its version number, and xzless passes the entire version number with attached parenthetical phrase as a number to "test $a -gt $b", producing the above confusing message. $ less-444 -V | head -1 less 444 $ less -V | head -1 less 456 (no regular expressions) So relax the pattern matched --- instead of expecting "less <number>", look for a line of the form "less <number>[ (extra parenthetical)]". While at it, improve the behavior when no matching line is found --- instead of producing a cryptic message, we can fall back on a LESSPIPE setting that is supported by all versions of ‘less’. The implementation uses "awk" for simplicity. Hopefully that’s portable enough. Reported-by: Jörg-Volker Peetz <jvpeetz@web.de> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
2012-02-22Fix exit status of xzgrep when grepping binary files.Lasse Collin1-1/+2
When grepping binary files, grep may exit before it has read all the input. In this case, gzip -q returns 2 (eating SIGPIPE), but xz and bzip2 show SIGPIPE as the exit status (e.g. 141). This causes wrong exit status when grepping xz- or bzip2-compressed binary files. The fix checks for the special exit status that indicates SIGPIPE. It uses kill -l which should be supported everywhere since it is in both SUSv2 (1997) and POSIX.1-2008. Thanks to James Buren for the bug report.
2011-07-31Fix exit status of "xzdiff foo.xz bar.xz".Lasse Collin1-0/+2
xzdiff was clobbering the exit status from diff in a case statement used to analyze the exit statuses from "xz" when its operands were two compressed files. Save and restore diff's exit status to fix this. The bug is inherited from zdiff in GNU gzip and was fixed there on 2009-10-09. Thanks to Jonathan Nieder for the patch and to Peter Pallinger for reporting the bug.
2011-04-18xzgrep: fix typo in $0 parsingMartin Väth1-2/+2
Reported-by: Diego Elio Pettenò <flameeyes@gentoo.org> Signed-off-by: Martin Väth <vaeth@mathematik.uni-wuerzburg.de> Signed-off-by: Mike Frysinger <vapier@gentoo.org>
2011-03-24Scripts: Better fix for xzgrep.Lasse Collin1-2/+6
Now it uses "grep -q". Thanks to Gregory Margo.
2011-03-24Scripts: Fix xzgrep -l.Lasse Collin1-2/+2
It didn't work at all. It tried to use the -q option for grep, but it appended it after "--". This works around it by redirecting to /dev/null. The downside is that this can be slower with big files compared to proper use of "grep -q". Thanks to Gregory Margo.
2011-03-19Scripts: Add lzop (.lzo) support to xzdiff and xzgrep.Lasse Collin4-16/+28
2010-12-13Scripts: Fix gzip and bzip2 support in xzdiff.Lasse Collin1-6/+6
2010-10-08Build: Add options to disable individual command line tools.Lasse Collin1-36/+26
2010-09-27Major man page updates.Lasse Collin4-21/+27
Lots of content was updated on the xz man page. Technical improvements: - Start a new sentence on a new line. - Use fairly short lines. - Use constant-width font for examples (where supported). - Some minor cleanups. Thanks to Jonathan Nieder for some language fixes.
2010-03-07Fix xzgrep to not break if filenames have spaces or quotes.Lasse Collin1-1/+1
Thanks to someone who reported the bug on IRC.
2010-02-12Collection of language fixes to comments and docs.Lasse Collin2-6/+6
Thanks to Jonathan Nieder.
2009-10-16Escape dashes in xzmore.1Jonathan Nieder1-2/+2
A minus sign is larger, easier to see in a printout, and more likely to use the same glyph as ASCII hyphen-minus in a terminal than a hyphen. Since broken manual pagers do not find hyphens when the user searches for a hyphen-minus, minus signs are also easier to search for. So use minus signs instead of hyphens to render sample terminal output.
2009-08-09“xzdiff a.xz b.xz” always failsJonathan Nieder1-1/+1
Attempts to compare two compressed files result in no output and exit status 2. Instead of going to standard output, ‘diff’ output is being captured in the xz_status variable along with the exit status from the decompression commands. Later, when this variable is examined for nonzero status codes, numerals from dates in the ‘diff’ output make it appear as though decompression failed. So let the ‘diff’ output leak to standard output with another file descriptor. (This trick is used in all similar contexts elsewhere in xzdiff and in the analogous context in gzip’s zdiff script.)
2009-08-09xzless: Support compressed standard inputJonathan Nieder1-1/+7
It can be somewhat confusing that less < some_file.txt works fine, whereas xzless < some_file.txt.xz does not. Since version 429, ‘less’ allows a filter specified in the LESSOPEN environment variable to preprocess its input even if it comes from standard input, if $LESSOPEN begins with ‘|-’. So set $LESSOPEN to take advantage of this feature. Check less’s version at runtime so xzless can continue to work with older versions.
2009-07-30xzdiff: add missing ;; to case statementJonathan Nieder1-3/+3
2009-07-19Added missing author notice to xzless.in.Lasse Collin1-0/+1
2009-07-06Use sed instead of $(SED) so that we don't need toLasse Collin1-14/+14
use AC_PROG_SED. We don't do anything fancy with sed, so this should work OK. libtool 2.2 sets SED but 1.5 doesn't, so $(SED) happened to work when using libtool 2.2.
2009-07-05Major update to the xzgrep and other scripts based onLasse Collin12-369/+689
the latest versions found from gzip CVS repository. configure will try to find a POSIX shell to be used by the scripts. This should ease portability on systems which have pre-POSIX /bin/sh. xzgrep and xzdiff support .xz, .lzma, .gz, and .bz2 files. xzmore and xzless support only .xz and .lzma files. The name of the xz executable used in these scripts is now correct even if --program-transform-name has been used.
2009-06-27Create correct symlinks even whenLasse Collin1-34/+46
--program-{prefix,suffix,transform} is passed to configure.
2009-05-21Install lzdiff, lzgrep, and lzmore as symlinksABCD1-2/+14
This adds lzdiff, lzgrep, and lzmore to the list of symlinks to install. It also installs symlinks for the manual pages and removes the new symlinks on uninstall.
2009-04-13Quick & dirty update to support xz in diff/grep/more scripts.Lasse Collin7-75/+119
2009-04-13Put the interesting parts of XZ Utils into the public domain.Lasse Collin1-0/+7
Some minor documentation cleanups were made at the same time.
2009-01-26remove trailing blanks from all but .xz filesJim Meyering2-5/+5
2007-12-09Imported to git.Lasse Collin7-0/+455