aboutsummaryrefslogtreecommitdiff
path: root/src/xz/xz.1
diff options
context:
space:
mode:
authorLasse Collin <lasse.collin@tukaani.org>2020-02-01 19:56:18 +0200
committerLasse Collin <lasse.collin@tukaani.org>2020-02-05 22:00:28 +0200
commitd0daa21792ff861e5423bbd82aaa6c8ba9fa0462 (patch)
treeb34a271afd4e6483c56607e17badba1f19cd7a8d /src/xz/xz.1
parentxz: Set the --flush-timeout deadline when the first input byte arrives. (diff)
downloadxz-d0daa21792ff861e5423bbd82aaa6c8ba9fa0462.tar.xz
xz: Limit --memlimit-compress to at most 4020 MiB for 32-bit xz.
See the code comment for reasoning. It's far from perfect but hopefully good enough for certain cases while hopefully doing nothing bad in other situations. At presets -5 ... -9, 4020 MiB vs. 4096 MiB makes no difference on how xz scales down the number of threads. The limit has to be a few MiB below 4096 MiB because otherwise things like "xz --lzma2=dict=500MiB" won't scale down the dict size enough and xz cannot allocate enough memory. With "ulimit -v $((4096 * 1024))" on x86-64, the limit in xz had to be no more than 4085 MiB. Some safety margin is good though. This is hack but it should be useful when running 32-bit xz on a 64-bit kernel that gives full 4 GiB address space to xz. Hopefully this is enough to solve this: https://bugzilla.redhat.com/show_bug.cgi?id=1196786 FreeBSD has a patch that limits the result in tuklib_physmem() to SIZE_MAX on 32-bit systems. While I think it's not the way to do it, the results on --memlimit-compress have been good. This commit should achieve practically identical results for compression while leaving decompression and tuklib_physmem() and thus lzma_physmem() unaffected.
Diffstat (limited to 'src/xz/xz.1')
-rw-r--r--src/xz/xz.121
1 files changed, 20 insertions, 1 deletions
diff --git a/src/xz/xz.1 b/src/xz/xz.1
index 6b949640..540d1364 100644
--- a/src/xz/xz.1
+++ b/src/xz/xz.1
@@ -5,7 +5,7 @@
.\" This file has been put into the public domain.
.\" You can do whatever you want with this file.
.\"
-.TH XZ 1 "2019-05-11" "Tukaani" "XZ Utils"
+.TH XZ 1 "2020-02-01" "Tukaani" "XZ Utils"
.
.SH NAME
xz, unxz, xzcat, lzma, unlzma, lzcat \- Compress or decompress .xz and .lzma files
@@ -1005,6 +1005,25 @@ instead of
until the details have been decided.
.RE
.IP ""
+For 32-bit
+.BR xz
+there is a special case: if the
+.I limit
+would be over
+.BR "4020\ MiB" ,
+the
+.I limit
+is set to
+.BR "4020\ MiB" .
+(The values
+.B 0
+and
+.B max
+aren't affected by this.
+A similar feature doesn't exist for decompression.)
+This can be helpful when a 32-bit executable has access
+to 4\ GiB address space while hopefully doing no harm in other situations.
+.IP ""
See also the section
.BR "Memory usage" .
.TP