diff options
| author | Lasse Collin <lasse.collin@tukaani.org> | 2020-02-01 19:56:18 +0200 |
|---|---|---|
| committer | Lasse Collin <lasse.collin@tukaani.org> | 2020-02-05 22:00:28 +0200 |
| commit | d0daa21792ff861e5423bbd82aaa6c8ba9fa0462 (patch) | |
| tree | b34a271afd4e6483c56607e17badba1f19cd7a8d /src/xz/xz.1 | |
| parent | xz: Set the --flush-timeout deadline when the first input byte arrives. (diff) | |
| download | xz-d0daa21792ff861e5423bbd82aaa6c8ba9fa0462.tar.xz | |
xz: Limit --memlimit-compress to at most 4020 MiB for 32-bit xz.
See the code comment for reasoning. It's far from perfect but
hopefully good enough for certain cases while hopefully doing
nothing bad in other situations.
At presets -5 ... -9, 4020 MiB vs. 4096 MiB makes no difference
on how xz scales down the number of threads.
The limit has to be a few MiB below 4096 MiB because otherwise
things like "xz --lzma2=dict=500MiB" won't scale down the dict
size enough and xz cannot allocate enough memory. With
"ulimit -v $((4096 * 1024))" on x86-64, the limit in xz had
to be no more than 4085 MiB. Some safety margin is good though.
This is hack but it should be useful when running 32-bit xz on
a 64-bit kernel that gives full 4 GiB address space to xz.
Hopefully this is enough to solve this:
https://bugzilla.redhat.com/show_bug.cgi?id=1196786
FreeBSD has a patch that limits the result in tuklib_physmem()
to SIZE_MAX on 32-bit systems. While I think it's not the way
to do it, the results on --memlimit-compress have been good. This
commit should achieve practically identical results for compression
while leaving decompression and tuklib_physmem() and thus
lzma_physmem() unaffected.
Diffstat (limited to 'src/xz/xz.1')
| -rw-r--r-- | src/xz/xz.1 | 21 |
1 files changed, 20 insertions, 1 deletions
diff --git a/src/xz/xz.1 b/src/xz/xz.1 index 6b949640..540d1364 100644 --- a/src/xz/xz.1 +++ b/src/xz/xz.1 @@ -5,7 +5,7 @@ .\" This file has been put into the public domain. .\" You can do whatever you want with this file. .\" -.TH XZ 1 "2019-05-11" "Tukaani" "XZ Utils" +.TH XZ 1 "2020-02-01" "Tukaani" "XZ Utils" . .SH NAME xz, unxz, xzcat, lzma, unlzma, lzcat \- Compress or decompress .xz and .lzma files @@ -1005,6 +1005,25 @@ instead of until the details have been decided. .RE .IP "" +For 32-bit +.BR xz +there is a special case: if the +.I limit +would be over +.BR "4020\ MiB" , +the +.I limit +is set to +.BR "4020\ MiB" . +(The values +.B 0 +and +.B max +aren't affected by this. +A similar feature doesn't exist for decompression.) +This can be helpful when a 32-bit executable has access +to 4\ GiB address space while hopefully doing no harm in other situations. +.IP "" See also the section .BR "Memory usage" . .TP |