| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
Translations and doc/xz-file-format.txt and doc/lzma-file-format.txt
were not touched.
COPYING.0BSD was added.
|
|
|
|
The new Filter ID is 0x0B.
Thanks to Chien Wong <m@xv97.com> for the initial version of the Filter,
the xz CLI updates, and the Autotools build system modifications.
Thanks to Igor Pavlov for his many contributions to the design of
the filter.
|
|
|
|
MinGW (formely a MinGW.org Project, later the MinGW.OSDN Project
at <https://osdn.net/projects/mingw/>) has GCC 9.2.0 as the
most recent GCC package (released 2021-02-02). The project might
still be alive but majority of people have switched to MinGW-w64.
Thus it seems clearer to refer to MinGW-w64 in our API headers too.
Building with MinGW is likely to still work but I haven't tested it
in the recent years.
|
|
This was done for both internal and API headers.
|
|
|
|
This further improves the documentation from commit
f36ca7982f6bd5e9827219ed4f3c5a1fbf5d7bdf. The previous wording of
"supported options" was slightly misleading since the options that are
printed are the ones that are relevant for encoding/decoding. It is not
about which options can or must be specified.
|
|
Reword "options required" to "supported options". The previous may have
suggested that the options listed were all required anytime a filter is
used for encoding or decoding. The reword makes this more clear that
adjusting the options is optional.
|
|
None of the liblzma functions may throw an exception, so this
attribute should be applied to all liblzma API functions.
|
|
The lzma_mt_block_size() was previously just an internal function for
the multithreaded .xz encoder. It is used to provide a recommended Block
size for a given filter chain.
This function is helpful to determine the maximum Block size for the
multithreaded .xz encoder when one wants to change the filters between
blocks. Then, this determined Block size can be provided to
lzma_stream_encoder_mt() in the lzma_mt options parameter when
intializing the coder. This requires one to know all the filter chains
they are using before starting to encode (or at least the filter chain
that will need the largest Block size), but that isn't a bad limitation.
|
|
AUTHORS was updated earlier, lzma.h was simply forgotten.
|
|
The xz man page timestamp was intentionally left unchanged.
|
|
This was left in by mistake since an early version of the ARM64 filter
used a different struct for its options.
|
|
The \mainpage command is used in the first block of comments in lzma.h.
This changes the previously nearly empty index.html to use the first
comment block in lzma.h for its contents.
lzma.h is no longer documented separately, but this is for the better
since lzma.h only defined a few macros that users do not need to use.
The individual API header files all have a disclaimer that they should
not be #included directly, so there should be no confusion on the fact
that lzma.h should be the only header used by applications.
Additionally, the note "See ../lzma.h for information about liblzma as
a whole." was removed since lzma.h is now the main page of the
generated HTML and does not have its own page anymore. So it would be
confusing in the HTML version and was only a "nice to have" when
browsing the source files.
|
|
|
|
lzma_lzma_preset() does not guarentee that the lzma_options_lzma are
usable in an encoder even if it returns false (success). If liblzma
is built with default configurations, then the options will always be
usable. However if the match finders hc3, hc4, or bt4 are disabled, then
the options may not be usable depending on the preset level requested.
The documentation was updated to reflect this complexity, since this
behavior was unclear before.
|
|
The '\n' renders as a newline when the comments are converted to html
by Doxygen.
|
|
Shorten the description for lzma_raw_encoder_memusage() and
lzma_raw_decoder_memusage().
|
|
|
|
All functions now explicitly specify parameter and return values.
The notes and code annotations were moved before the parameter and
return value descriptions for consistency.
Also, the description above lzma_filter_encoder_is_supported() about
not being able to list available filters was removed since
lzma_str_list_filters() will do this.
|
|
|
|
|
|
|
|
lzma_microlzma_decoder -> lzma_microlzma_encoder
|
|
Standardizing each function to always specify parameters and return
values. Also moved the parameters and return values to the end of each
function description.
|
|
Use "member" to refer to struct members as that's the term used
by the C standard.
Use lzma_options_delta.dist and such in docs so that in Doxygen's
HTML output they will link to the doc of the struct member.
Clean up a few trailing white spaces too.
|
|
|
|
|
|
Also adjusted preset value => preset level.
|
|
Standardizing each function to always specify parameters and return
values. Also moved the parameters and return values to the end of each
function description.
A few small things were reworded and long sentences broken up.
|
|
All functions now explicitly specify parameter and return values.
|
|
All functions now explicitly specify parameter and return values.
Also moved the note about SHA-256 functions not being exported to the
top of the file.
|
|
All functions now explicitly specify parameter and return values.
|
|
|
|
Add \private above this field and its sub-fields since it is not meant
to be modified by users.
|
|
LZMA_MEMLIMIT_ERROR was missing the "<" character needed to put
documentation after a member.
|
|
Standardizing each function to always specify params and return values.
Also fixed a small grammar mistake.
|
|
|
|
Added [out] annotations to parameters that are pointers and can have
their value changed. Also added a clarification to lzma_vli_is_valid.
|
|
Document LZMA_DELTA_DIST_MIN and LZMA_DELTA_DIST_MAX for completeness
and to avoid Doxygen warnings.
|
|
All functions now explicitly specify parameter and return values.
Also reworded the description of lzma_index_hash_init() for readability.
|
|
Specified parameter and return values for API functions and documented
a few more of the macros.
|
|
Added a few sentences to the description for lzma_block_encoder() and
lzma_block_decoder() to highlight that the Block Header must be coded
before calling these functions.
|
|
|
|
|
|
|
|
|
|
Standardizing each function to always specify params and return values.
Output pointer parameters are also marked with doxygen style [out] to
make it clear. Any note sections were also moved above the parameter and
return sections for consistency.
|
|
The flag description for LZMA_STR_NO_VALIDATION was previously confusing
about the treatment for filters than cannot be used with .xz format
(lzma1) without using LZMA_STR_ALL_FILTERS. Now, it is clear that
LZMA_STR_NO_VALIDATION is not a super set of LZMA_STR_ALL_FILTERS.
|
|
The previous documentation for lzma_str_to_filters() was technically
correct, but misleading. lzma_str_to_filters() returns NULL on success,
which is in practice always defined to 0. This is the same value as
LZMA_OK, but lzma_str_to_filters() does not return lzma_ret so we should
be more clear.
|
|
This prevents the reserved fields from being part of the generated
Doxygen documentation.
|
|
This improves the generated Doxygen HTML files to better highlight
how to properly use the liblzma API header files.
|
|
5.5.0alpha won't be released, it's just to mark that
the branch is not for stable 5.4.x.
Once again there is no API/ABI stability for new features
in devel versions. The major soname won't be bumped even
if API/ABI of new features breaks between devel releases.
|
|
|
|
|
|
This also sorts the symbol names alphabetically in liblzma_*.map.
|
|
It has some complicated downsides and its usefulness is more limited
than I originally thought. So this change is bad for certain very
specific situations but a generic solution that works for other
filters (and is otherwise better too) is planned anyway. And this
way 7-Zip can use the same compatible filter for the .7z format.
This is still marked as experimental with a new temporary Filter ID.
|
|
|
|
|
|
Thanks to Jia Tan.
|
|
lzma_str_to_filters() uses static error messages which makes
them not very precise. It tells the position in the string
where an error occurred though which helps quite a bit if
applications take advantage of it. Dynamic error messages can
be added later with a new flag if it seems important enough.
|
|
Some file formats need support for LZMA1 streams that don't use
the end of payload marker (EOPM) alias end of stream (EOS) marker.
So far liblzma API has supported decompressing such streams via
lzma_alone_decoder() when .lzma header specifies a known
uncompressed size. Encoding support hasn't been available in the API.
Instead of adding a new LZMA1-only API for this purpose, this commit
adds a new filter ID for use with raw encoder and decoder. The main
benefit of this approach is that then also filter chains are possible,
for example, if someone wants to implement support for .7z files that
use the x86 BCJ filter with LZMA1 (not BCJ2 as that isn't supported
in liblzma).
|
|
A tiny downside of this is that now a 1-4 tiny allocations are made
for every Block because each worker thread needs its own copy of
the filter chain.
|
|
This is small but convenient and should have been added
a long time ago.
|
|
|
|
These were caught by clang -Wdocumentation.
|
|
|
|
This is incompatible with the previous version.
This has space/tab fixes in filter_*.c and bcj.h too.
|
|
Thanks to Michał Górny for the original patch.
|
|
Support for format version 0 was removed from lzip 1.18 for some
reason. .lz format version 0 files are rare (and old) but some
source packages were released in this format, and some people might
have personal files in this format too. It's very little extra code
to support it along side format version 1 so this commits adds
support for both.
The Sync Flush marker extentension to the original .lz format
version 1 isn't supported. It would require changes to the
LZMA decoder itself. Such files are very rare anyway.
See the API doc for lzma_lzip_decoder() for more details about
the .lz format support.
Thanks to Michał Górny for the original patch.
|
|
|
|
The bug was fixed in 660739f99ab211edec4071de98889fb32ed04e98.
|
|
Thanks to Jia Tan.
|
|
That is, the Filter ID will be changed once the design is final.
The current version will be removed. So files created with the
tempoary Filter ID won't be supported in the future.
|
|
lzma_stream_encoder() and lzma_stream_encoder_mt() always assumed
this. Before this patch, failing lzma_filters_copy() could result
in free(invalid_pointer) or invalid memory reads in stream_encoder.c
or stream_encoder_mt.c.
To trigger this, allocating memory for a filter options structure
has to fail. These are tiny allocations so in practice they very
rarely fail.
Certain badness in the filter chain array could also make
lzma_filters_copy() fail but both stream_encoder.c and
stream_encoder_mt.c validate the filter chain before
trying to copy it, so the crash cannot occur this way.
|
|
|
|
It will now return LZMA_DATA_ERROR (not LZMA_OK or LZMA_BUF_ERROR)
if LZMA_FINISH is used and there isn't enough input to finish
decoding the Block Header or the Block. The use of LZMA_DATA_ERROR
is simpler and the less risky than LZMA_BUF_ERROR but this might
be changed before 5.4.0.
|
|
In most cases if the input file is corrupt the application won't
care about the uncompressed content at all. With this new flag
the threaded decoder will return an error as soon as any thread
has detected an error; it won't wait to copy out the data before
the location of the error.
I don't plan to use this in xz to keep the behavior consistent
between single-threaded and multi-threaded modes.
|
|
I realize that this is about a decade late.
Big thanks to Sebastian Andrzej Siewior for the original patch.
I made a bunch of smaller changes but after a while quite a few
things got rewritten. So any bugs in the commit were created by me.
|
|
If Check is unsupported, it will be silently ignored.
It's the caller's job to handle it.
|
|
Thanks to Jia Tan.
|
|
|
|
By default, MSVC always sets __cplusplus to 199711L. The real
C++ standard version is available in _MSVC_LANG (or one could
use /Zc:__cplusplus to set __cplusplus correctly).
Fixes <https://sourceforge.net/p/lzmautils/discussion/708858/thread/f6bc3b108a/>.
Thanks to Dan Weiss.
|
|
It still exists primarily for EROFS but MicroLZMA is
a more generic name (that hopefully doesn't clash with
something that already exists).
|
|
The caller must still not specify an uncompressed size bigger
than the actual uncompressed size.
As a downside, this now needs the exact compressed size.
|
|
Right now this is just a planned extra-compact format for use
in the EROFS file system in Linux. At this point it's possible
that the format will either change or be abandoned and removed
completely.
The special thing about the encoder is that it uses the
output-size-limited encoding added in the previous commit.
EROFS uses fixed-sized blocks (e.g. 4 KiB) to hold compressed
data so the compressors must be able to create valid streams
that fill the given block size.
|
|
https://fossies.org/linux/misc/xz-5.2.5.tar.xz/codespell.html
|
|
This should silence the equivalent of -Wundef in compilers that
don't define __GNUC__.
|
|
LZMA_TIMED_OUT is *internally* used as a value for lzma_ret
enumeration. Previously it was #defined to 32 and cast to lzma_ret.
That way it wasn't visible in the public API, but this was hackish.
Now the public API has eight LZMA_RET_INTERNALx members and
LZMA_TIMED_OUT is #defined to LZMA_RET_INTERNAL1. This way
the code is cleaner overall although the public API has a few
extra mysterious enum members.
|
|
|
|
|
|
|
|
In C++11, the `throw()` specifier is deprecated and `noexcept` is
preffered instead.
|
|
|
|
|
|
Also mention LZMA_SEEK in xz/message.c to silence a warning.
|
|
The 0 got treated specially in a buggy way and as a result
the function did nothing. The API doc said that 0 was supposed
to return LZMA_PROG_ERROR but it didn't.
Now 0 is treated as if 1 had been specified. This is done because
0 is already used to indicate an error from lzma_memlimit_get()
and lzma_memusage().
In addition, lzma_memlimit_set() no longer checks that the new
limit is at least LZMA_MEMUSAGE_BASE. It's counter-productive
for the Index decoder and was actually needed only by the
auto decoder. Auto decoder has now been modified to check for
LZMA_MEMUSAGE_BASE.
|
|
|
|
It returned LZMA_PROG_ERROR, which was done to avoid zero as
the limit (because it's a special value elsewhere), but using
LZMA_PROG_ERROR is simply inconvenient and can cause bugs.
The fix/workaround is to treat 0 as if it were 1 byte. It's
effectively the same thing. The only weird consequence is
that then lzma_memlimit_get() will return 1 even when 0 was
specified as the limit.
This fixes a very rare corner case in xz --list where a specific
memory usage limit and a multi-stream file could print the
error message "Internal error (bug)" instead of saying that
the memory usage limit is too low.
|
|
|
|
The idea of 99 is that it looks a bit weird in this context.
For new features there's no API/ABI stability in devel versions.
|
|
|
|
I know that soname != app version, but I skip AGE=1
in -version-info to make the soname match the liblzma
version anyway. It doesn't hurt anything as long as
it doesn't conflict with library versioning rules.
|
|
|
|
|
|
|
|
|
|
|
|
Note that this slightly changes how lzma_block_header_decode()
has been documented. Earlier it said that the .version is set
to the lowest required value, but now it says that the .version
field is kept unchanged if possible. In practice this doesn't
affect any old code, because before this commit the only
possible .version was 0.
|
|
|
|
It can be confusing that two header files have the same name.
The public API file is still lzma.h.
|
|
|
|
Thanks to Tomer Chachamu.
|
|
|
|
In the single-threaded encoder LZMA_FULL_BARRIER is simply
an alias for LZMA_FULL_FLUSH.
|
|
This also adds a new internal function
lzma_block_buffer_bound64() which is similar to
lzma_block_buffer_bound() but uses uint64_t instead
of size_t.
|
|
This adds lzma_get_progress() to liblzma and takes advantage
of it in xz.
lzma_get_progress() collects progress information from
the thread-specific structures so that fairly accurate
progress information is available to applications. Adding
a new function seemed to be a better way than making the
information directly available in lzma_stream (like total_in
and total_out are) because collecting the information requires
locking mutexes. It's waste of time to do it more often than
the up to date information is actually needed by an application.
|
|
There is a tiny risk of causing breakage: If an application
assigns lzma_stream.allocator to a non-const pointer, such
code won't compile anymore. I don't know why anyone would do
such a thing though, so in practice this shouldn't cause trouble.
Thanks to Jan Kratochvil for the patch.
|
|
|
|
|
|
This way people hopefully won't complain if these APIs
change and break code that used an older API.
|
|
This is the simplest method to do threading, which splits
the uncompressed data into blocks and compresses them
independently from each other. There's room for improvement
especially to reduce the memory usage, but nevertheless,
this is a good start.
|
|
|
|
It didn't mention the return value that is used if
an error occurs.
|
|
|
|
This return value was missing from the API comments of
four functions.
|
|
|
|
|
|
|
|
|
|
Adding support for LZMA_FINISH for Index encoding and
decoding needed tiny additions to the relevant .c files too.
|
|
|
|
|
|
The spec isn't finished and the code didn't compile anymore.
It won't be included in XZ Utils 5.0.0. It's easy to get it
back once the spec is done.
|
|
It isn't really useful so omitting it makes things
shorter and slightly more readable.
|
|
Thanks to Jonathan Nieder.
|
|
lzma_block.version has to be initialized even for
lzma_block_header_decode(). This way a future version
of liblzma won't allocate memory in a way that an old
application doesn't know how to free it.
The subtlety of this change is that all current apps
using lzma_block_header_decode() will keep working for
now, because the only possible version value is zero,
and lzma_block_header_decode() unconditionally sets the
version to zero even now. Unless fixed, these apps will
break in the future if a new version of the Block options
is ever needed.
|
|
This affects lzma_memusage() and lzma_memlimit_get().
|
|
This breaks API and ABI but most apps are not affected
since most apps don't use this part of the API. You will
get a compile error if you are using anything that got
broken.
Summary of changes:
- Ability to store Stream Flags, which are needed
for random-access reading in multi-Stream files.
- Separate function to set size of Stream Padding.
- Iterator structure makes it possible to read the same
lzma_index from multiple threads at the same time.
- A lot faster code to locate Blocks.
- Removed lzma_index_equal() without adding anything
to replace it. I don't know what it should do exactly
with the new features and what actually needs this
function in the first place other than test_index.c,
which now has its own code to compare lzma_indexes.
|
|
|
|
I understood that this is nicer, because then people
don't need to worry about the LZMA_API_STATIC macro.
Thanks to Charles Wilson and Keith Marshall.
|
|
The Index decoder code didn't perfectly match the API docs,
which said that *i will be set to point to the decoded Index
only after decoding has succeeded. The docs were a bit unclear
too.
Now the decoder will initially set *i to NULL. *i will be set
to point to the decoded Index once decoding has succeeded.
This simplifies applications too, since it avoids dangling
pointers.
|
|
I had hoped to keep liblzma as purely a compression
library as possible (e.g. file I/O will go into
a different library), but it seems that applications
linking agaisnt liblzma need some way to determine
the memory usage limit, and knowing the amount of RAM
is one reasonable way to help making such decisions.
Thanks to Jonathan Nieder for the original patch.
|
|
Originally the idea was that using LZMA_FULL_FLUSH
with Stream encoder would read the filter chain
from the same array that was used to intialize the
Stream encoder. Since most apps wouldn't use
LZMA_FULL_FLUSH, most apps wouldn't need to keep
the filter chain available after initializing the
Stream encoder. However, due to my mistake, it
actually required keeping the array always available.
Since setting the new filter chain via the array
used at initialization time is not a nice way to do
it for a couple of reasons, this commit ditches it
and introduces lzma_filters_update(). This new function
replaces also the "persistent" flag used by LZMA2
(and to-be-designed Subblock filter), which was also
an ugly thing to do.
Thanks to Alexey Tourbin for reminding me about the problem
that Stream encoder used to require keeping the filter
chain allocated.
|
|
It was meant to be lzma_filters_copy(), not lzma_filters_dup().
|
|
This will be needed internally by liblzma once I fix
a design mistake in the encoder API. This function may
be useful to applications too so it's good to export it.
|
|
|
|
|
|
Thanks to Christian Weisgerber for pointing out some of these.
|
|
|
|
Added lzma_nothrow for every function. It adds
throw() when the header is used in C++ code.
Some lzma_attrs were added or removed.
Lots of comments were improved.
|
|
the Autotools based build system. It's not good yet, more
fixes will follow.
|
|
via lzma_block structure.
This changes ABI but not doesn't break API.
|
|
Some minor documentation cleanups were made at the same time.
|
|
|
|
Now configure.ac will get the version number directly from
src/liblzma/api/lzma/version.h. The intent is to reduce the
number of places where the version number is duplicated. In
future, support for displaying Git commit ID may be added too.
|
|
need to #define when linking against static liblzma on
platforms like Windows. Most developers don't need to
care about LZMA_API_STATIC at all.
|
|
other compilers than MinGW. This may hurt readability
of the API headers slightly, but I don't know any
better way to do this.
|
|
functions, and cleaned up filter.h API header a little.
May be very buggy, not tested yet.
|
|
on Windows.
|
|
on Windows. sysdefs.h no longer #includes lzma.h, so lzma.h
has to be #included separately where needed.
|
|
beginning of the line.
|
|
stdint.h or inttypes.h.
|
|
|
|
|
|
|
|
|
|
lzma_memlimit_encoder and lzma_memlimit_decoder to
lzma_raw_encoder_memlimit and lzma_raw_decoder_memlimit. :-(
Now it is fixed. Hopefully it doesn't cause too much trouble
to those who already thought API is stable.
|
|
of 4.999.7beta.
|
|
|
|
|
|
The internal implementation is still using the name "simple".
It may need some cleanups, so I look at it later.
|
|
Half of developers were already forgetting to use these
functions, which could have caused total breakage in some future
liblzma version or even now if --enable-small was used. Now
liblzma uses pthread_once() to do the initializations unless
it has been built with --disable-threads which make these
initializations thread-unsafe.
When --enable-small isn't used, liblzma currently gets needlessly
linked against libpthread (on systems that have it). While it is
stupid for now, liblzma will need threads in future anyway, so
this stupidity will be temporary only.
When --enable-small is used, different code CRC32 and CRC64 is
now used than without --enable-small. This made the resulting
binary slightly smaller, but the main reason was to clean it up
and to handle the lack of lzma_init_check().
The pkg-config file lzma.pc was renamed to liblzma.pc. I'm not
sure if it works correctly and portably for static linking
(Libs.private includes -pthread or other operating system
specific flags). Hopefully someone complains if it is bad.
lzma_rc_prices[] is now included as a precomputed array even
with --enable-small. It's just 128 bytes now that it uses uint8_t
instead of uint32_t. Smaller array seemed to be at least as fast
as the more bloated uint32_t array on x86; hopefully it's not bad
on other architectures.
|
|
The API and ABI should now be very close to stable,
although the code behind it isn't yet.
|
|
be added back in some form later, but the current version
wasn't modular, so it would need fixing anyway.
|
|
|
|
|
|
- Updated to the latest, probably final file format version.
- Command line tool reworked to not use threads anymore.
Threading will probably go into liblzma anyway.
- Memory usage limit is now about 30 % for uncompression
and about 90 % for compression.
- Progress indicator with --verbose
- Simplified --help and full --long-help
- Upgraded to the last LGPLv2.1+ getopt_long from gnulib.
- Some bug fixes
|
|
|
|
|
|
|
|
- LZMA_VLI_VALUE_MAX -> LZMA_VLI_MAX
- LZMA_VLI_VALUE_UNKNOWN -> LZMA_VLI_UNKNOWN
- LZMA_HEADER_ERRRO -> LZMA_OPTIONS_ERROR
|
|
|
|
|
|
|
|
code from block_private.h to block_decoder.c. Now the Block
encoder doesn't need compressed_size and uncompressed_size
from lzma_block structure to be initialized.
|
|
|
|
|
|
|
|
broken. API has changed a lot and it will still change a
little more here and there. The command line tool doesn't
have all the required changes to reflect the API changes, so
it's easy to get "internal error" or trigger assertions.
|
|
literal coder as part of the main LZMA encoder or
decoder structure.
Make the LZMA decoder to rely on the current internal API
to free the allocated memory in case an error occurs.
|
|
specification. Simplify things by removing most of the
support for known uncompressed size in most places.
There are some miscellaneous changes here and there too.
The API of liblzma has got many changes and still some
more will be done soon. While most of the code has been
updated, some things are not fixed (the command line tool
will choke with invalid filter chain, if nothing else).
Subblock filter is somewhat broken for now. It will be
updated once the encoded format of the Subblock filter
has been decided.
|
|
|
