Missing support for dictionary compression (unstable, non-public API)

[yihuang]

missing binding to the LZ4F_compressFrame_usingCDict.

[jmerdich]

I'm looking into implementing this but hit an upstream roadblock: lz4/lz4#791. The frame dictionary APIs are still considered unstable (despite their age) and are only available with statically linked copies of lz4. Today, python-lz4 supports both but prefers the dynamically-linked system copy of lz4 if available.

We can fail at runtime still if we are using a dynamically linked copy, but how lz4 was built into the package is pretty opaque to the user.

[jonathanunderwood]

Yup - we won't be adding this to the python bindings until the API is stable and public.

[jmerdich]

@jonathanunderwood-- do you think having this as experimental only and forcing static links for experimental builds is a good middle ground? This seems like it only isn't stable because there hasn't been a major lz4 release in the last three years.

[jonathanunderwood]

@jonathanunderwood-- do you think having this as experimental only and forcing static links for experimental builds is a good middle ground? This seems like it only isn't stable because there hasn't been a major lz4 release in the last three years.

I'd consider it if someone submitted a PR and the added complexity in setup.py etc wasn't high and the submitter was prepared to stick around and support it. I have very limited bandwidth to support this package, and this really isn't a high priority compared to the other stuff that is outstanding (fixing/rewriting the frame bindings, Aarch64 builds, type hinting). But honestly, before that, I'd need to see convincing numbers demonstrating it's actually worth it.

[jonathanunderwood]

Note also issue #236 means we'll be making changes to setup.py and how compilers and flags are set up.

[danielhrisca]

Looks like dictionaries are now fully enabled in the LZ4 https://github.com/lz4/lz4/releases/tag/v1.10.0