-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Function signatures #14
Comments
Yeah, had the same thoughts. This whole thing started as a hack since I couldn't find what I needed in other libraries. But it if we choose to change it, better sooner than later. About encoding to I chose to make DeltaBin public since I thought that it may be useful to someone to use the binary encoding (with all the code generation) without the block encoding part that is specific to this library. Does that make sense? |
Publishing makes sense. You definitely want testers and second opinions on something like this. The v1 tag is unfortunate indeed. You could switch the namespace to Just remove the tags first! It will not affect the content on pkg.go.dev. There's a migrate/rename option in GitHub, which preserves the history, and it redirects the current location(s). Flip the go.mod declaration and good to go? 🙃 Bit handling in Java is a disaster, partially due to big-endian on SPARC, and mostly because optimization never made it as a priority there. 🤭 Go switched to native "word moves" a few years ago. I am not sure how ARM64 deals with memory alignment, now I come to think of it. Needs some testing indeed. I would love it if you drop copyrights and the Apache library in favour of CC0. In that way I can reuse the code in my code generator. No hard feelings if you don't. |
Yup, the alignment holds us back, especially since binary append can only handle one word at a time. I'd go for |
The output should always apply 64-bit words, regardless of the input width. This is too much of an overhaul perhaps. We got something here that works. Perhaps I can try an alternative version, without copyrights, and share back with you that way? |
Thanks for the benchmarks. Conversely, what are the drawbacks of uint slice for compressed data, appart the aesthetics aspect? |
Simply produce and error on unaligned input. This should never happen as we read and consume full words only. A warning in the function documentation should be enough. if unsafe.Pointer(&in[0]) & 7 != 0 {
return errors.New("incomp: unaligned input denied")
} How'd you store or read a []uint64? 😉 |
Fine for checking that input is aligned. |
New allocs are always aligned. Just won't work when people continue with custom slices. |
Maybe uint64 slices are not so bad after all, with Go's alignment guarantees in place. https://pkg.go.dev/github.com/pascaldekloe/[email protected]/pack64#Write Only problem is making that header fit without too much waste. |
I just came across this nice library and was wondering what's the best way to write the compressed slice to a byte buffer. |
binary package provides a way to serialize primitive types to a byte slice. If you don't need portability converting slices using unsafe is faster. If you like exploring stuff, I have created another repo ronanh/compexperiments which takes a different approach and uses same type of compression. It's mostly useful for in-memory compressed slices which can be read and modified (append) without decompressing the whole buffer. |
Encoding as uint32 or uint64 slices is a bit cumbersome. We can use byte slices instead without any loss of performance. Go even optimizes the binary.ByteOrder into native operations.
Decoding needs an error return. The Uncompress functions are panic bombs now. This is bad for performance, e.g, pull request #13.
The output destination should always be the first argument, like a C
copy
orfprintf
, and like a Goio.Copy
orfmt.Fprintf
. Append functions are better of by naming their behaviour explicitly, e.g.,strconv.AppendInt
,utf8.AppendRune
ortime.AppendFormat
.DeltaBin and DeltaVar describe the internal mechanism, which is not useful to the API reader/user.
The trailing-zero optimization is just as useful on signed and unsigned integers.
I propose the following API.
The text was updated successfully, but these errors were encountered: