[FEATURE] Open source the C++ backend and Python package build process

[zrnsm]

Description

It seems like the current state of affairs with the 4.0.1 release on PyPi is that the backend is locked in a compiled shared library that simply must be trusted. Is the strategy going forward to keep that module closed source?

Use Cases

No response

[khustup2]

Hi @zrnsm thanks for raising this question. The process of open sourcing the C++ backend is ongoing. We will keep you posted once it's done.

I'm curious, if you are interested in contributing to the C++ codebase?

Thanks.

[zrnsm]

@khustup2 I'd definitely be interested in contributing if deeplake works for my use case. I'm still just evaluating some options in this space.

How is the binding implemented? pybind11?

[khustup2]

Great! We will keep you posted.
Hopefully deeplake will work on your use case. In case it's not, please let us know what didn't go well.

Yes, bindings are implemented using pybind11.