fix: allow writing to DeltaTable objects across Python threads

[rtyler]

FREEZE!

Looks like we were really close to full multi-threaded support before,
and just needed to sprinkle a little macro magic on the pyclass
definition.

See pyo3 docs

Fixes #3594

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 3 lines in your changes missing coverage. Please review.

Project coverage is 74.50%. Comparing base (765e129) to head (d41465c).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
python/src/lib.rs	0.00%	3 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3618      +/-   ##
==========================================
- Coverage   74.53%   74.50%   -0.03%     
==========================================
  Files         146      146              
  Lines       44919    44919              
  Branches    44919    44919              
==========================================
- Hits        33480    33467      -13     
- Misses       9247     9251       +4     
- Partials     2192     2201       +9     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[ion-elgreco]

@rtyler nice stuff, didn't know about Frozen but makes total sense. The struct properties are only updated within rust and not python

I don't think Polars allows the DataFrame to be references across a threadpoolexecutor, only processpoolexecutor with spawn

[rtyler]

I don't think Polars allows the DataFrame to be references across a threadpoolexecutor, only processpoolexecutor with spawn

There is no cross thread dataframe access here, this is entirely about our DeltaTable

:😕

[ion-elgreco]

I don't think Polars allows the DataFrame to be references across a threadpoolexecutor, only processpoolexecutor with spawn

There is no cross thread dataframe access here, this is entirely about our DeltaTable

:😕

The multi thread test is using a Polars Dataframe, but it causes this error:

       return self._df.__arrow_c_stream__(requested_schema)
E       RuntimeError: Already borrowed

.venv/lib/python3.9/site-packages/polars/dataframe/frame.py:1466: RuntimeError

[rtyler]

I don't know why that was failing in CI, it didn't fail for me locally. A new run also hasn't failed 🤔 🤷