-
-
Notifications
You must be signed in to change notification settings - Fork 311
make asynccache/addcol-shell thread-safe, and runnable in macros and replay #2826
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: develop
Are you sure you want to change the base?
Conversation
09a33ec
to
b6f1a43
Compare
Demonstration of the errors
This error can be demonstrated with
To demonstrate the race condition possibility, put this in .visidatarc:
then run |
The use of However, I don't know anything about how the removal of the GIL will affect this implementation. But for today's CPython, this PR is safer than the existing code. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for figuring this out, @midichef! The fix looks good to me.
The straightforward way to assign to d[k] is not thread-safe: d[k] = vd.execAsync(_func, k, *args, **kwargs) because events can happen in this order: 1) _func finishes, assigning func()'s return value to d[k], then 2) execAsync's return value is assigned to d[k], overwriting it. The same failure happens when replay mode temporarily uses execSync in place of every execAsync call. Using setdefault() is safe, assigning to d[k] only if _func hasn't yet done so. The setdefault() technique is commonly said to be atomic on CPython, for a standard dictionary with no custom methods.
b6f1a43
to
9510c9e
Compare
Okay, I've moved the comments into the commit message. |
This PR fixes failure of
addcol-shell
in replay mode or macros, where the resulting column has no shell output, just an error in every cell. It also fixes a race condition where most cells would have the expected output, but a rare cell could be empty with an error. Closes #2786.