Actual behavior of __context__ for execution modules?

[dbushong]

Per the docs "developing modules" section about __context__:

When running an execution module __context__ persists across all module executions until the modules are refreshed; such as when saltutil.sync_all or state.apply are executed.
...
Note: Because __context__ may or may not have been destroyed, always be sure to check for the existence of the key in __context__ and generate the key before using it.

And indeed, when I look at some of the builtin modules, e.g., they certainly seem to assume that it works that way (i.e. that __context__ is useful as a cache between runs)

However, when I actually try a simple example of (re-)using __context__, it seems to be reset on every call, e.g.:

root@d245ff4b26d3:/home/salt# salt mywin mystuff.counter
mywin:
    1
root@d245ff4b26d3:/home/salt# salt mywin mystuff.counter
mywin:
    1
root@d245ff4b26d3:/home/salt# salt mywin mystuff.counter
mywin:
    1

And counter.py is:

def counter():
    if "mycounter" in __context__:
        count = __context__["mycounter"]
    else:
        count = 0
    count += 1
    __context__["mycounter"] = count
    return count

Additionally, I turned on debug logging, and tried repeatedly running chocolatey.chocolatey_version that I linked above, and AFAICT from the logs, it is also never seeing any persistent values in __context__ and is always running from scratch.

Do I have some minion config settings wrong? (I tried with multiprocess: False and True with no difference)

Am I misunderstanding the documentation?

[max-arnold]

I think it might persist between multiple module calls that happen within a single run.

[mattp-]

@dwoz i think you introduced a regression with https://github.com/saltstack/salt/blame/master/salt/minion.py#L1990
gen_modules flushes all loader state, if you call that on thread return it makes it behave as @max-arnold described. looking at this now, even if it worked, it wouldn't work for multi-master. it wouldn't be too hard to fix, just attach a dict onto minionmanager as a class var, and use that. thoughts?

[mattp-]

#68055 noticed log_beacon relies on context, this is probably making it read entire file every iteration

[dwoz]

This would have broke with the loader re-factor which fixed several race conditions/threading bugs in the loader. If we want this functional again we'll need a comprehensive solution that handle multiprocessing with some locks around __context__.

I'm also curious what the expected behavior should be when a minion re-starts.

[dwoz]

@mattp- ^^

[mattp-]

@dwoz im not actually sure what the full expected behavior is :) i wouldnt expect something serialized to disk/it should be fresh on restart. shared state across all minion objects id probably expect to work / would make sense.