Support keeping existing annotations when mirroring

[flo-02-mu]

Currently when mirroring a secret, all existing annotations are stripped in the target location.

Example:

# Create secret in source and annotate it
kubectl -n ns-origin create secret generic dummy-reflector-secret
kubectl -n ns-origin annotate secret dummy-reflector-secret annotation-key=annotation-value    # <-- This annotation is not mirrored
kubectl -n ns-origin annotate secret dummy-reflector-secret reflector.v1.k8s.emberstack.com/reflection-allowed: "true"
kubectl -n ns-origin annotate secret dummy-reflector-secret reflector.v1.k8s.emberstack.com/reflection-allowed-namespaces=ns-target
kubectl -n ns-origin annotate secret dummy-reflector-secret reflector.v1.k8s.emberstack.com/reflection-auto-enabled="true" 

# Check secret in the target namespace
kubectl -n ns-target get secret dummy-reflector-secret -o yaml

metadata:
  annotations:
    reflector.v1.k8s.emberstack.com/auto-reflects: "True"
    reflector.v1.k8s.emberstack.com/reflected-at: '"2023-06-27T08:53:41.0909994+00:00"'
    reflector.v1.k8s.emberstack.com/reflected-version: "1137728225"
    reflector.v1.k8s.emberstack.com/reflects: ns-origin/dummy-reflector-secret

Our use case would be to use the reflected secret in combination with the reloader-operator to automatically restart pods once a secret changes. Due to the lost annotation, this does not work.

Any chance to keep the existing annotations?

[szechp]

what you can do is disable auto reflect and then create a separate secret in the namespace you want. add the annotation reflector.v1.k8s.emberstack.com/reflects: "namespace/your-secret" there. you can now add annotation or anything else to the mirrored secrets, like you would in any other secret. (This is useful for labels you want in the argocd namespace for instance)

this is documented here: https://github.com/emberstack/kubernetes-reflector#2-annotate-the-mirror-secret-or-configmap

[david-garcia-garcia]

This would be a very interesting feature.....

[smoskova-slm]

At my company we wanted to use the reflector operator in a combination with the reloader operator, as well. The pull-sync option does not work for us because the secret data gets reflected only after it was created. After it was changed the data does not get synched again, so the reloading does not work. Maybe we are doing something wrong, but I could not get it to work this way.

We are instrested in a feature to keep existing annotations and are open to doing the implementation ourselves and contributing. Is this something we can do?

[winromulus]

@smoskova-slm that's not how reflector works and something is wrong with your setup. Reflector picks up changes to the secret when k8s emits an event that the source has changed so reflection occurs each time. Something is broken there.

[smoskova-slm]

Hi, thank you for replying so fast.

Master secret is annotated like this:
annotations: reflector.v1.k8s.emberstack.com/reflection-allowed: "true" reflector.v1.k8s.emberstack.com/reflection-allowed-namespaces: "goal-namespace"
The reflected secret is annotated like this and is without data.
annotations: reflector.v1.k8s.emberstack.com/reflects: "original-namespace/secret-name" reloader.stakater.com/auto: "true"
The secret gets reflected after re-deployment but does not change afterwords. We've tested the reloder with the push mechanism and by manually annotating the secret. That worked, so it must be something with the reflector that is missing. Are there any special permissions or configuration that are necessary? Maybe it is important to mension that the master secret gets deleted and recreated, does that change the behaviour?

[winromulus]

Yes, that does affect the behavior. That's not an update, that's a recreate. Reflector listens for changes and create events. If the source gets deleted, the mirror stays put but a new create does not match the revision. You need to modify your scripts or deployment method to update the source. Also try this out with a simple secret that gets mirrored and watch the logs. Use kubectl or Lens or something to edit the source and you'll see reflector getting that event from k8s and updating the mirror.