SSH occasional disconnection after vMotion of a containerVM finishes

[keramix]

Details

If a user is attached to a containerVM that gets vMotioned, there is a chance the ssh connection will get disconnected after vMotion completes. The containerVM will be powered on but the user might get detached after vmotion completes. This event happens if the user is typing on the stdin right before the containerVM goes to the downtime state (usually between 58% and 72% of the vMotion progress)

By further investigating the issue, it turns out that the SSH Server at the tether side closes the connection because of a MAC (Message Authentication Code) failure. This possibly means that a few bytes that the SSH client sends do not get read by the SSH server while vMotion is happening leading to an incorrect SSH MAC. Attached are the logs (enabled the crypto/ssh logs as well). Most relevant are the tether logs.

Steps to Reproduce

1- Run a container and attach to it
2- Start vmotion
3- Type random keys on the containerVM stdin between 58% and 72% of vmotion progress

log10.zip

[mdubya66]

This edge case needs a release note.

[mhagen-vmware]

please prioritize, estimate and move to backlog

[stuclem]

Proposed release note:

---

• Occasional disconnection during vMotion. #4484
  If you are attached to a container VM that is migrated by vMotion, the SSH connection to the container VM might drop when vMotion completes.

---

@mhagen-vmware do we need to say anything more?

[mhagen-vmware]

lgtm

[mhagen-vmware]

could also add, repair steps - issue another docker attach after vmotion completes to reattach.

[stuclem]

Done. Thanks @mhagen-vmware.

[corrieb]

@stuclem The fact that SSH is being used as the transport is completely hidden from the client - it's purely an implementation detail. So we should phrase the note purely in terms of docker attach.