Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/install.md
Original file line number Diff line number Diff line change
Expand Up @@ -245,7 +245,7 @@ The example below customizes the following:
- the default timeout from 60 minutes to 20 minutes.
- the default `app.kubernetes.io/managed-by` label is applied to all Pods created to execute `TaskRuns`.
- the default Pod template to include a node selector to select the node where the Pod will be scheduled by default.
For more information, see [`PodTemplate` in `TaskRuns`](./taskruns.md#pod-template) or [`PodTemplate` in `PipelineRuns`](./pipelineruns.md#pod-template).
For more information, see [`PodTemplate` in `TaskRuns`](./taskruns.md#specifying-a-pod-template) or [`PodTemplate` in `PipelineRuns`](./pipelineruns.md#specifying-a-pod-template).

```yaml
apiVersion: v1
Expand All @@ -270,7 +270,7 @@ To customize the behavior of the Pipelines Controller, modify the ConfigMap `fea

- `disable-affinity-assistant` - set this flag to disable the [Affinity Assistant](./workspaces.md#affinity-assistant-and-specifying-workspace-order-in-a-pipeline)
that is used to provide Node Affinity for `TaskRun` pods that share workspace volume.
The Affinity Assistant pods may be incompatible with NodeSelector and other affinity rules
The Affinity Assistant is incompatible with other affinity rules
configured for `TaskRun` pods.

**Note:** Affinity Assistant use [Inter-pod affinity and anti-affinity](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#inter-pod-affinity-and-anti-affinity)
Expand Down
6 changes: 4 additions & 2 deletions docs/workspaces.md
Original file line number Diff line number Diff line change
Expand Up @@ -209,13 +209,15 @@ The `subPath` specified in a `Pipeline` will be appended to any `subPath` specif

Sharing a `Workspace` between `Tasks` requires you to define the order in which those `Tasks`
write to or read from that `Workspace`. Use the `runAfter` field in your `Pipeline` definition
to define when a `Task` should be executed. For more information, see the [`runAfter` documentation](pipelines.md#runAfter).
to define when a `Task` should be executed. For more information, see the [`runAfter` documentation](pipelines.md#using-the-runafter-parameter).

When a `PersistentVolumeClaim` is used as volume source for a `Workspace` in a `PipelineRun`,
an Affinity Assistant will be created. The Affinity Assistant acts as a placeholder for `TaskRun` pods
sharing the same `Workspace`. All `TaskRun` pods within the `PipelineRun` that share the `Workspace`
will be scheduled to the same Node as the Affinity Assistant pod. This means that Affinity Assistant is incompatible
with e.g. NodeSelectors or other affinity rules configured for the `TaskRun` pods. The Affinity Assistant
with e.g. other affinity rules configured for the `TaskRun` pods. If the `PipepineRun` has a custom
[PodTemplate](pipelineruns.md#specifying-a-pod-template) configured, the `NodeSelector` and `Tolerations` fields
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks!

will also be set on the Affinity Assistant pod. The Affinity Assistant
is deleted when the `PipelineRun` is completed. The Affinity Assistant can be disabled by setting the
[disable-affinity-assistant](install.md#customizing-basic-execution-parameters) feature gate.

Expand Down
16 changes: 15 additions & 1 deletion pkg/reconciler/pipelinerun/affinity_assistant.go
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,18 @@ func affinityAssistantStatefulSet(name string, pr *v1beta1.PipelineRun, claimNam
// We want a singleton pod
replicas := int32(1)

// use tolerations from default podTemplate if specified
var tolerations []corev1.Toleration
if pr.Spec.PodTemplate != nil {
tolerations = pr.Spec.PodTemplate.Tolerations
}

// use nodeSelector from default podTemplate if specified
var nodeSelector map[string]string
if pr.Spec.PodTemplate != nil {
nodeSelector = pr.Spec.PodTemplate.NodeSelector
}

containers := []corev1.Container{{
Name: "affinity-assistant",

Expand Down Expand Up @@ -171,7 +183,9 @@ func affinityAssistantStatefulSet(name string, pr *v1beta1.PipelineRun, claimNam
Labels: getStatefulSetLabels(pr, name),
},
Spec: corev1.PodSpec{
Containers: containers,
Containers: containers,
Tolerations: tolerations,
NodeSelector: nodeSelector,
Affinity: &corev1.Affinity{
PodAntiAffinity: &corev1.PodAntiAffinity{
PreferredDuringSchedulingIgnoredDuringExecution: []corev1.WeightedPodAffinityTerm{repelOtherAffinityAssistantsPodAffinityTerm},
Expand Down
52 changes: 52 additions & 0 deletions pkg/reconciler/pipelinerun/affinity_assistant_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,58 @@ func TestCreateAndDeleteOfAffinityAssistant(t *testing.T) {
}
}

func TestThatCustomTolerationsAndNodeSelectorArePropagatedToAffinityAssistant(t *testing.T) {
prWithCustomPodTemplate := &v1beta1.PipelineRun{
TypeMeta: metav1.TypeMeta{Kind: "PipelineRun"},
ObjectMeta: metav1.ObjectMeta{
Name: "pipelinerun-with-custom-podtemplate",
},
Spec: v1beta1.PipelineRunSpec{
PodTemplate: &v1beta1.PodTemplate{
Tolerations: []corev1.Toleration{{
Key: "key",
Operator: "Equal",
Value: "value",
Effect: "NoSchedule",
}},
NodeSelector: map[string]string{
"disktype": "ssd",
},
},
},
}

stsWithTolerationsAndNodeSelector := affinityAssistantStatefulSet("test-assistant", prWithCustomPodTemplate, "mypvc")

if len(stsWithTolerationsAndNodeSelector.Spec.Template.Spec.Tolerations) != 1 {
t.Errorf("expected Tolerations in the StatefulSet")
}

if len(stsWithTolerationsAndNodeSelector.Spec.Template.Spec.NodeSelector) != 1 {
t.Errorf("expected a NodeSelector in the StatefulSet")
}
}

func TestThatTheAffinityAssistantIsWithoutNodeSelectorAndTolerations(t *testing.T) {
prWithoutCustomPodTemplate := &v1beta1.PipelineRun{
TypeMeta: metav1.TypeMeta{Kind: "PipelineRun"},
ObjectMeta: metav1.ObjectMeta{
Name: "pipelinerun-without-custom-podtemplate",
},
Spec: v1beta1.PipelineRunSpec{},
}

stsWithoutTolerationsAndNodeSelector := affinityAssistantStatefulSet("test-assistant", prWithoutCustomPodTemplate, "mypvc")

if len(stsWithoutTolerationsAndNodeSelector.Spec.Template.Spec.Tolerations) != 0 {
t.Errorf("unexpected Tolerations in the StatefulSet")
}

if len(stsWithoutTolerationsAndNodeSelector.Spec.Template.Spec.NodeSelector) != 0 {
t.Errorf("unexpected NodeSelector in the StatefulSet")
}
}

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉 thanks for redoing these twice 🙏

func TestDisableAffinityAssistant(t *testing.T) {
for _, tc := range []struct {
description string
Expand Down