Skip to content

data-science-pipelines-operator: Dataflow

Controller Watches

Kubernetes resources this controller monitors for changes. Each watch triggers reconciliation when the watched resource is created, updated, or deleted.

Type GVK Source
For api/v1/DataSciencePipelinesApplication controllers/dspipeline_controller.go:796
Owns /v1/ConfigMap controllers/dspipeline_controller.go:799
Owns /v1/PersistentVolumeClaim controllers/dspipeline_controller.go:802
Owns /v1/Secret controllers/dspipeline_controller.go:798
Owns /v1/Service controllers/dspipeline_controller.go:800
Owns /v1/ServiceAccount controllers/dspipeline_controller.go:801
Owns apps/v1/Deployment controllers/dspipeline_controller.go:797
Owns networking.k8s.io/v1/NetworkPolicy controllers/dspipeline_controller.go:803
Owns rbac.authorization.k8s.io/v1/Role controllers/dspipeline_controller.go:804
Owns rbac.authorization.k8s.io/v1/RoleBinding controllers/dspipeline_controller.go:805
Owns route/v1/Route controllers/dspipeline_controller.go:806

Programmatic Resource Operations

Verb Kind Group Condition
update DataSciencePipelinesApplication api

Reconciliation Flow

How the controller interacts with the Kubernetes API during reconciliation.

sequenceDiagram
    %% Static dataflow for data-science-pipelines-operator

    participant KubernetesAPI as Kubernetes API
    participant data_science_pipelines_operator_controller_manager as data-science-pipelines-operator-controller-manager
    participant mariadb as mariadb
    participant minio as minio
    participant the_deployment as the-deployment

    KubernetesAPI->>+data_science_pipelines_operator_controller_manager: Watch DataSciencePipelinesApplication (reconcile)
    data_science_pipelines_operator_controller_manager->>KubernetesAPI: Create/Update ConfigMap
    data_science_pipelines_operator_controller_manager->>KubernetesAPI: Create/Update PersistentVolumeClaim
    data_science_pipelines_operator_controller_manager->>KubernetesAPI: Create/Update Secret
    data_science_pipelines_operator_controller_manager->>KubernetesAPI: Create/Update Service
    data_science_pipelines_operator_controller_manager->>KubernetesAPI: Create/Update ServiceAccount
    data_science_pipelines_operator_controller_manager->>KubernetesAPI: Create/Update Deployment
    data_science_pipelines_operator_controller_manager->>KubernetesAPI: Create/Update NetworkPolicy
    data_science_pipelines_operator_controller_manager->>KubernetesAPI: Create/Update Role
    data_science_pipelines_operator_controller_manager->>KubernetesAPI: Create/Update RoleBinding
    data_science_pipelines_operator_controller_manager->>KubernetesAPI: Create/Update Route

    Note over data_science_pipelines_operator_controller_manager: Exposed Services
    Note right of data_science_pipelines_operator_controller_manager: data-science-pipelines-operator-service:8080/TCP [metrics]
    Note right of data_science_pipelines_operator_controller_manager: ds-pipeline-workflow-controller-metrics-template-value:9090/TCP [metrics]
    Note right of data_science_pipelines_operator_controller_manager: mariadb:3306/TCP []
    Note right of data_science_pipelines_operator_controller_manager: mariadb-template-value:3306/TCP []
    Note right of data_science_pipelines_operator_controller_manager: minio:9000/TCP [https]
    Note right of data_science_pipelines_operator_controller_manager: minio:9001/TCP [console]
    Note right of data_science_pipelines_operator_controller_manager: minio-service:9000/TCP [http]
    Note right of data_science_pipelines_operator_controller_manager: minio-template-value:9000/TCP [http]
    Note right of data_science_pipelines_operator_controller_manager: minio-template-value:80/TCP [kfp-ui-http]
    Note right of data_science_pipelines_operator_controller_manager: ml-pipeline:8443/TCP [proxy]
    Note right of data_science_pipelines_operator_controller_manager: ml-pipeline:8888/TCP [http]
    Note right of data_science_pipelines_operator_controller_manager: ml-pipeline:8887/TCP [grpc]
    Note right of data_science_pipelines_operator_controller_manager: pypi-server:8080/TCP [pypi-server]
    Note right of data_science_pipelines_operator_controller_manager: template-value:8443/TCP [proxy]
    Note right of data_science_pipelines_operator_controller_manager: template-value:8888/TCP [http]
    Note right of data_science_pipelines_operator_controller_manager: template-value:8887/TCP [grpc]
    Note right of data_science_pipelines_operator_controller_manager: template-value:8443/TCP [webhook]
    Note right of data_science_pipelines_operator_controller_manager: the-service:8666/TCP []
    Note right of data_science_pipelines_operator_controller_manager: the-service:8666/TCP []

    Note over KubernetesAPI: Defined CRDs
    Note right of KubernetesAPI: DataSciencePipelinesApplication (datasciencepipelinesapplications.opendatahub.io/v1)
    Note right of KubernetesAPI: ScheduledWorkflow (kubeflow.org/v1beta1)
    Note right of KubernetesAPI: Pipeline (pipelines.kubeflow.org/v2beta1)
    Note right of KubernetesAPI: PipelineVersion (pipelines.kubeflow.org/v2beta1)

Webhooks

Name Type Path Failure Policy Service Overlays Enable Condition Sources
pipelineversions.pipelines.kubeflow.org mutating /webhooks/mutate-pipelineversion Fail template-value/template-value config/internal/webhook/mutating_webhook.yaml.tmpl
pipelineversions.pipelines.kubeflow.org validating /webhooks/validate-pipelineversion Fail template-value/template-value config/internal/webhook/validating_webhook.yaml.tmpl

HTTP Endpoints

Method Path Source
* / .gomod-cache/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/cmd/trace/main.go:188
* / .gopath-loader/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/cmd/trace/main.go:188
* / .gopath-loader/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/net/http/triv.go:130
* / .gopath-loader/pkg/mod/golang.org/x/net@v0.52.0/webdav/litmus_test_server.go:83
* / .gomod-cache/golang.org/x/net@v0.52.0/webdav/litmus_test_server.go:83
* / .gomod-cache/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/net/http/triv.go:130
* /args .gopath-loader/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/net/http/triv.go:136
* /args .gomod-cache/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/net/http/triv.go:136
* /bar .gomod-cache/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/net/http/doc.go:67
* /bar .gopath-loader/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/net/http/doc.go:67
* /block .gomod-cache/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/cmd/trace/main.go:210
* /block .gopath-loader/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/cmd/trace/main.go:210
* /chan .gopath-loader/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/net/http/triv.go:134
* /chan .gomod-cache/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/net/http/triv.go:134
* /counter .gomod-cache/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/net/http/triv.go:129
* /counter .gopath-loader/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/net/http/triv.go:129
* /date .gopath-loader/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/net/http/triv.go:138
* /date .gomod-cache/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/net/http/triv.go:138
* /debug/health .gomod-cache/github.com/docker/distribution@v2.8.3+incompatible/health/health.go:305
* /debug/health .gopath-loader/pkg/mod/github.com/docker/distribution@v2.8.3+incompatible/health/health.go:305
* /debug/health/down .gopath-loader/pkg/mod/github.com/docker/distribution@v2.8.3+incompatible/health/api/api.go:35
* /debug/health/down .gomod-cache/github.com/docker/distribution@v2.8.3+incompatible/health/api/api.go:35
* /debug/health/up .gomod-cache/github.com/docker/distribution@v2.8.3+incompatible/health/api/api.go:36
* /debug/health/up .gopath-loader/pkg/mod/github.com/docker/distribution@v2.8.3+incompatible/health/api/api.go:36
* /debug/pprof/ .gopath-loader/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/manager/internal.go:329
* /debug/pprof/ .gomod-cache/sigs.k8s.io/controller-runtime@v0.23.3/pkg/manager/internal.go:329
* /debug/pprof/cmdline .gopath-loader/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/manager/internal.go:330
* /debug/pprof/cmdline .gomod-cache/sigs.k8s.io/controller-runtime@v0.23.3/pkg/manager/internal.go:330
* /debug/pprof/profile .gomod-cache/sigs.k8s.io/controller-runtime@v0.23.3/pkg/manager/internal.go:331
* /debug/pprof/profile .gopath-loader/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/manager/internal.go:331
* /debug/pprof/symbol .gomod-cache/sigs.k8s.io/controller-runtime@v0.23.3/pkg/manager/internal.go:332
* /debug/pprof/symbol .gopath-loader/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/manager/internal.go:332
* /debug/pprof/trace .gomod-cache/sigs.k8s.io/controller-runtime@v0.23.3/pkg/manager/internal.go:333
* /debug/pprof/trace .gopath-loader/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/manager/internal.go:333
* /debug/vars .gopath-loader/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/expvar/expvar.go:382
* /debug/vars .gomod-cache/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/expvar/expvar.go:382
* /flags .gopath-loader/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/net/http/triv.go:135
* /flags .gomod-cache/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/net/http/triv.go:135
* /foo .gomod-cache/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/net/http/doc.go:65
* /foo .gopath-loader/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/net/http/doc.go:65
* /go/ .gopath-loader/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/net/http/triv.go:132
* /go/ .gomod-cache/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/net/http/triv.go:132
* /go/hello .gomod-cache/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/net/http/triv.go:137
* /go/hello .gopath-loader/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/net/http/triv.go:137
* /goroutine .gopath-loader/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/cmd/trace/main.go:203
* /goroutine .gomod-cache/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/cmd/trace/main.go:203
* /goroutines .gopath-loader/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/cmd/trace/main.go:202
* /goroutines .gomod-cache/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/cmd/trace/main.go:202
* /io .gomod-cache/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/cmd/trace/main.go:209
* /io .gopath-loader/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/cmd/trace/main.go:209
* /jsontrace .gopath-loader/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/cmd/trace/main.go:198
* /jsontrace .gomod-cache/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/cmd/trace/main.go:198
* /mmu .gomod-cache/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/cmd/trace/main.go:206
* /mmu .gopath-loader/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/cmd/trace/main.go:206
GET /ping .gomod-cache/github.com/anthhub/forwarder@v1.1.0/example/httpserver/main.go:18
GET /ping .gopath-loader/pkg/mod/github.com/anthhub/forwarder@v1.1.0/example/httpserver/main.go:18
* /regionblock .gopath-loader/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/cmd/trace/main.go:216
* /regionblock .gomod-cache/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/cmd/trace/main.go:216
* /regionio .gopath-loader/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/cmd/trace/main.go:215
* /regionio .gomod-cache/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/cmd/trace/main.go:215
* /regionsched .gopath-loader/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/cmd/trace/main.go:218
* /regionsched .gomod-cache/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/cmd/trace/main.go:218
* /regionsyscall .gomod-cache/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/cmd/trace/main.go:217
* /regionsyscall .gopath-loader/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/cmd/trace/main.go:217
* /sched .gopath-loader/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/cmd/trace/main.go:212
* /sched .gomod-cache/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/cmd/trace/main.go:212
* /static/ .gomod-cache/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/cmd/trace/main.go:199
* /static/ .gopath-loader/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/cmd/trace/main.go:199
* /syscall .gopath-loader/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/cmd/trace/main.go:211
* /syscall .gomod-cache/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/cmd/trace/main.go:211
* /trace .gomod-cache/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/cmd/trace/main.go:197
* /trace .gopath-loader/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/cmd/trace/main.go:197
* /userregion .gopath-loader/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/cmd/trace/main.go:222
* /userregion .gomod-cache/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/cmd/trace/main.go:222
* /userregions .gopath-loader/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/cmd/trace/main.go:221
* /userregions .gomod-cache/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/cmd/trace/main.go:221
* /usertask .gopath-loader/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/cmd/trace/main.go:226
* /usertask .gomod-cache/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/cmd/trace/main.go:226
* /usertasks .gomod-cache/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/cmd/trace/main.go:225
* /usertasks .gopath-loader/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/cmd/trace/main.go:225
GET /{user-id} .gomod-cache/github.com/emicklei/go-restful/v3@v3.13.0/doc.go:82
GET /{user-id} .gopath-loader/pkg/mod/github.com/emicklei/go-restful/v3@v3.13.0/doc.go:19
GET /{user-id} .gomod-cache/github.com/emicklei/go-restful/v3@v3.13.0/doc.go:19
GET /{user-id} .gopath-loader/pkg/mod/github.com/emicklei/go-restful/v3@v3.13.0/doc.go:82
* G .gopath-loader/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/testing/slogtest/slogtest.go:225
* G .gopath-loader/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/testing/slogtest/slogtest.go:97
* G .gomod-cache/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/testing/slogtest/slogtest.go:97
* G .gomod-cache/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/testing/slogtest/slogtest.go:203
* G .gomod-cache/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/testing/slogtest/slogtest.go:109
* G .gomod-cache/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/testing/slogtest/slogtest.go:225
* G .gopath-loader/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/testing/slogtest/slogtest.go:203
* G .gopath-loader/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/testing/slogtest/slogtest.go:109
* GET /debug/vars .gomod-cache/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/expvar/expvar.go:384
* GET /debug/vars .gopath-loader/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/expvar/expvar.go:384
* header .gopath-loader/pkg/mod/golang.org/x/net@v0.52.0/quic/qlog.go:211
* header .gopath-loader/pkg/mod/golang.org/x/net@v0.52.0/quic/qlog.go:267
* header .gomod-cache/golang.org/x/net@v0.52.0/quic/qlog.go:165
* header .gomod-cache/golang.org/x/net@v0.52.0/quic/qlog.go:211
* header .gomod-cache/golang.org/x/net@v0.52.0/quic/qlog.go:267
* header .gopath-loader/pkg/mod/golang.org/x/net@v0.52.0/quic/qlog.go:165
* header .gomod-cache/golang.org/x/net@v0.52.0/quic/qlog.go:187
* header .gopath-loader/pkg/mod/golang.org/x/net@v0.52.0/quic/qlog.go:187
* raw .gomod-cache/golang.org/x/net@v0.52.0/quic/qlog.go:217
* raw .gopath-loader/pkg/mod/golang.org/x/net@v0.52.0/quic/qlog.go:217
* raw .gopath-loader/pkg/mod/golang.org/x/net@v0.52.0/quic/qlog.go:193
* raw .gopath-loader/pkg/mod/golang.org/x/net@v0.52.0/quic/qlog.go:172
* raw .gomod-cache/golang.org/x/net@v0.52.0/quic/qlog.go:193
* raw .gomod-cache/golang.org/x/net@v0.52.0/quic/qlog.go:172
* request .gopath-loader/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/log/slog/doc.go:137
* request .gomod-cache/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/log/slog/doc.go:137
* vantage_point .gomod-cache/golang.org/x/net@v0.52.0/quic/qlog.go:96
* vantage_point .gopath-loader/pkg/mod/golang.org/x/net@v0.52.0/quic/qlog.go:96

Configuration

ConfigMaps and Helm values that control this component's runtime behavior.

ConfigMaps

Name Data Keys Source
workflow-controller-configmap config/argo/configmap.workflow-controller-configmap.yaml