Skip to main content
Version: 6.x (Latest)

Pipelines

Pipelines allow you to fully customize the execution logic in DevSpace, i.e. they define what happens when a users run commands such as devspace dev, devspace build, devspace deploy or devspace run-pipeline my-custom-pipeline

Pipelines are defined in POSIX shell syntax and a DevSpace pipeline reads as a regular POSIX script. However, DevSpace implements certain special commands that can be used inside the POSIX script to tell DevSpace when to build, deploy or start developing. For a complete function reference, please take a look below.


pipelines:
  deploy: |-
    # Pipelines are a great tool to define your custom execution order of
    # building, deploying and starting dev configurations. This works for
    # all special commands such as: build_images, create_deployments, start_dev
    # run_dependencies and run_pipelines.

    # Run two pipelines in parallel
    run_pipelines other-pipeline-1 other-pipeline-2

    # Wait until all 4 deployments were deployed
    echo "All deployments are deployed"

  other-pipeline-1: |-
    # Deploys deployments deploy-a and deploy-b in parallel
    create_deployments deploy-a deploy-b
    echo "Deployment deploy-a and deploy-b are deployed"

  other-pipeline-2: |-
    # Deploys deployments deploy-c and then deploy-d
    create_deployments deploy-c
    echo "Deployment deploy-c is deployed"
    create_deployments deploy-d
    echo "Deployment deploy-d is deployed"


deployments:
  deploy-a: ...
  deploy-b: ...
  deploy-c: ...
  deploy-d: ...

Using Pipelines​

1. Define Pipelines​

An example pipelines section could look like this in devspace.yaml:

version: v2beta1
pipelines:
  # Override the default pipeline for 'devspace dev'
  dev: |-
    run_dependency_pipelines --all    # 1. Deploy any projects this project needs (see "dependencies")
    create_deployments --all          # 2. Deploy Helm charts and manifests specfied as "deployments"
    start_dev app                     # 3. Start dev mode "app" (see "dev" section)
  deploy: |-
    run_dependency_pipelines --all                    # 1. Deploy any projects this project needs (see "dependencies")
    build_images --all -t $(git describe --always)    # 2. Build, tag (git commit hash) and push all images (see "images")
    create_deployments --all                          # 3. Deploy Helm charts and manifests specfied as "deployments"
  build:
    # Here we are using the long-form using `run:` instead of passing the script directly
    run: |-
      build_images --all -t $(git describe --always)
  custom: ...

Pipelines can:

  • Call built-in functions such as build_images, create_deployments or start_dev
  • Call functions defined in devspace.yaml or in any import
  • Use bash-like syntax including if []; then ... fi, &&, || or ;
  • Access environment variables and DevSpace variables

2. Run Pipelines​

Pipelines are invoked via:

devspace run-pipeline [name]
Cross-Platform Execution

Write all pipeline scrips in bash fashion. DevSpace is using a library to make them cross-platform executable.

Default Pipelines​

Internally DevSpace uses pipelines for the following commands that can be overriden according to your preferences. DevSpace provides default pipeline scripts for the following top-level commands:

dev​

The dev pipeline is executed by running devspace dev and has the following default workflow:

run_dependencies --all
ensure_pull_secrets --all
build_images --all
create_deployments --all
start_dev --all

deploy​

The deploy pipeline is executed by running devspace deploy and has the following default workflow:

run_dependencies --all
ensure_pull_secrets --all
build_images --all
create_deployments --all

build​

The build pipeline is executed by running devspace build and has the following default workflow:

run_dependencies --all --pipeline build
build_images --all

purge​

The purge pipeline is executed by running devspace purge and has the following default workflow:

stop_dev --all
purge_deployments --all
run_dependencies --all --pipeline purge

Custom Flags​

To add custom flags to commands such as devspace dev or devspace run-pipeline my-custom-pipeline, you can specify the flags field for the respective pipeline.

The following example defines two flags for devspace dev:

  • --logs / -l with type not specified (defaults to bool)
  • --env / -e with type: stringArray
version: v2beta1
pipelines:
  dev:
    flags:
    - name: logs
      short: l
    - name: env
      short: e
      type: stringArray
    run: |-
      extraEnv=($(get_flag "env"))    # Retrieve the value of the `env` flag and store it in an array variable
      echo ${extraEnv[0]}             # Arrays are zero indexed

      TERMINAL_ENABLED=true
      if [ $(get_flag "logs") == "true" ]; then     # Test if --logs/-l flag is used or not
        TERMINAL_ENABLED=false                      # Disable terminal, so DevSpace falls back to log streaming
      fi

      start_dev app --set terminal.enabled=$TERMINAL_ENABLED

Built-In Functions​

DevSpace provides a set of built-in functions. There are two types of functions:

  1. Pipeline-Only Functions
  2. Global Functions

Pipeline-Only Functions​

Pipeline-only functions can only be used inside the scripts within the pipelines section of devspace.yaml.

Images

build_images [image-1] [image-2] ... pipeline only​

Builds all images passed as arguments in parallel

--tag / -t []string pipeline only​

If enabled will override the default tags

--skip bool pipeline only​

If enabled will skip building

--skip-push bool pipeline only​

Skip pushing

--skip-push-on-local-kubernetes bool pipeline only​

Skip pushing

--force-rebuild bool pipeline only​

Skip pushing

--sequential bool pipeline only​

Skip pushing

--max-concurrent int pipeline only​

A pointer to an integer

--all bool pipeline only​

Build all images

--except []string pipeline only​

If used with --all, will exclude the following images

--set []string pipeline only​

Set configuration

--set-string []string pipeline only​

Set configuration as string

--from []string pipeline only​

Reuse an existing configuration

--from-file []string pipeline only​

Reuse an existing configuration from a file

ensure_pull_secrets [image-1] [image-2] ... pipeline only​

Creates pull secrets for all images passed as arguments

--set []string pipeline only​

Set configuration

--set-string []string pipeline only​

Set configuration as string

--from []string pipeline only​

Reuse an existing configuration

--from-file []string pipeline only​

Reuse an existing configuration from a file

--all bool pipeline only​

Ensure all pull secrets

--except []string pipeline only​

If used with --all, will exclude the following pull secrets

get_image [image] string pipeline only​

Returns the most recently built image and/or tag for a given image name

--dependency string pipeline only​

Retrieves the image from the named dependency

--only string pipeline only​

Displays either only the tag or only the image

Deployments

create_deployments [deployment-1] [deployment-2] ... pipeline only​

Creates all deployments passed as arguments in parallel

--skip-deploy bool pipeline only​

If enabled, will skip deploying

--force-redeploy bool pipeline only​

Forces redeployment

--sequential bool pipeline only​

Sequentially deploys the deployments

--render bool pipeline only​

If true, prints the rendered manifests to the stdout instead of deploying them

--set []string pipeline only​

Set configuration

--set-string []string pipeline only​

Set configuration as string

--from []string pipeline only​

Reuse an existing configuration

--from-file []string pipeline only​

Reuse an existing configuration from a file

--all bool pipeline only​

Deploy all deployments

--except []string pipeline only​

If used with --all, will exclude the following deployments

purge_deployments [deployment-1] [deployment-2] ... pipeline only​

Purges all deployments passed as arguments

--force-purge bool pipeline only​

Forces purging of deployments even though they might be still in use by other DevSpace projects

--all bool pipeline only​

Deploy all deployments

--except []string pipeline only​

If used with --all, will exclude the following deployments

--sequential bool pipeline only​

Sequentially purges the deployments

Dev

start_dev [dev-1] [dev-2] ... pipeline only​

Starts all dev modes passed as arguments

--continue-on-terminal-exit bool pipeline only​

Continue on terminal exit

--disable-sync bool pipeline only​

If enabled will not start any sync configuration

--disable-port-forwarding bool pipeline only​

If enabled will not start any port forwarding configuration

--disable-pod-replace bool pipeline only​

If enabled will not replace any pods

--disable-open bool pipeline only​

If enabled will not auto-open the URL

--set []string pipeline only​

Set configuration

--set-string []string pipeline only​

Set configuration as string

--from []string pipeline only​

Reuse an existing configuration

--from-file []string pipeline only​

Reuse an existing configuration from a file

--all bool pipeline only​

Start all dev configurations

--except []string pipeline only​

If used with --all, will exclude the following dev configs

stop_dev [dev-1] [dev-2] ... pipeline only​

Stops all dev modes passed as arguments

--force-purge bool pipeline only​

Forces purging of deployments even though they might be still in use by other DevSpace projects

--all bool pipeline only​

Stop all dev configurations

--except []string pipeline only​

If used with --all, will exclude the following dev configs

Pipelines

run_pipelines [pipeline-1] [pipeline-2] ... pipeline only​

Runs all pipelines passed as arguments

--background bool pipeline only​

Run the pipeline in the background

--sequential bool pipeline only​

Run pipelines one after another

--set-flag []string pipeline only​

Set a pipeline flag

--set []string pipeline only​

Set configuration

--set-string []string pipeline only​

Set configuration as string

run_default_pipeline [pipeline] pipeline only​

Runs the default pipeline passed as arguments

run_dependency_pipelines [dependency-1] [dependency-2] ... pipeline only​

Runs a pipeline of each dependency passed as arguments

--pipeline string pipeline only​

The pipeline to deploy from the dependency

--exclude []string pipeline only​

Dependencies to exclude

--only []string pipeline only​

Dependencies to include

--sequential bool pipeline only​

Run dependencies one after another

--set-flag []string pipeline only​

Set a pipeline flag

--all bool pipeline only​

Deploy all dependencies

--except []string pipeline only​

If used with --all, will exclude the following dependencies

Checks

is_dependency int pipeline only​

Returns exit code 0 if the pipeline currently being executed is run because the project is a dependency of another project

Other

select_pod string pipeline only​

Returns the name of a Kubernetes pod

--image-selector string pipeline only​

The image selector to use to select the container

--label-selector string pipeline only​

The label selector to use to select the container

--container string pipeline only​

The container to use

--namespace / -n string pipeline only​

The namespace to use

--disable-wait bool pipeline only​

If true, will not wait for the container to become ready

--timeout int64 pipeline only​

The timeout to wait. Defaults to 5 minutes

wait_pod [command] pipeline only​

Waits for a pod to become running

--image-selector string pipeline only​

The image selector to use to select the container

--label-selector string pipeline only​

The label selector to use to select the container

--container string pipeline only​

The container to use

--namespace / -n string pipeline only​

The namespace to use

--disable-wait bool pipeline only​

If true, will not wait for the container to become ready

--timeout int64 pipeline only​

The timeout to wait. Defaults to 5 minutes

exec_container [command] pipeline only​

Executes the command provided as argument inside a container

--image-selector string pipeline only​

The image selector to use to select the container

--label-selector string pipeline only​

The label selector to use to select the container

--container string pipeline only​

The container to use

--namespace / -n string pipeline only​

The namespace to use

--disable-wait bool pipeline only​

If true, will not wait for the container to become ready

--timeout int64 pipeline only​

The timeout to wait. Defaults to 5 minutes

get_config_value [json.path] string pipeline only​

Returns the value of the config loaded from devspace.yaml

Global Functions​

Global functions can be used anywhere in devspace.yaml, either in config fields that expect a bash script such as within functions or using $(command) vars in any other config field.

Checks

is_empty [value] int pipeline only​

Returns exit code 0 if the value of the argument is empty string

is_equal [value-1] [value-2] int pipeline only​

Returns exit code 0 if the values of both arguments provided are equal

is_in [value-1] [value-2] int pipeline only​

Returns exit code 0 if the value of the first argument can be found in the second argument (second argument being a blank-separated list of strings e.g "bananas apples peaches")

is_os [os] darwin linux windows aix android dragonfly freebsd hurd illumos ios js nacl netbsd openbsd plan9 solaris zos int pipeline only​

Returns exit code 0 if the current operating system equals the value provided as argument

is_true [value] int pipeline only​

Returns exit code 0 if the value of the argument is "true"

Other

cat [file-path] string pipeline only​

Returns the content of a file

get_flag [flag-name] string pipeline only​

Returns the value of the flag that is provided as argument

run_watch [command] pipeline only​

Executes the command provided as argument and watches for conditions to restart the command

--fail-on-error bool pipeline only​

If true the command will fail on an error while running the sub command

--skip-initial bool pipeline only​

If true will not execute the command immediately.

--silent bool pipeline only​

If true will not print any warning about restarting the command.

--skip-and-silent / -s bool pipeline only​

If enabled will not print when the command is restarted and not execute the command initially.

--exclude / -e []string pipeline only​

The paths to ignore. Can be patterns in the form of ./**/my-file.txt

--path / -p []string pipeline only​

The paths to watch. Can be patterns in the form of ./**/my-file.txt

sleep [seconds] pipeline only​

Pauses the script execution for the number of seconds provided as argument

xargs [command] pipeline only​

Reads from stdin, splits input by blanks and executes the command provided as argument for each blank-separated input value (often used in pipes, e.g. echo 'image-1 image-2' | xargs build_images)

Config Reference​

pipelines required <pipeline_name>:object ​

Pipelines are the work blocks that DevSpace should execute when devspace dev, devspace build, devspace deploy or devspace purge is called. Pipelines are defined through a special POSIX script that allows you to use special commands such as create_deployments, start_dev, build_images etc. to signal DevSpace you want to execute a specific functionality. The pipelines dev, build, deploy and purge are special and will override the default functionality of the respective command if defined. All other pipelines can be either run via the devspace run-pipeline command or used within another pipeline through run_pipelines.

<pipeline_name> required string dev
deploy
build
purge
.*​

Name of the pipeline, will be filled automatically

run required string ​

Run is the actual shell command that should be executed during this pipeline

flags required object[] ​

Flags are extra flags that can be used for running the pipeline via devspace run-pipeline.

name required string ​

Name is the name of the flag

short required string ​

Short is optional and is the shorthand name for this flag. E.g. 'g' converts to '-g'

type required string bool bool
int
string
stringArray​

Type is the type of the flag. Defaults to bool

default required ​

Default is the default value for this flag

description required string ​

Description is the description as shown in devspace run-pipeline my-pipe -h

continueOnError required boolean false ​

ContinueOnError will not fail the whole job and pipeline if a call within the step fails.