Learn Code Stream

Learn Code Stream

VMware vRealize Automation

Pipelines

A Pipeline is the primary mechanism for sequencing all the tasks that need to be performed, and is composed of one or more Stage, with one or more tasks in each stage.

General Pipeline Settings

The pipeline settings allow you to set the pipeline name, concurrency, description, icon and tags.

Being able to change the concurrency of the pipeline is often useful if you’re using shared resources that either don’t have the capcity to host multiple running executions, or there are resources that cannot be shared
Pipeline general settings
Pipeline general settings

Pipeline configuration

When editing a Pipeline there are four tabs to configure:

The workspace tab configures the environment in which the pipeline runs

  • Host specifies a Docker endpoint on which CI tasks and Custom Integrations will execute
  • Builder image URL configures the container image that will be used for CI tasks or Custom Integrations. You can specify using the just an official image (e.g. python), the image and a tag (e.g. python:3.10.0a6-alpine) or a full URL (e.g. projects.registry.vmware.com/antrea/prom-prometheus:v2.19.3)
    The container image can be almost any image but it needs to have wget or curl in order to download and install the Code Stream CI Agent, which is installed when the container is spun up.
  • Image Registry selects the Docker Registry endpoint to use to pull the Builder image - if the registry requires credentials to pull an image you can specify them as part of the endpoint and those will be used.
  • Working directory is the directory within a container image that will be used when running commands (workingDir, by default) in a CI task - more often than not, you can leave this blank to default to /build
  • Cache is accessible to each Pipeline run and can be used to cache files and folders that are common between Pipeline runs - for example if you download dependencies before building a Go project, those dependencies could be cached to speed up future executions of the same pipeline.
  • Environment Variables can be used to pass environment variables to a container (similar to the -e VAR_NAME="value" flag in the docker run command)
  • CPU limit if a CI task requires significant resources, the container’s allocated CPU can be increased - it’s not often required
  • Memory limit if a CI task requires significant resources, container’s allocated Memory can be increased - it’s not often required
  • Git clone - if the pipeline is triggered by a Git webhook , CI tasks will automatically clone the Git repository.
    Note: You will need to configure the pipeline Inputs with the Git auto-inject parameters for this to work!

Pipeline Workspace Configuration

Pipeline Input Configuration

The Model tab is where you configure the Stages and Tasks of the pipeline - it’s where you spend most of your time when creating and editing pipelines.

  • A Stage is an encapsulation mechanism for tasks and are used for grouping the individual task execution statuses and results.
  • A Task performs individual actions based on its type and configuration. Tasks can deploy VMware Cloud Templates , and perform actions on configured endpoints, or more generic tasks such as prompting for user interations with User Operation , polling a 3rd party data source with the Poll task, or even perform a REST call.

Pipeline Model Configuration

Outputs can be mapped to values produced by tasks in a pipeline and can be useful when you’re nesting pipelines using the Pipeline task to return the results to the parent pipeline.

Pipeline Output Config

Variables in Pipelines

Most configurable fields within a Pipeline can also use [Variables], references to Input parameters, the output of other tasks or general pipeline properties by using a reference.

These can be accessed using by typing $, which will bring up the auto-completion:

Reference auto-completion
Reference auto-completion

Referring back to previous tasks is done using a heirarchy that matches the structure of the pipeline, for example: ${Build Stage.Build Task.output.exports.variableName} would refer to the value of a variable called variableName that was exported from a task called Build Task in the stage Build Stage.

Tasks return their output as JSON, and it’s often useful to look at a previously executed task to find the correct path to an output variable - if you look at the Execution of a Pipeline and examine the task details, you can click “View Output JSON” and use the “Path finder” option to discover the correct path: Find the correct output path

Notifications

The notifications tab allows you to configure notifications for pipeline events (completion, waiting for user interaction , failure, cancellation, and starting) using either an Email endpoint , Jira endpoint , or by creating a Webhook with a POST, PUT or PATCH payload.

Pipeline Email Notifications
Pipeline Email Notifications
Pipeline Jira Notifications
Pipeline Jira Notifications
Pipeline Webhook Notifications
Pipeline Webhook Notifications

More

Reference

Last updated on 12 Apr 2021
 Edit on GitHub