Adding Continuous Security Intelligence to your DevOps Pipeline

In order to continuously gather pipeline intelligence, DeployHub must become part of your pipeline. DeployHub integrates with Jenkins and other CI/CD engines using the Ortelius Command Line (CLI). The Ortelius CLI gathers supply chain data based on a single pipeline workflow at the build and deploy steps. The build step gathers Swagger, SBOM, Readme, licenses, Git data, Docker image, and other build output. The deploy step records when a release occurs, what was sent and where the objects were sent to.

The Ortelius CLI is maintained by the Ortelius Open Source Community under the governance of the Linux Foundation’s Continuous Delivery Foundation.

For the most up to date information on the Ortelius CLI visit the Ortelius GitHub Repository. You will find a complete list of parameters for collecting Swagger, SBOM and other tool reports and results. Install the Ortelius CLI where your CI/CD server is running.

Ortelius CLI Data Gathering using .toml

The Ortelius CLI reads from a .toml file. The .toml file contains non-derived information for each artifact that you create at your build step. In DeployHub, an artifact is referred to as a Component. A Component is a Container, DB Object, or file object (.jar, Lamda Function, Apex file, etc.). The .toml file will provide the ’non-derived’ data for the Component you are tracking in DeployHub, which includes the Component name, owner, Component type, and owner contact details. The Ortelius CLI will read the .toml file from the Git Repository associated to your pipeline. If you are using a Mono Repository for your entire codebase, you will need a separate Component.toml file for each Component managed in sub-directories.

Once created, your .toml file does not need to be updated unless the non-derived information changes or you want to reorganize to which Applications or Domains the Component has been assigned. For example, a Component has been reassigned to a new owner and new team represented by a Domain or Application.

Variable Resolution

The ${VARIABLE} syntax represents an environment variable or derived variable that will be resolved at execution time. In the below example in Step 2, ${GIT_TAG} is a variable that will be derived by the CLI from git. The derived value will be inserted in to the ${GIT_TAG} variable placeholders.

Derived Variables

Derived Files

The CLI will look for Readme, License, Swagger and OpenAPI files and upload those files to the Component Version.

Readme File Names Scanned for

• README
• README.md
• readme
• readme.md

License File Names Scanned for

• LICENSE
• LICENSE.md
• license
• license.md

Swagger and OpenAPI File Names Scanned for

• swagger.yaml
• swagger.yml
• swagger.json
• openapi.json
• openapi.yaml
• openapi.yml

Perform the following steps to add your Components using the .toml file

Step 1 - Define Your DeployHub Pipeline Variables

The following variables should be set at the beginning of your Pipeline.

Example

export DHURL=https://deployhub.example.com
export DHUSER=Stella99
export DHPASS=password
export DOCKERREPO=quay.io/DeployHub/hello-world
export IMAGE_TAG=1.0.0

Step 2 - Create Your Component.toml File

Cut and paste the following into a component.toml file, update ‘your’ information, and commit/push it to your Git Repository.

# Application Name and Version - not required. If not used the Component will not be associated to an Application
Application = "GLOBAL."your Application Name"
Application_Version = "your Application Version" 

# Define Component Name, Variant and Version - required
Name = "GLOBAL.your Component Name"
Variant = "${GIT_BRANCH}"
Version = "vyour Component Version.${BUILD_NUM}-g${SHORT_SHA}"

# Key/Values to associate to the Component Version
[Attributes]
    DockerBuildDate = "${BLDDATE}"
    DockerRepo = "${DOCKERREPO}"
    DockerSha = "${DIGEST}"
    DockerTag = "${IMAGE_TAG}"
    DiscordChannel = "your Discord channel"
    SlackChannel = "your Slack Channel" 
    ServiceOwner = "${DHUSER}"
    ServiceOwnerEmail = "your Component Owner Email"

For example:

# Application Name and Version
Application = "GLOBAL.Santa Fe Software.Online Store Company.Hipster Store.Prod.helloworld app"
Application_Version = "1" 

# Define Component Name, Variant and Version
Name = "GLOBAL.Santa Fe Software.Online Store Company"
Variant = "${GIT_BRANCH}"
Version = "v1.0.0.${BUILD_NUM}-g${SHORT_SHA}"


# Key/Values to associate to the Component Version
[Attributes]
    DockerBuildDate = "${BLDDATE}"
    DockerRepo = "${DOCKERREPO}"
    DockerSha = "${DIGEST}"
    DockerTag = "${IMAGE_TAG}"
    DiscordChannel = "https://discord.gg/wM4b5yEFzS"
    ServiceOwner= "${DHUSER}"
    ServiceOwnerEmail = "[email protected]"

Note: For SaaS users, you will have a second high-level qualifier that was created as part of your sign-up. This second high-level qualifier must be used as the start of your Application Name and Component Name. For example: GLOBAL.Santa Fe Software.Online Store.

Note: Component Versioning Schema - A more advanced Component Version Schema is represented by the ‘Version" variable. This Variable will allow you to define a custom component versioning schema to suit your organizations requirements.

Example:

Version = "v1.0.0.${BUILD_NUM}-g${SHORT_SHA}"

Step 3 - Add a step in your pipeline to run Syft if you are not generating SBOMS (Optional)

DeployHub can consume any SPDX and CycloneDX formated SBOM. If you are already generating SBOMs, you will pass the name of the SBOM results to DeployHub is step 4 below. If you are not generating SBOMs as part of your pipeline process, you will need to add SBOM generation to collect the lower dependency data. Following is how to add Syft to your workflow to include the collection of SBOM data.

Syft SBOM tool will generate Software Bill of Material Reports for popular coding languages and package managers, including Docker images.

The following code example scans a Docker Image to generate the SBOM. See Syft Options to scan other objects and coding languages.

# install Syft
curl -sSfL https://raw.githubusercontent.com/anchore/syft/main/install.sh | sh -s -- -b $PWD

# create the SBOM
./syft packages $DOCKERREPO:$IMAGE_TAG --scope all-layers -o cyclonedx-json > cyclonedx.json

# display the SBOM
cat cyclonedx.json

Step 4 - Run the Ortelius CLI to add Your Component and Create an Application

Note: To complete the process you will need to install the Ortelius CLI where your CI/CD server is running. Refer to the Ortelius GitHub CLI Documentation for installation instructions.

Execute the following calls to the Ortelius CLI as part of your workflow. It should be called after the build and SBOM generation:

With CycloneDX SBOM

dh updatecomp --rsp component.toml --deppkg "cyclonedx@name of your SBOM file"

Example:
dh updatecomp --rsp component.toml --deppkg "[email protected]"

With SPDX SBOM

dh updatecomp --rsp component.toml --deppkg "spdx@name of your SBOM file. "

Example:
dh updatecomp --rsp component.toml --deppkg "[email protected]"

Without SBOM

dh updatecomp --rsp component.toml 

Results using the CLI in your CI/CD pipeline

Application to Component Dependencies

Select Your Application from the ‘Application View.’ It should show you one Component as a dependency.

Application Level SBOM and CVE

Review the Application SBOM and vulnerabilities. Note: CVE Results may vary depending on the time of the scan.

Component Ownership

Go to the ‘Component View’. You should see your Component Ownership and Detail, including its SBOM and vulnerabilities.

Supply Chain “Package” Search

Go to the ‘Application View.’ Select ‘Package Search’ from the high-level menu. Enter a package name such as ‘spring’ to identify all locations where the package is used.

Recording DORA Metrics for Deployment

Recording deployment frequency and lead time to change provides needed Dora Metrics. This can be done via the CI/CD integration. The output from the deployment can be passed to the CLI to be persisted with the Application Version and Component Versions.

The CLI accepts a json file that defines the Application Version, Environment and deployment return code. The Application Version is an existing application version that was created manually or earlier in your pipeline. The Environment the environment, Dev, QA or Prod, that the Application Version was deployed to. The deployment return code is 0 for successful deployment or 1 for failed deployment.

Deploy.json File

{
 "application": "GLOBAL.Online Store Company.Hipster Store.Test.Hipster Store;Labor Day Sale;1_2_10_2",
 "environment": "GLOBAL.Online Store Company.Hipster Store.qa3-windows",
 "rc": 0
}

Running the CLI

The following variables should be set at the beginning of your Pipeline.

Example

export DHURL=https://deployhub.example.com
export DHUSER=Stella99
export DHPASS=password

dh deploy --deploydata deploy.json --logdeployment

Results

In the DeployHub web UI, navigate to the Application list view. You will see that the Application Version has been deployed to Environment with a Deployment number. The Deployment number is generated by the CLI.

The Deployment Frequency report is accessible by selecting an Application Version in the list view, then menu item Reports -> Deployment Frequency.

The data time the top and bottom graphs are the same but use different visualizations in order to under the the Deployment Frequency.
The graphs are interactive, so float-overs and clicking on the labels will highlight and filter the data.