Advanced Technical Information

Overview

This provides advanced technical information related to the repo integrations.

Files that Can Trigger a Repository Scan

Adding/changing/deleting one of the files on the list below will automatically trigger a scan of your repository using the Mend Repository Integration.

• bower.json

• build.gradle

• build.gradle.kts

• build.sbt

• conanfile.py

• conanfile.txt

• cargo.toml

• composer.json

• dependencies.scala

• environment.yml

• Gemfile.lock

• glide.lock

• go.mod

• Godeps.lock

• gogradle.lock

• Gopkg.lock

• gradle.lockfile

• gradle.properties

• libs.gradle

• libs.versions.toml

• package-lock.json

• package.json

• paket.dependencies

• packages.config

• packages.lock.json

• packrat.lock

• Pipfile

• pipfile.lock

• pnpm-lock.yaml

• Podfile

• poetry.lock

• pom.xml

• pubspec.yaml

• pyproject.toml

• requirements.txt

• settings.gradle

• setup.cfg

• setup.py

• vendor.conf

• versions.kt

• yarn.lock

• Any metafile with one of the following extensions: 

  • asp

  • aspx

  • config

  • csproj

  • do

  • gitmodules

  • htm

  • html

  • jsp

  • shtml

  • tf

  • xhtml

• Cargo.lock

Technical Information for Self-Hosted Integrations

Modifying the Scanner Dockerfile

The wss-scanner image Dockerfile is located in the wss-scanner\docker\ folder.
By default, the following package managers are installed:

• Maven (3.5.4)

• npm

• Bower

• Yarn

• Gradle

• pip and pip3 (Python)

If you would like to add support for additional package managers, uncomment the relevant lines in the Dockerfile. The following package managers are available as part of the commented lines in the Dockerfile:

• Mix (Elixir)

• Hex (Erlang)

• Go Modules, Dep, godep, VNDR, govendor, gopm, glide (Go)

• Cabal (Haskell)

• Bazel (Java)

• Paket, NuGet (.NET)

• Composer (PHP)

• Poetry (Python)

• Packrat (R)

• Bundler (Ruby)

• Cargo (Rust)

• SBT (Scala)

• Cocoapods (Swift)

List of Installed Package Managers

The table below lists the installed package manager versions on Self-Hosted Integrations Doker images, which work out-of-the-box.
The default version will be used, unless dynamic version detection is in effect.
When other installed versions are present, they can be used by manually setting the version in the configuration.

Python Support

Automatic Detection of Python Versions

By default, the SCA scanner automatically detects the required Python version for a project by checking standard configuration files within the repository, and uses them to resolve dependencies.

The scanner checks for Python version specifications in the following files, in order of precedence:

 If no version file is found, the SCA scanner will default to the global configuration (e.g., Python 3.9).

Limitations

• The feature is only supported in version 26.3.1 of the integration or above.

• Custom or non-standard Python version files are not supported.

• Python versions specified using local identifiers like "system", or those starting with "ref:" or "path:" are skipped.

• Version epochs are skipped.

• Developmental releases are not supported.

• Explicit Python version settings set in the scanning configuration of the repository integration will override auto-detection.

• In mono-repos, only one version is selected for all sub-projects (the last one found).

• For pipenv, using versions older than 3.10.x will fail.

• Unified Agent configurations explicitly defining the version to use take precedence.

• Only types of version schemas, logic operators and comparison operators that are defined by PEP 440 or are valid semver strings are supported.

• The + local version identifier is not supported. The SCA scanner will omit it from the version specification and attempt to use that instead (e.g., 1.2.3+debian1 → 1.2.3).

• The SCA scanner will attempt to convert the === arbitrary equality operator used to specify non-PEP 440 versions to exact match (==).

• The SCA scanner will strip pre-release versions (e.g., 3.13.0a1, 3.13.0b2, 3.13.0rc1) to a final version and attempt to use that instead (e.g., 3.13.0a1 → 3.13.0).

• The SCA scanner will attempt to convert Conda’s environment.yaml which contains a hash (e.g., python=3.9.7=h12debd9_0) to the semver version without the hash (e.g., 3.9.7).

• The SCA scanner will use regex to detect Python versions in Python setup.py files, but variables are not supported (e.g., python_requires=PYTHON_REQ).

• Only cpython versions are supported. The SCA scanner will attempt to switch from other implementations (pypy, graalpy, ironpython) to cpython.

Manual Setting of Python Versions

You can specify the Python version for your repository based on the supported versions.
For example:
2.7.18, 3.7.17, 3.9.18, 3.13, etc.

For this you will need to perform the following procedure:

1. Ensure the relevant Python version is uncommented in your scanner container’s Dockerfile.

2. Add or edit the .whitesource configuration file in your repository.

3. Use the configMode parameter and set it to either LOCAL or EXTERNAL.

4. Create a whitesource.config file and add the following:

   python.invokePipAsModule=true
   python.path=python3.9
   python.installVirtualenv=true
   

Enable git LFS in the Scanner

To enable git LFS, uncomment the last two lines of this section of the Dockerfile:

# install git lfs
#ARG GIT_LFS_VERSION=v3.5.1
#RUN install-tool git-lfs

Note: This feature will not work with Jgit as the cloning utility (i.e., WS_GIT_CONNECTOR=false, as documented here).

You can also refer the official page of the extension at https://git-lfs.com/.

Required Open Ports

The wss-scanner Docker Container

The wss-scanner Docker container communicates with the following components using the following ports:

• Mend SaaS API → Port 443

• Your repository platform’s git protocol → The default is port 9418

• Private/public package registries (npmjs/pypi/ruby gems, etc.) which use the standard ports

The wss-gls-app/wss-ghe-app/wss-bb-app Docker Container

The wss-app Docker container communicates with the following components using the following ports:

• Your repository platform instance API → Check the port number with your repository platform Admin.

• The Mend SaaS API → Port 443

• The wss-remediate server Docker container port as configured by the user → The default is 8080.

The wss-remediate Docker Container

The wss-remediate Docker container communicates with the following components using the following ports:

• Inbound:

  • Requests are received via a single port (default is 8080) from the wss-app Docker container

• Outbound:

  • Internally:

    • Your repository platform instance over https (default port is 443)

  • Externally:

    • Private/public package registries (npmjs/pypi/ruby gems, etc.) which use the standard ports

Repository Platform

Your repository platform instance requires the ability to communicate with the following components using the following ports:

• wss-<integration_type>-app Docker container → Recommended is 5678

• wss-remediate Docker container → Recommended is 8080

• wss-scanner Docker container → Recommended is 9393

NOTE: All port numbers on the Docker containers are the user’s choice

Enabling HTTPS Support for the Webhook Interceptor

The app container supports two ways of enabling HTTPS protocol for the webhook interceptor:

• Using Java KeyStore containing valid certificate and private key, as environment variables:

  • WS_KEYSTORE_FILE_PATH - path to the keystore file

  • WS_KEYSTORE_PASSWORD - password for the keystore file

• Directly provide a certificate and private key files, as environment variables:

  • WS_HTTPS_CERT_FILE_PATH - path to the certificate file

  • WS_HTTPS_KEY_FILE_PATH - path to the private key file

App Container Startup Check

Upon startup, the app container provides a clear indication of the connectivity status between itself and the remediate container, the repository platform (SCM) API, and the Mend application server. The startup check also validates the activation key provided in the initial configuration. If needed, error messages are displayed. Each check results in one of three status types, as listed here:

• SUCCESS

• FAILED

• SKIPPED

When all checks are finished, a summary table will be written to the log, for example:

Environmental Variables

To view the environmental variables and their configurations related to the Mend Repo Integrations, refer to the Environment Variables for Self-Hosted Integrations documentation.

Automate Secret Encryption for Private Registries

You can encrypt secrets from the CLI, using the curl, echo, jq, gpg, grep and tr CLI programs.
Here is an example:

curl https://app.renovatebot.com/renovate.pgp --output renovate.pgp
echo -n '{"o":"your-organization", "r":"your-repository (optional)", "v":"your-secret-value"}' | jq . -c | gpg --encrypt -a --recipient-file renovate.pgp | grep -v '^----' | tr -d '\n'

The above script uses:

• curl to download the Mend Renovate hosted app's public key

• echo to echo a JSON object into jq

• jq to validate the JSON and then compact it

• gpg to encrypt the contents

• grep and tr to extract the encrypted payload which we will use

The jq step is optional, you can leave it out if you wish. Its primary value is validating that the string you echo to gpg is valid JSON and compact.

Encrypted secrets usually have a single organization. But you may encrypt a secret with more than one organization, for example, org1,org2. This way the secret can be used in both the org1 and org2 organizations.

For more information on how to use secrets for private packages, refer to the Private package support documentation.