Dynamic Tool Installation Mechanism

Overview

Dynamic tool installation is a key feature within Mend Repository Integrations, designed to tackle the challenge of managing programming language and package manager versions during project scans. Instead of limiting versions of the used tools to the predetermined static one, Mend determines the necessary versions based on your project's specifications and installs them as needed.

How It Works

When scanning projects, Mend Repository Integrations uses “Runinstall” for dynamic tool installation at runtime. Runinstall intercepts tool commands issued by the Scanner, such as "poetry install" or "mvn install". Instead of relying on the scanner to understand every tool and version, the intelligence is pushed down to the dynamic tool installation layer.

Here's a simplified breakdown of the process:

The scanner issues a tool command to the operating system, such as “mvn”.
Runinstall intercepts the command.
It evaluates the project's specifications to determine the required versions of package managers.
If necessary, it installs the required versions.
Finally, it passes on the command to the installed tool for execution.

In cases where the dynamic tool installation fails to detect or install the required versions, a fallback mechanism is in place. Mend Repository Integrations revert to using the last installed version of the tool (the one that is statically specified in Scanner), ensuring the scanning process continues smoothly without interruption.

The Mend Scanner often issues multiple commands to the same tool in a row in the same repository. To avoid doing duplicate work, Runinstall remembers the repo and tool it most recently installed and will skip over the same again for subsequent commands.

Usage and configuration for Self-hosted integrations

Environment variables in the Scanner using the RUNINSTALL_ prefix are used to control the functionality.

Enabling Runinstall

In order to install languages and package managers dynamically at runtime, you need open internet access to reach the various servers involved (GitHub.com, Apache.org, etc.). If your containers cannot connect to the internet, this tool will not work.

To enable the Dynamic package manager install for selected package managers, you must uncomment these lines from the Dockerfile.

CODE

## Uncomment the below to enable Dynamic package manager install for selected package managers
##
# COPY --from=containerbase/runinstall:20240327 /home/ubuntu/bin/runinstall ${USER_HOME}/.local/bin/runinstall
# RUN ln -s ${USER_HOME}/.local/bin/runinstall ${USER_HOME}/.local/bin/mvn
# RUN ln -s ${USER_HOME}/.local/bin/runinstall ${USER_HOME}/.local/bin/pipenv
# RUN ln -s ${USER_HOME}/.local/bin/runinstall ${USER_HOME}/.local/bin/poetry

Note: You don't need to uncomment all three tools. You can choose just one or two of them if preferred.

Common Configuration

Name

Description

RUNINSTALL_MATCH

Required. Comma-delimited strings/paths.

Use this configuration option to slowly test/opt-in to Runinstall.

The strings provided will be matched against the git remote for the repository, for example the remote might be like “https://github.com/someorg/somerepo.git”

Runinstall will work only with git remotes matching one of these strings. Examples:

“someorg/somerepo” - match the repo of this name
“someorg/somerepo,someorg/otherrepo” - match two repos
“someorg/” - match all repos in the organization

RUNINSTALL_KEY_ID

Optional. AWS Key ID, set this in order to send logging to CloudWatch (akin to AWS_ACCESS_KEY_ID)

RUNINSTALL_ACCESS_KEY

Optional. AWS Secret Access Key, set this in order to send logging to CloudWatch (akin to AWS_SECRET_ACCESS_KEY)

Note: It is recommended that you set up an AWS CloudWatch instance to receive Runinstall logs at, because otherwise you won’t know if/when the dynamic install is working. Logs will be sent to the group name and stream name of “runinstall”.

Because the Mend Scanner parses stdout/stderr from tools, Runinstall cannot send its own logs there as it would interfere. Therefore it supports this “out of band” logging to CloudWatch using the CloudWatch Logs API instead of stdout/stderr.

Dynamic Tool Supported Package Managers

Maven
Poetry
Pipenv