Renovate Deployment Guide - Self-Managed Repository Integrations

This is a guide to be used for a Self-Hosted Repo Integration Renovate deployment and will address both Remediate and Renovate functionality. It is supplemental to the official Remediate documentation and does not duplicate it.

Remediate automates pull requests (PRs) for vulnerable OSS libraries
Renovate automates PRs for out-of-date libraries, Dockerfiles, GitHub actions, K8s and ansible manifest files, both OSS and internal private libraries published to private repositories such as Artifactory, Nexus, Google Artifact Registry, etc.

Both capabilities are included in the application deployed on the Remediate container(s).

Terminology

To avoid confusion, the following terminology will be used in this guide:

Term	Description
R/R	The combined functionality of both Remediate and Renovate including the deployed sub-system (e.g., the R/R container[s])
Remediate	Automated PR functionality for vulnerable OSS dependencies
Renovate	Functionality for updating out-of-date dependencies (both OSS and internal)
PRs	Git Pull Requests

Overview

When using a global configuration and R/R is turned on, it will be enabled globally on all repos, in essence, a built-in "big bang". With scaling and production constraints in mind, a R/R deployment should be conducted in two phases, at a minimum:

Initial Pilot
General Availability (GA)

R/R Scalability

For scalability purposes, R/R supports two operating modes:

Server - single instance that orchestrates R/R activity via an in-memory job queue
Workers - pull and process jobs off the Server queue

There can be only one R/R Server instance (container/pod) per Self-Hosted Repo integration environment.

By default, a R/R container runs in both modes. For planning purposes, a single R/R worker can handle up to ~1000 repos. Actual performance and scalability vary greatly depending on the size of repos, number of dependencies, R/R configuration, and container infrastructure.

Before the GA phase, a customer will need to adjust their R/R container deployment to include a single replica R/R container pod (K8s service) configured for Server mode and multiple replica R/R container pod configured as R/R workers. R/R mode is controlled by container environment variables.

R/R Deployment Strategy

Mend recommends the following initial R/R deployment strategy:

Remediate Disabled
Renovate Enabled
Renovate Opt-in Configuration
Dependency Dashboard Enabled
Automated PRs Disabled
Merge Confidence Enabled
Smart Merge Control
Daily R/R Cron Schedule

Details are provided in the table below:

Feature	Description / Rationale
Remediate Disabled	Since Remediate PRs cannot be disabled, Remediate should be disabled to avoid overwhelming build infrastructure. Fortunately, Renovate provides more flexibility and capabilities which easily offset the loss of Remediate functionality.
Renovate Enabled	Renovate should be enabled selectively during the initial pilot and globally for GA (see Opt-In Mode)
Renovate Opt-In Configuration	Renovate can be enabled globally, but with a configuration that disables almost all R/R activity until enabled by a local repo override. This is recommended to avoid a “big bang” deployment that overwhelms infrastructure and api rate limits. Application teams can be instructed to enable Renovate functionality in a phased sequence by adding a pre-configured renovate.json file to their local repositories, thus “opting in”. This renovate.json override will have no impact on Mend’s security SCA functionality.
Dependency Dashboard Enabled	When opting in, we recommend dev teams utilize Renovate’s Dependency Dashboard for tracking and manually triggering update Pull Requests. The Dependency Dashboard is a special Git Issue automatically opened by Renovate which tracks out-of-date dependencies and allows developers to trigger Renovate update PRs.
Automated PRs Disabled	To avoid overwhelming build infrastructure, we recommend disabling automatic Pull Request creation, and utilize the Dependency Dashboard to manually trigger PR creation.
Merge Confidence Enabled	Merge confidence should be enabled with explicit inline rules which will avoid unnecessary GitHub lookups that would otherwise impact GH api rate limits.
Smart Merge Control	Allows users to control how Renovate PRs are delivered based on Merge Confidence values including update success, age and adoption.
Daily R/R Cron Schedule	R/R schedules a background job per repo to refresh out of date library versions, re-base outstanding PRs, and other background operations. By default, these jobs run hourly. To minimize the impact on the GitHub api rate limit, we recommend limiting these jobs to once per day at midnight. This schedule is configured via a R/R container environment variable (see below).

Ultimately, this deployment strategy allows R/R to be enabled globally, while minimizing build system infrastructure and api rate limit impact.

Initial Pilot

The goals of the initial pilot are:

Demonstrate and test basic Renovate capabilities on a small set of production repositories
Validate recommended Renovate global configuration and local repo override capabilities

Scope

A customer should identify a dozen or so repositories to include in the initial Renovate pilot.

R/R Configuration

For the initial pilot, R/R should NOT be enabled globally and will require pilot repos to be allowed to deploy .whitesource files overriding the global configuration to enable Renovate. This is recommended to minimize system risk during the initial pilot.

Specific configurations are below can be found at https://github.com/mend-toolkit/mend-examples

repo-config.json
- Override allow list for pilot repos
- Global renovate configuration
.whitesource
- Example config override to enable Renovate for pilot repos
renovate.json
- Example Renovate override configuration for pilot repos

If the customer has transitioned to container-based package manager configuration for private registry access (away from hostRules) before the Initial Pilot, see changes outlined in step #3 below

Container Deployment Changes

To prepare for a Renovate expansion and scaling, the customer should update their R/R container deployment to include a single R/R Server pod, and R/R Worker pod with multiple replicas.

Here’s a list of K8s deployment changes required:

Replace the existing R/R K8s service with two new services according to this documentation:
1. Remediate-Server
2. Remediate-Worker
Decrease the scheduled R/R cron job from default hourly to daily at midnight
1. Add the following environment variable to the Remediate-Server pod:

Environment Variable	Value
SCHEDULER_CRON	0 0 * * *

Add the necessary package manager configurations for private registry and public repo mirroring
1. https://github.com/mend-toolkit/mend-examples/tree/main/Repo-Integration/Private-Registries-No-HostRules
Deploy a Redis cluster for R/R scaling and Github API usage reduction
1. Redis Deployment Guide - Self-Managed Repository Integrations
Implement R/R queue monitoring via the /status API endpoint
1. Documentation to be added

General Availability (GA)

After a successful pilot, a customer can enable R/R functionality in their global configuration.