Operator Guides on Spinnaker

Custom CAs for Webhooks

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Webhook stages enable Spinnaker to make HTTP(S) calls to external web services. If the configured webhook URL has the https:// scheme, Spinnaker will use TLS to communicate with the external server. Spinnaker will attempt to validate the certificate presented by the server by building a chain of trust back to a trusted certification authority (CA) and will refuse to connect if the certificate cannot be validated.

By default Spinnaker uses the trust store provided by the JVM as its source of trusted CAs. The default behavior is sufficient for webhooks to public-facing servers where it is possible to build a chain of trust back to a root CA. Internal servers, however, may have certificates issued by a company-specific CA that is not trusted by a root CA. Webhooks to these servers over https:// will fail using the default configuration.

Custom Job Stages

Mon, 01 Jan 0001 00:00:00 +0000

For users of Kubernetes or Titus, Custom Job Stages offer a native solution for enhancing Spinnaker’s functionality with the power of containers. Custom Job Stages enable operators to extend Spinnaker without having to maintain a custom build of Orca.

Note: Custom Job stages are only supported for the Kubernetes and Titus providers.

How it works

At its core, Custom Job Stages use the Run Job stage to start a Task (Titus) or Job (Kubernetes) to perform some type of work. These jobs are defined as configuration for Orca and registered when the application starts. When used in a pipeline, Orca takes this configuration, generates a Run Job stage and then executes it. To support a more native experience, we provide configuration options for things like stage name, description and parameters so that this stage feels less like a Run Job stage and more like an out of the box stage.

Custom Webhook Stages

Mon, 01 Jan 0001 00:00:00 +0000

Custom webhook stages provide a simple, yet powerful, way of adding custom stages to Spinnaker. These stages are typically used to make quick API calls to an external system as part of a pipeline. Instead of extending the various components through code, users can simply add configuration to Orca for these stages. They appear in the UI as if they were a native stage.

Creating a Custom Webhook Stage

To set up a custom webhook stage, you’ll need to modify the orca-local.yml configuration file. The webhook.preconfigured property allows you to define multiple webhook stages. Here’s an example configuration:

Echo: Cassandra to In-Memory

Mon, 01 Jan 0001 00:00:00 +0000

Echo’s scheduler can be run completely in-memory. On startup or redeploy, echo will check cron schedules to see if it needs to retroactively execute any missed triggers. This migration only requires configuration changes.

1. Disable Cassandra in echo.yml

spinnaker:
  cassandra:
    enabled: false

2. Enable in-memory backend in echo.yml

spinnaker:
  inMemory:
    enabled: true

3. Enable the scheduler compensation job in echo.yml

scheduler:
  compensationJob:
    enabled: true
    windowMs: 1800000 # optional

The windowMs property dictates how far in the past echo will look to find missed schedules. By default this is 30 minutes.

Front50: Cassandra to Object Store (S3, Azure, or GCS)

Mon, 01 Jan 0001 00:00:00 +0000

Cassandra is no longer an actively maintained or supported persistence store.

A migration to S3, GCS, or AZS is recommended.

1. Create a bucket and folder

Make up a bucket name (or account name if using Azure) that is consistent with the naming policies for the underlying storage service. For purposes of this document, we’ll pick ${USER}-spinnaker since many storage services (including Amazon S3, Google GCS, and Azure AZS) require globally unique names.

Front50: Cassandra to Redis

Mon, 01 Jan 0001 00:00:00 +0000

1. Install Redis

2. Disable Cassandra in front50.yml

cassandra:
  enabled: false

3. Enable Redis in front50.yml

spinnaker:
  redis:
    enabled: true

4. Export existing applications, pipelines, strategies, notifications and projects

#!/bin/sh

rm applications.json
curl http://FRONT50_HOSTNAME:FRONT50_PORT/global/applications | json_pp > applications.json

rm pipelines.json
curl http://FRONT50_HOSTNAME:FRONT50_PORT/pipelines | json_pp > pipelines.json

rm strategies.json
curl http://FRONT50_HOSTNAME:FRONT50_PORT/strategies | json_pp > strategies.json

rm notifications.json
curl http://FRONT50_HOSTNAME:FRONT50_PORT/notifications | json_pp > notifications.json

rm projects.json
curl http://FRONT50_HOSTNAME:FRONT50_PORT/v2/projects | json_pp | jq '._embedded.projects' > projects.json

5. Deploy new Front50

6. Import applications, pipelines, strategies, notifications and projects

#!/bin/sh

curl -X POST -H "Content-type: application/json" --data-binary @"notifications.json" http://FRONT50_HOSTNAME:FRONT50_PORT/notifications/batchUpdate
curl -X POST -H "Content-type: application/json" --data-binary @"strategies.json" http://FRONT50_HOSTNAME:FRONT50_PORT/strategies/batchUpdate
curl -X POST -H "Content-type: application/json" --data-binary @"pipelines.json" http://FRONT50_HOSTNAME:FRONT50_PORT/pipelines/batchUpdate
curl -X POST -H "Content-type: application/json" --data-binary @"applications.json" http://FRONT50_HOSTNAME:FRONT50_PORT/global/applications/batchUpdate
curl -X POST -H "Content-type: application/json" --data-binary @"projects.json" http://FRONT50_HOSTNAME:FRONT50_PORT/v2/projects/batchUpdate

Hiding Stages

Mon, 01 Jan 0001 00:00:00 +0000

Stages that are not provider-specific will be available by default. To hide specific stages from end-users, set the hiddenStages property in Deck’s settings-local.js to a list of the keys of stages you wish to hide. For example, to hide the Gremlin and Travis stages, include the following in settings-local.js:

 window.spinnakerSettings.hiddenStages = ['gremlin', 'travis'];

Orca: Redis to SQL Migration

Mon, 01 Jan 0001 00:00:00 +0000

If you are not migrating an existing Orca deployment, refer to Orca SQL Setup instead.

Migrate from Redis to SQL

Migrating without downtime from Redis to SQL is a three-step process:

When DualExecutionRepository is running, writes will be routed to either Redis or SQL. Executions will only be migrated to SQL once they’ve completed (either successfully or terminally): This keeps the migration story simple. As such, the migration agents will need to run for awhile. At Netflix, we ran the migration cluster for two weeks, as we had long pipeline executions due to canaries. You may only need to run the migration cluster for an hour.