Skip to main content

GCP Deployment Guide

This guide provides instructions on how to deploy the Sodot Exchange API Vault on Google Cloud Platform (GCP) using the Terraform module provided by Sodot.
It covers both the initial setup and the upgrade process for an existing deployment.

Deployment Architecture​

The Exchange API Vault is designed to provide a secure and scalable environment for managing your exchange API keys using Multi-Party Computation (MPC) technology.

The deployment consists of the following services:

  1. Three Vertex servers that work together as a cluster to securely store and manage your API keys.
  2. A Relay Server that facilitates secure communication between your applications and the Vertex cluster.
  3. A Kubernetes pod that serves the management UI of the Exchange API Vault.
  4. The required databases and networking components required to support the above services.

The deployment is managed using Terraform, which automates the provisioning and configuration of all necessary resources on GCP.
Each Vertex server is deployed within a Google Kubernetes Engine (GKE) cluster, utilizing confidential computing features to enhance security and limit access to sensitive data stored in the external database.

The deployment is also built to be easily customizable, allowing you to adjust configurations such as instance types, storage options and networking settings to fit your organization's needs.

Requirements​

Before starting the installation process, please make sure to perform the following prerequisites:

  1. Install the terraform CLI.
  2. Install the gcloud CLI tool for GCP.
  3. Make sure you are logged in (i.e gcloud auth application-default login).
  4. Make sure your account has permissions to create resources (e.g. clusters or load balancers) and the required APIs are enabled in your GCP project(s):
    • Kubernetes Engine API
    • Cloud Key Management Service (KMS) API
    • Cloud SQL Admin API
    • Compute Engine API
  5. Obtain Sodot Artifactory credentials with access to Terraform modules, Helm charts, and Docker images (ARTIFACTORY_USER, ARTIFACTORY_TOKEN).
note

This guide assumes you're running the provided Terraform module as the root module, although most steps are applicable when running it as a submodule as well.

Security Notice

This guide assumes that the deploying user has access to create and manage all of the Vertex servers.

For a better separation of duties and to limit administrative access, you may split the deployment process into different GCP projects, each managed by a different human user. If you choose to deploy each component separately, follow the instructions in the Vertex GCP Deployment Guide for each Vertex server, and the Relay GCP Deployment Guide for the Relay server. The Terraform module shown in this guide can be used as a reference for the required resources and configurations for connecting the separate components together.

Initial Setup Process​

The initial setup process consists of the following steps:

  1. Collecting the installation parameters
  2. Obtaining a TLS certificate
  3. Obtaining the Terraform module
  4. Applying the Terraform module to your cloud environment
  5. Setting the DNS Record for URL_ENDPOINT to point to the Vertex
  6. Verifying the deployment

1. Collecting Parameters​

At this step you should collect the following pieces of information: At the end of each bullet we put in parentheses a capitalized name which will be used to refer to the value being set throughout the rest of this guide.

General Deployment parameters:

  • Deployment name (DEPLOYMENT_NAME) used as a prefix for all created resources, e.g. my-eav
  • Sodot Artifactory credentials (ARTIFACTORY_USER, ARTIFACTORY_TOKEN)

Relay parameters:

  • Public DNS name for the Relay (RELAY_DNS_ADDRESS)
  • Pre-configured API key for the Relay (RELAY_API_KEY) encoded as KEY,NAME where KEY is a base64url (not plain base64) encoding of 16 random bytes and NAME is the key's display name. Example key format appears below.
  • TLS certificate name in GCP Certificate Manager for the Relay (RELAY_TLS_CERT_NAME)
  • Relay Helm chart version to deploy (RELAY_HELM_CHART_VERSION), e.g. 3.1.16
Relay Key Encoding

To generate and encode the Relay key, you can use the following command:

# 16-byte key in hex -> base64url (no padding changes needed for this key length)
echo -n '00112233445566778899aabbccddeeff' | xxd -r -p | base64 | tr '+/' '-_'
# => ABEiM0RVZneImaq7zN3u_w==

Pass the value as relay_api_key=ABEiM0RVZneImaq7zN3u_w==,AdminKey.

General Vertex cluster parameters:

  • Vertex Helm chart version to deploy (VERTEX_HELM_CHART_VERSION), e.g. 1.15.3

Per Vertex parameters (for each Vertex of the three in the cluster):

  • GCP project ID for the Vertex resources (PROJECT_ID)
  • GCP region for the Vertex resources (LOCATION), e.g. us-central1
  • TLS certificate name in GCP Certificate Manager for each Vertex (TLS_CERT_NAME)
  • Public DNS name that will point to each Vertex (VERTEX_X_DNS_ADDRESS)

UI parameters:

  • UI admin email address (UI_ADMIN_EMAIL), e.g. admin@example.com
  • UI Helm chart version to deploy (UI_HELM_CHART_VERSION), e.g. 1.15.3
  • TLS certificate name in GCP Certificate Manager for the UI (UI_TLS_CERT_NAME)
  • Public DNS name for the UI (UI_DNS_ADDRESS), e.g. sodot-eav.example.com

Authentication parameters:

You will need to choose the authentication method for the UI access.
Currently Google Login and Entra ID are supported:

Google Login

For this, you will need to follow the instructions in the Google Authentication Configuration guide to register the required applications in your Google project and obtain the following values:

  • 3 Application IDs for the UI application (GOOGLE_CLIENT_ID_X).
    Note that you will need to register 3 separate applications, one for each Vertex server.

2. Obtaining a TLS Certificate​

For each public endpoint (Relay, each Vertex DNS address and UI), create or import a Classic Certificate in GCP Certificate Manager. Keep the certificate names for the Relay (RELAY_TLS_CERT_NAME) and for each Vertex (TLS_CERT_NAME).

3. Obtaining the Terraform Module​

  1. Fetch the desired release (replace <VERSION> with a tag such as 1.15.3):
curl -u "<ARTIFACTORY_USER>:<ARTIFACTORY_TOKEN>" \
  -O "https://repo.sodot.dev/artifactory/sodot-terraform-modules/sodot/exchange-api-vault/google/<VERSION>.zip"

Alternatively, you can use the Artifactory GUI instead.

  1. Unzip and make it your root module:
unzip <VERSION>.zip -d exchange-api-vault
cd exchange-api-vault

4. Applying the Terraform Module to Your Cloud Environment​

Initialize Terraform and apply the module with your collected parameters.

To initialize Terraform, run:

terraform login repo.sodot.dev

terraform init

Now, set the required variables either via a terraform.tfvars file or by passing them as command-line arguments to terraform apply. We recommend using command-line arguments for sensitive values such as tokens.

An example for applying the Terraform module using command-line variables is shown below:

terraform apply \
  -var "name=<DEPLOYMENT_NAME>" \
  -var "artifactory_username=<ARTIFACTORY_USER>" \
  -var "artifactory_password=<ARTIFACTORY_TOKEN>" \
  -var "vertex_helm_chart_version=<VERTEX_HELM_CHART_VERSION>" \
  -var "ui_helm_chart_version=<UI_HELM_CHART_VERSION>" \
  -var "ui_dns_address=<UI_DNS_ADDRESS>" \
  -var "ui_gcp_managed_certificate_name=<UI_TLS_CERT_NAME>" \
  -var "relay_helm_chart_version=<RELAY_HELM_CHART_VERSION>" \
  -var "relay_dns_address=<RELAY_DNS_ADDRESS>" \
  -var "relay_api_key=<RELAY_API_KEY>" \
  -var "relay_gcp_managed_certificate_name=<RELAY_TLS_CERT_NAME>" \
  -var 'vertices=[
    { gcp_project_id="<GCP_PROJECT_1>", gcp_region="<GCP_REGION_1>", gcp_managed_certificate_name="<VERTEX_0_TLS_CERT_NAME>", dns_address="<VERTEX_0_DNS_ADDRESS>", google_client_id="<GOOGLE_CLIENT_ID_0>", ui_admin={ email="<UI_ADMIN_EMAIL>", type="google" } },
    { gcp_project_id="<GCP_PROJECT_2>", gcp_region="<GCP_REGION_2>", gcp_managed_certificate_name="<VERTEX_1_TLS_CERT_NAME>", dns_address="<VERTEX_1_DNS_ADDRESS>", google_client_id="<GOOGLE_CLIENT_ID_1>", ui_admin={ email="<UI_ADMIN_EMAIL>", type="google" } },
    { gcp_project_id="<GCP_PROJECT_3>", gcp_region="<GCP_REGION_3>", gcp_managed_certificate_name="<VERTEX_2_TLS_CERT_NAME>", dns_address="<VERTEX_2_DNS_ADDRESS>", google_client_id="<GOOGLE_CLIENT_ID_2>", ui_admin={ email="<UI_ADMIN_EMAIL>", type="google" } }
  ]'

What gets created:

  • For each Vertex:
    • A GKE cluster with confidential nodes only (as configured), Cloud SQL Postgres instance, KMS key ring key
    • A Vertex Helm release
    • A Kubernetes Ingress with your provided TLS certificate and hostname
  • One Relay deployment via Relay Terraform module
  • UI deployment within the Relay GKE cluster

5. Setting DNS Records​

After terraform apply completes, IP addresses will be output for the Relay, each Vertex instance in the cluster and the UI pod. Create A records for each DNS name you provided, pointing to the corresponding IP addresses.

6. Verifying the Deployment​

Verify that the Relay is reachable and the API key works:

curl -H "Authorization: Bearer <RELAY_API_KEY>" https://RELAY_DNS_ADDRESS/create_room/2
# Expect: 200 OK with a new random room ID string

Verify that each Vertex is reachable via its health endpoint:

curl -vv https://VERTEX_X_DNS_ADDRESS/health
# Expect: 200 OK with empty_ body

Upgrade Process​

To upgrade an existing deployment of the Exchange API Vault on GCP you can upgrade all or only some of the components that make up the deployment.
Follow these steps to upgrade your deployment:

  1. Pull the new desired version of the Terraform module from Sodot Artifactory.
  2. In case you wish to upgrade the Vertex used, choose your desired version of the Vertex image from the Sodot Release Notes. Update the vertex_helm_chart_version variable in your Terraform configuration to the desired version.
  3. In case you wish to upgrade the UI serving image, choose a desired version from the Sodot Release Notes. Make sure that the vertex image version you are upgrading to is compatible with the UI image version. Update the ui_image_tag variable in your Terraform configuration to the desired version.
  4. In case you wish to upgrade the Relay server, choose a desired version of the Relay Helm chart from the Sodot Release Notes. Update the relay_helm_chart_version variable in your Terraform configuration to the desired version.
  5. Run terraform plan to review the changes that will be applied.
  6. Run terraform apply to apply the changes and upgrade the deployment.
UI Upgrade Recommendation

For upgrading UI versions that do not include a requirement to upgrade the Vertex version in their changelog announcement, the UI can be upgraded independently without impacting the Vertex deployment.

Upgrade Strategy

During the upgrade process, a rolling upgrade strategy is used to prevent downtime. This means that only when the new version of a Vertex server is verified to be healthy, the previous version is terminated.