Skip to main content

AWS Deployment Guide

This guide provides instructions on how to deploy the Sodot Exchange API Vault on Amazon Web Services (AWS) using the Terraform module provided by Sodot.
It covers both the initial setup and the upgrade process for an existing deployment.

Deployment Architecture​

The Exchange API Vault is designed to provide a secure and scalable environment for managing your exchange API keys using Multi-Party Computation (MPC) technology.

The deployment consists of the following services:

  1. Three Vertex servers that work together as a cluster to securely store and manage your API keys.
  2. A Relay Server that facilitates secure communication between your applications and the Vertex cluster.
  3. A Kubernetes pod that serves the management UI of the Exchange API Vault.
  4. The required databases and networking components required to support the above services.

The deployment is managed using Terraform, which automates the provisioning and configuration of all necessary resources on AWS.
Each Vertex server is deployed as an EC2 Auto Scaling Group with AWS Nitro Enclaves, utilizing confidential computing features to enhance security and limit access to sensitive data stored in the external database.

The deployment is also built to be easily customizable, allowing you to adjust configurations such as instance types, storage options and networking settings to fit your organization's needs.

Requirements​

Before starting the installation process, please make sure to perform the following prerequisites:

  1. Install the terraform CLI.
  2. Install the aws CLI tool for AWS.
  3. Configure AWS CLI profiles for each of the 3 AWS accounts (e.g., eav-vertex-0, eav-vertex-1, eav-vertex-2).
  4. Make sure you are logged in with proper credentials for all three accounts.
  5. Make sure your accounts have permissions to create resources (e.g. VPCs, EC2 instances, EKS clusters, load balancers):
    • VPC and networking resources (subnets, security groups, NAT gateways)
    • EC2 instances and Auto Scaling Groups
    • Elastic Kubernetes Service (EKS)
    • RDS databases
    • Network Load Balancers (NLB) and Application Load Balancers (ALB)
    • AWS Certificate Manager (ACM)
    • AWS Key Management Service (KMS)
    • AWS Secrets Manager
  6. Obtain Sodot Artifactory credentials with access to Terraform modules, Helm charts, and Docker images (ARTIFACTORY_USER, ARTIFACTORY_TOKEN).
note

This guide assumes you're running the provided Terraform module as the root module, although most steps are applicable when running it as a submodule as well.

Security Notice

This guide assumes that the deploying user has access to create and manage all of the Vertex servers.

For a better separation of duties and to limit administrative access, you may split the deployment process into different AWS accounts, each managed by a different human user. If you choose to deploy each component separately, follow the instructions in the Vertex AWS Deployment Guide for each Vertex server, and the Relay AWS Deployment Guide for the Relay server. The Terraform module shown in this guide can be used as a reference for the required resources and configurations for connecting the separate components together.

Initial Setup Process​

The initial setup process consists of the following steps:

  1. Collecting the installation parameters
  2. Obtaining TLS certificates
  3. Obtaining the Terraform module
  4. Applying the Terraform module to your cloud environment
  5. Setting the DNS Records to point to the load balancers
  6. Verifying the deployment

1. Collecting Parameters​

At this step you should collect the following pieces of information: At the end of each bullet we put in parentheses a capitalized name which will be used to refer to the value being set throughout the rest of this guide.

General Deployment parameters:

  • Deployment name (DEPLOYMENT_NAME) used as a prefix for all created resources, e.g. my-eav
  • Sodot Artifactory credentials (ARTIFACTORY_USER, ARTIFACTORY_TOKEN)

Relay parameters:

  • Public DNS name for the Relay (RELAY_DNS_ADDRESS)
  • Pre-configured API key for the Relay (RELAY_API_KEY) encoded as KEY,NAME where KEY is a base64url (not plain base64) encoding of 16 random bytes and NAME is the key's display name. Example key format appears below.
  • ACM certificate ARN for the Relay LoadBalancer (RELAY_LB_ACM_CERTIFICATE_ARN)
  • Relay Helm chart version to deploy (RELAY_HELM_CHART_VERSION), e.g. 3.1.17
Relay Key Encoding

To generate and encode the Relay key, you can use the following command:

# 16-byte key in hex -> base64url (no padding changes needed for this key length)
echo -n '00112233445566778899aabbccddeeff' | xxd -r -p | base64 | tr '+/' '-_'
# => ABEiM0RVZneImaq7zN3u_w==

Pass the value as relay_api_key=ABEiM0RVZneImaq7zN3u_w==,AdminKey.

General Vertex cluster parameters:

  • Vertex image tag to deploy (VERTEX_IMAGE_TAG), e.g. 1.17.0

Per Vertex parameters (for each Vertex of the three in the cluster):

  • AWS account profile name for the Vertex resources (AWS_ACCOUNT_PROFILE), e.g. eav-vertex-0
  • AWS region for the Vertex resources (AWS_REGION), e.g. us-east-1
  • ACM certificate ARN for each Vertex NLB (LB_ACM_CERTIFICATE_ARN)
  • Public DNS name that will point to each Vertex (VERTEX_X_DNS_ADDRESS)

UI parameters:

  • UI admin email address (UI_ADMIN_EMAIL), e.g. admin@example.com
  • UI Helm chart version to deploy (UI_HELM_CHART_VERSION), e.g. 1.16.0
  • ACM certificate ARN for the UI LoadBalancer (UI_LB_ACM_CERTIFICATE_ARN)
  • Public DNS name for the UI (UI_DNS_ADDRESS), e.g. sodot-eav.example.com

Authentication parameters:

You will need to choose the authentication method for the UI access.
Currently Google Login and Entra ID are supported:

Google Login

For this, you will need to follow the instructions in the Google Authentication Configuration guide to register the required applications in your Google project and obtain the following values:

  • 3 Application IDs for the UI application (GOOGLE_CLIENT_ID_X).
    Note that you will need to register 3 separate applications, one for each Vertex server.

2. Obtaining TLS Certificates​

For each public endpoint (Relay, each Vertex DNS address and UI), create or import an ACM Certificate in AWS Certificate Manager in the appropriate region.

  • Vertex 0 certificate must be in the region specified for Vertex 0
  • Vertex 1 certificate must be in the region specified for Vertex 1
  • Vertex 2 certificate must be in the region specified for Vertex 2
  • Relay and UI certificates must be in the same region as the first Vertex (where the EKS cluster will be created)

Keep the certificate ARNs for the Relay (RELAY_LB_ACM_CERTIFICATE_ARN), UI (UI_LB_ACM_CERTIFICATE_ARN), and for each Vertex (LB_ACM_CERTIFICATE_ARN).

3. Obtaining the Terraform Module​

  1. Fetch the desired release (replace <VERSION> with a tag such as 1.15.3):
curl -u "<ARTIFACTORY_USER>:<ARTIFACTORY_TOKEN>" \
  -O "https://repo.sodot.dev/artifactory/sodot-terraform-modules/sodot/exchange-api-vault/aws/<VERSION>.zip"

Alternatively, you can use the Artifactory GUI instead.

  1. Unzip and make it your root module:
unzip <VERSION>.zip -d exchange-api-vault
cd exchange-api-vault

4. Applying the Terraform Module to Your Cloud Environment​

Initialize Terraform and apply the module with your collected parameters.

To initialize Terraform, run:

terraform login repo.sodot.dev

terraform init

Now, set the required variables either via a terraform.tfvars file or by passing them as command-line arguments to terraform apply. We recommend using command-line arguments for sensitive values such as tokens.

An example for applying the Terraform module using command-line variables is shown below:

terraform apply \
  -var "app_name=<DEPLOYMENT_NAME>" \
  -var "artifactory_username=<ARTIFACTORY_USER>" \
  -var "artifactory_password=<ARTIFACTORY_TOKEN>" \
  -var "vertex_image_tag=<VERTEX_IMAGE_TAG>" \
  -var "ui_helm_chart_version=<UI_HELM_CHART_VERSION>" \
  -var "ui_image_tag=<UI_IMAGE_TAG>" \
  -var "ui_dns_address=<UI_DNS_ADDRESS>" \
  -var "ui_lb_acm_certificate_arn=<UI_LB_ACM_CERTIFICATE_ARN>" \
  -var "relay_helm_chart_version=<RELAY_HELM_CHART_VERSION>" \
  -var "relay_address=<RELAY_DNS_ADDRESS>" \
  -var "relay_api_key=<RELAY_API_KEY>" \
  -var "relay_lb_acm_certificate_arn=<RELAY_LB_ACM_CERTIFICATE_ARN>" \
  -var 'vertices=[
    { aws_account_profile="<AWS_ACCOUNT_PROFILE_0>", aws_region="<AWS_REGION_0>", lb_acm_certificate_arn="<LB_ACM_CERTIFICATE_ARN_0>", dns_address="<VERTEX_0_DNS_ADDRESS>", google_client_id="<GOOGLE_CLIENT_ID_0>", ui_admin={ email="<UI_ADMIN_EMAIL>", type="google" } },
    { aws_account_profile="<AWS_ACCOUNT_PROFILE_1>", aws_region="<AWS_REGION_1>", lb_acm_certificate_arn="<LB_ACM_CERTIFICATE_ARN_1>", dns_address="<VERTEX_1_DNS_ADDRESS>", google_client_id="<GOOGLE_CLIENT_ID_1>", ui_admin={ email="<UI_ADMIN_EMAIL>", type="google" } },
    { aws_account_profile="<AWS_ACCOUNT_PROFILE_2>", aws_region="<AWS_REGION_2>", lb_acm_certificate_arn="<LB_ACM_CERTIFICATE_ARN_2>", dns_address="<VERTEX_2_DNS_ADDRESS>", google_client_id="<GOOGLE_CLIENT_ID_2>", ui_admin={ email="<UI_ADMIN_EMAIL>", type="google" } }
  ]'

What gets created:

  • For each Vertex:
    • A VPC with private and public subnets across multiple availability zones
    • An EC2 Auto Scaling Group with AWS Nitro Enclaves for confidential computing
    • An RDS PostgreSQL instance for Vertex data
    • A Network Load Balancer with your provided ACM certificate and hostname
    • KMS keys for encryption
    • AWS Secrets Manager secrets for sensitive data
  • One shared EKS cluster in the first Vertex's AWS account
  • One Relay deployment via Relay Terraform module on the EKS cluster
  • UI deployment within the EKS cluster
  • One shared RDS PostgreSQL instance for audit logs in the first Vertex's account

5. Setting DNS Records​

After terraform apply completes, DNS names for the Network Load Balancers will be output for the Relay, each Vertex instance in the cluster and the UI pod. Create CNAME records for each DNS name you provided, pointing to the corresponding load balancer DNS names.

Alternatively, you can create A records using Route 53 Alias records if managing DNS in AWS.

6. Verifying the Deployment​

Verify that the Relay is reachable and the API key works:

curl -H "Authorization: Bearer <RELAY_API_KEY>" https://<RELAY_DNS_ADDRESS>/create_room/2
# Expect: 200 OK with a new random room ID string

Verify that each Vertex is reachable via its health endpoint:

curl -vv https://<VERTEX_X_DNS_ADDRESS>/health
# Expect: 200 OK with empty body

You can also access the EKS cluster to verify Relay and UI deployments:

# Configure kubectl for the EKS cluster
aws eks update-kubeconfig \
  --name <DEPLOYMENT_NAME>-eks \
  --profile <AWS_ACCOUNT_PROFILE_0> \
  --region <AWS_REGION_0>

# Check Relay pods
kubectl get pods -n relay

# Check UI pods
kubectl get pods -n eav-ui

Upgrade Process​

To upgrade an existing deployment of the Exchange API Vault on AWS you can upgrade all or only some of the components that make up the deployment.
Follow these steps to upgrade your deployment:

  1. Pull the new desired version of the Terraform module from Sodot Artifactory.
  2. In case you wish to upgrade the Vertex used, choose your desired version of the Vertex image from the Sodot Release Notes. Update the vertex_image_tag variable in your Terraform configuration to the desired version.
  3. In case you wish to upgrade the UI serving image, choose a desired version from the Sodot Release Notes. Make sure that the vertex image version you are upgrading to is compatible with the UI image version. Update the ui_image_tag variable in your Terraform configuration to the desired version.
  4. In case you wish to upgrade the Relay server, choose a desired version of the Relay Helm chart from the Sodot Release Notes. Update the relay_helm_chart_version and relay_image_tag variables in your Terraform configuration to the desired version.
  5. Run terraform plan to review the changes that will be applied.
  6. Run terraform apply to apply the changes and upgrade the deployment.
UI Upgrade Recommendation

For upgrading UI versions that do not include a requirement to upgrade the Vertex version in their changelog announcement, the UI can be upgraded independently without impacting the Vertex deployment.

Upgrade Strategy

During the Vertex upgrade process, a rolling update strategy is used via EC2 Auto Scaling Groups to prevent downtime. This means that only when the new EC2 instance with the updated Vertex version is verified to be healthy, the previous instance is terminated. For Relay and UI components running on EKS, Kubernetes performs rolling updates of the pods.