Skip to main content

AWS PrivateLink

You can use AWS PrivateLink to provide connectivity between VPCs, AWS services, your on-premises systems, and ClickHouse Cloud without having your traffic go across the internet. This document describes how to connect to ClickHouse Cloud using AWS PrivateLink, and how to disable access to your ClickHouse Cloud services from addresses other than AWS PrivateLink addresses using ClickHouse Cloud IP Access Lists.

Only available in production environments

AWS PrivateLink is only available in ClickHouse Cloud Production services. Development services are not supported.

Please complete the following steps to enable AWS Private Link:

  1. Obtain Endpoint Service name.
  2. Create a service endpoint.
  3. Add Endpoint ID to ClickHouse Cloud organization.
  4. Add Endpoint ID to service(s) allow list.

Find complete Terraform example for AWS Private Link here.

Prerequisites​

Before you get started you will need:

  1. An AWS account.
  2. An API key with the necessary permissions to create and manage private links.

Steps​

Follow these steps to connect your ClickHouse Cloud to your AWS PrivateLink.

Obtain Endpoint Service name​

ClickHouse Cloud console​

In the ClickHouse Cloud console, open the service that you would like to connect via PrivateLink, then open the Settings menu. Click on the Set up private endpoint button. Copy the Service name for which will be used for setting up Private Link.

Private Endpoints

API​

First, set the following environment variables before running any commands:

REGION=<Your region code using the AWS format>
PROVIDER=aws
KEY_ID=<Your key ID>
KEY_SECRET=<Your key secret>
ORG_ID=<Your ClickHouse organization ID>
SERVICE_NAME=<Your ClickHouse service name>

Get the desired instance ID by filtering by region, provider, and service name:

export INSTANCE_ID=$(curl --silent --user ${KEY_ID:?}:${KEY_SECRET:?} \
https://api.clickhouse.cloud/v1/organizations/$ORG_ID/services | \
jq ".result[] | select (.region==\"${REGION:?}\" and .provider==\"${PROVIDER:?}\" and .name==\"${SERVICE_NAME:?}\") | .id " -r)

Obtain an AWS Service Name for your Private Link configuration:

curl --silent --user ${KEY_ID:?}:${KEY_SECRET:?} \
https://api.clickhouse.cloud/v1/organizations/${ORG_ID:?}/services/${INSTANCE_ID:?}/privateEndpointConfig | \
jq .result

This command should return something like:

{
    ...
    "endpointServiceId": "com.amazonaws.vpce.yy-xxxx-N.vpce-svc-xxxxxxxxxxxx",
    ...
}

Make a note of the endpointServiceId and move onto step 2.

Create a service endpoint​

Next, you need to create a service endpoint using the endpointServiceId from previous step. Open the the AWS console and Go to VPC → Endpoints → Create endpoints.

Select Other endpoint services and use the endpointServiceId you got from the previous step. Once you're done, click Verify service:

Next, select your VPC and subnets:

Select VPC and subnets

As an optional step, assign Security groups/Tags:

Ports

Make sure that ports 8443 and 9440 are allowed in the security group.

After creating the VPC Endpoint, make a note of the Endpoint ID value; you'll need it for an upcoming step.

VPC endpoint ID

AWS CloudFormation​

Make sure to use correct subnet IDs, security groups, and VPC ID.

Resources:
  ClickHouseInterfaceEndpoint:
    Type: 'AWS::EC2::VPCEndpoint'
    Properties:
      VpcEndpointType: Interface
      PrivateDnsEnabled: false
      ServiceName: <use endpointServiceId from 'Obtain AWS Service Name for Private Link' step>
      VpcId: vpc-vpc_id
      SubnetIds:
        - subnet-subnet_id1
        - subnet-subnet_id2
        - subnet-subnet_id3
      SecurityGroupIds:
        - sg-security_group_id1
        - sg-security_group_id2
        - sg-security_group_id3

Terraform​

resource "aws_vpc_endpoint" "this" {
  vpc_id            = var.vpc_id
  service_name      = "<use endpointServiceId from 'Obtain AWS Service Name for Private Link' step>"
  vpc_endpoint_type = "Interface"
  security_group_ids = [
    Var.security_group_id1,var.security_group_id2, var.security_group_id3,
  ]
  subnet_ids          = [var.subnet_id1,var.subnet_id2,var.subnet_id3]
  private_dns_enabled = false
}

Modify Private DNS Name for Endpoint​

This step injects private DNS zone <region code>.vpce.aws.clickhouse.cloud configuration into AWS VPC.

DNS resolver

If you use own DNS resolver, create a <region code>.vpce.aws.clickhouse.cloud DNS zone and point a wildcard record *.<region code>.vpce.aws.clickhouse.cloud to the Endpoint ID IP addresses.

AWS Console​

Navigate to VPC Endpoints, right click the VPC Endpoint, then select Modify private DNS name:

Endpoints menu

On the page that opens, select Enable private DNS names:

Modify DNS names

AWS CloudFormation​

Update the CloudFormation template and set PrivateDnsEnabled to true:

PrivateDnsEnabled: true

Apply the changes.

Terraform​

  • Change the aws_vpc_endpoint resource in Terraform code and set private_dns_enabled to true:
private_dns_enabled = true

Apply the changes.

Add Endpoint ID to ClickHouse Cloud organization​

ClickHouse Cloud console​

To add an endpoint to organization, proceed to the Add Endpoint ID to service(s) allow list step. Adding the Endpoint ID using the ClickHouse Cloud console to the services allow list automatically adds it to organization.

To remove an endpoint, open Organization details -> Private Endpoints and click the delete button to remove the endpoint.

endpoints

API​

Set the following environment variables before running any commands:

PROVIDER=aws
KEY_ID=<Key ID>
KEY_SECRET=<Key secret>
ORG_ID=<please set ClickHouse organization ID>
ENDPOINT_ID=<Endpoint ID from previous step>
REGION=<region code, please use AWS format>

Set the VPC_ENDPOINT environment variable using data from the previous step.

To add an endpoint, run:

cat <<EOF | tee pl_config_org.json
{
  "privateEndpoints": {
    "add": [
      {
        "cloudProvider": "aws",
        "id": "${ENDPOINT_ID:?}",
        "description": "An aws private endpoint",
        "region": "${REGION:?}"
      }
    ]
  }
}
EOF

curl --silent --user ${KEY_ID:?}:${KEY_SECRET:?} \
-X PATCH -H "Content-Type: application/json" \
https://api.clickhouse.cloud/v1/organizations/${ORG_ID:?} \
-d @pl_config_org.json

To remove an endpoint, run:

cat <<EOF | tee pl_config_org.json
{
  "privateEndpoints": {
    "remove": [
      {
        "cloudProvider": "aws",
        "id": "${ENDPOINT_ID:?}",
        "region": "${REGION:?}"
      }
    ]
  }
}
EOF

curl --silent --user ${KEY_ID:?}:${KEY_SECRET:?} \
-X PATCH -H "Content-Type: application/json" \
https://api.clickhouse.cloud/v1/organizations/${ORG_ID:?} \
-d @pl_config_org.json

Add Endpoint ID to service(s) allow list​

ClickHouse Cloud console​

In the ClickHouse Cloud console, open the service that you would like to connect via PrivateLink then navigate to Settings. Enter the Endpoint ID obtained from the previous step.

note

If you want to allow access from an existing PrivateLink connection, use the existing endpoint drop-down menu.

Private Endpoints

API​

You need to add an Endpoint ID to the allow-list for each instance that should be available using PrivateLink.

Set the following environment variables before running any commands:

PROVIDER=aws
KEY_ID=<Key ID>
KEY_SECRET=<Key secret>
ORG_ID=<please set ClickHouse organization ID>
ENDPOINT_ID=<Endpoint ID from previous step>
INSTANCE_ID=<Instance ID>

To add an endpoint ID to an allow-list:

cat <<EOF | tee pl_config.json
{
  "privateEndpointIds": {
    "add": [
      "${ENDPOINT_ID:?}"
    ]
  }
}
EOF

curl --silent --user ${KEY_ID:?}:${KEY_SECRET:?} \
-X PATCH -H "Content-Type: application/json" \
https://api.clickhouse.cloud/v1/organizations/${ORG_ID:?}/services/${INSTANCE_ID:?} \
-d @pl_config.json | jq

To remove an endpoint ID from an allow-list:

cat <<EOF | tee pl_config.json
{
  "privateEndpointIds": {
    "remove": [
      "${ENDPOINT_ID:?}"
    ]
  }
}
EOF

curl --silent --user ${KEY_ID:?}:${KEY_SECRET:?} \
-X PATCH -H "Content-Type: application/json" \
https://api.clickhouse.cloud/v1/organizations/${ORG_ID:?}/services/${INSTANCE_ID:?} \
-d @pl_config.json | jq

Each instance with configured a Private Link filter has a public and private endpoint. In order to connect to your service using PrivateLink you need to use the private endpoint privateDnsHostname.

note

The private DNS hostname is only available from your AWS VPC. Do not try to resolve a DNS host from a local machine.

Getting Private DNS Hostname​

ClickHouse Cloud console​

In the ClickHouse Cloud console, navigate to Settings. Click on the Set up private endpoint button. In the opened flyout, copy the DNS Name.

Private Endpoints

API​

Set the following environment variables before running any commands:

KEY_ID=<Key ID>
KEY_SECRET=<Key secret>
ORG_ID=<please set ClickHouse organization ID>
INSTANCE_ID=<Instance ID>
curl --silent --user ${KEY_ID:?}:${KEY_SECRET:?} \
https://api.clickhouse.cloud/v1/organizations/${ORG_ID:?}/services/${INSTANCE_ID:?}/privateEndpointConfig | \
jq .result

This should output something like:

{
  "endpointServiceId": "com.amazonaws.vpce.yy-xxxx-N.vpce-svc-xxxxxxxxxxxx",
  "privateDnsHostname": "xxxxxxx.yy-xxxx-N.vpce.aws.clickhouse.cloud"
}

In this example connection to xxxxxxx.yy-xxxx-N.vpce.aws.clickhouse.cloud host name will be routed to PrivateLink, but xxxxxxx.yy-xxxx-N.aws.clickhouse.cloud will be routed over the internet.

Troubleshooting​

If you require two or more AWS Private Links within the same AWS region, then please note: In ClickHouse, we have a VPC Endpoint service at a regional level. When you setup two or more VPC Endpoints in the same VPC - from the AWS VPC perspective - you are utilizing just a single AWS Private Link. In such a situation where you need two or more AWS Private Links configured within the same region, please just create just one VPC Endpoint in your VPC, and request that ClickHouse configure the same VPC Endpoint ID for all of your ClickHouse services in the same AWS region.

Connection to private endpoint timed out​

  • Please attach security group to VPC Endpoint.
  • Please verify inbound rules on security group attached to Endpoint and allow ClickHouse ports.
  • Please verify outbound rules on security group attached to VM which is used to connectivity test and allow connections to ClickHouse ports.

Private Hostname: Not found address of host​

  • Please check "Private DNS names" option is enabled, visit step for details

Connection reset by peer​

  • Most likely Endpoint ID was not added to service allow list, please visit step

Checking Endpoint filters​

Set the following environment variables before running any commands:

KEY_ID=<Key ID>
KEY_SECRET=<Key secret>
ORG_ID=<please set ClickHouse organization ID>
INSTANCE_ID=<Instance ID>
curl --silent --user ${KEY_ID:?}:${KEY_SECRET:?} \
-X GET -H "Content-Type: application/json" \
https://api.clickhouse.cloud/v1/organizations/${ORG_ID:?}/services/${INSTANCE_ID:?} | \
jq .result.privateEndpointIds

Connecting to a remote database​

Let's say you are trying to use the MySQL or PostgreSQL table functions in ClickHouse Cloud and connect to your database hosted in an Amazon Web Services (AWS) VPC. AWS PrivateLink cannot be used to enable this connection securely. PrivateLink is a one-way, unidirectional connection. It allows your internal network or Amazon VPC to connect securely to ClickHouse Cloud, but it does not allow ClickHouse Cloud to connect to your internal network.

According to the AWS PrivateLink documentation:

Use AWS PrivateLink when you have a client/server set up where you want to allow one or more consumer VPCs unidirectional access to a specific service or set of instances in the service provider VPC. Only the clients in the consumer VPC can initiate a connection to the service in the service provider VPC.

To do this, configure your AWS Security Groups to allow connections from ClickHouse Cloud to your internal/private database service. Check the default egress IP addresses for ClickHouse Cloud regions, along with the available static IP addresses.