ClickHouse

ClickHouse Cloud is a cloud-based distributed column-oriented database service designed for online analytical processing (OLAP) that allows generation of analytical data reports in real-time. The ClickHouse integration enables Relyance AI to discover and analyze data across your ClickHouse Cloud organizations, services, and metrics.

The ClickHouse integration provides:

• Organization Discovery: Scan all organizations, members, and invitations
• Service Monitoring: Discover services and their configurations
• Metrics Analysis: Extract Prometheus metrics from ClickHouse services
• Data Inspection: Analyze data schemas and identify personal information types
• Service Mapping: Visualize service-to-service communication flows

Prerequisites

Before connecting ClickHouse to Relyance AI, you must:

1. Have an active ClickHouse Cloud account
2. Have administrative access to create API keys
3. Create a dedicated API key for Relyance AI integration
4. (Optional) Set up database user credentials if enabling direct database inspection

Authentication Setup

Step 1: Create API Credentials in ClickHouse Cloud

ClickHouse uses API key-based authentication to secure access to your organizations and services.

1. Log in to your ClickHouse Cloud Console
2. Navigate to Organization Settings in the left-hand menu
3. Click on API Keys in the settings navigation
4. Click Create New API Key
5. Configure the API key:
   • Name: Enter a descriptive name (e.g., "Relyance AI Integration")
   • Expiration date: Set to "Never" or choose an appropriate expiration
   • Roles: Select "Admin" role (or appropriate role with read permissions)
   • Allow access from: Choose "Anywhere" for simplicity, or "Specific locations" for enhanced security
6. Click Generate API Key
7. Important: Copy both the Key ID and Key Secret immediately. The secret will not be shown again.

Make sure to securely store both values:

Key ID: (copy from the screen)
Key Secret: (copy from the screen - click the eye icon to reveal)

Step 2: (Optional) Create Database User for Data Inspection

If you want Relyance AI to perform data inspection on your ClickHouse databases:

1. Connect to your ClickHouse service using a SQL client
2. Create a dedicated read-only user for Relyance:

CREATE USER relyance_user IDENTIFIED WITH sha256_password BY 'your_secure_password';

1. Grant necessary read permissions:

-- Grant read access to specific databases
GRANT SELECT ON database_name.* TO relyance_user;

-- Or grant read access to all databases
GRANT SELECT ON *.* TO relyance_user;

-- Grant SHOW permissions to list databases and tables
GRANT SHOW DATABASES ON *.* TO relyance_user;
GRANT SHOW TABLES ON *.* TO relyance_user;

1. Verify the user can connect and query:

-- Test with:
SHOW DATABASES;
SHOW TABLES FROM database_name;
SELECT count(*) FROM database_name.table_name LIMIT 1;

Security Note:

This user only needs READ permissions. Do not grant INSERT, UPDATE, DELETE, or administrative privileges.

Integration Configuration in Relyance AI

Step 3: Add ClickHouse Integration

1. Log in to your Relyance AI account
2. Navigate to Settings→ Integrations
3. In the Vendor Integrations tab, search for "ClickHouse"
4. Click the ClickHouse integration card to open its connections.

Step 4: Configure Connection Settings

In the Basic Auth configuration form, fill in the following fields:

Product (auto-filled)

from_clickhouse

Key ID (required) Paste the Key ID you copied from ClickHouse Cloud Console.

Key Secret (required) Paste the Key Secret you copied from ClickHouse Cloud Console.

Database User (optional) Enter the database username if you created one for data inspection (e.g., relyance_user or default). Leave empty if you only want to scan organization and service metadata.

Database Password (optional) Enter the database password for the user above. Leave empty if not using database inspection.

Database Configuration (optional) Configure which databases to scan using JSON format:

{
  "allow_list": ["production_db", "analytics_db"],
  "block_list": ["test_db", "development_db"]
}

Configuration options:

• allow_list: Only scan databases in this list (empty array means scan all)
• block_list: Exclude specific databases from scanning

Example configurations:

Scan all databases:

{
  "allow_list": [
  ],
  "block_list": [
  ]
}

Scan only specific databases:

{
  "allow_list": [
    "user_data",
    "transactions"
  ],
  "block_list": [
  ]
}

Scan all except specific databases:

{
  "allow_list": [
  ],
  "block_list": [
    "temp_data",
    "cache"
  ]
}

Step 5: Configure Data Inspection (Optional)

The Data Inspection feature allows Relyance AI to analyze actual data content to identify personal information types and categories.

• Enable Data Inspection: Check this box to enable deep data scanning. Relyance will sample data from tables to identify PI types.
• Minimum Confidence Level: Set the minimum confidence threshold for PI detection. Possible values: Very Low, Low, Medium, High, Very High. Recommended: High for production environments.

Note:

Data inspection requires database credentials to be configured in the previous step.

Step 6: Configure Asset Discovery

Enable Asset Discovery: Check this box to enable automatic discovery of ClickHouse services. This creates a service map showing inter-service communication.

Step 7: Save and Connect

1. Click Authenticate to finalize the integration
2. Relyance will begin the initial scan of your ClickHouse environment
3. You'll see a confirmation on the Vendor Integrations page showing the connection is active

The integration will show a green "Connected" status with the connection date.

What Data Does Relyance AI Collect?

Relyance AI will discover and analyze:

The Authentication step asks for:

• Product: Enter the product name for the APM service; required.
• Key ID: required; held as a secret.
• Key Secret: required; held as a secret.
• Database User: optional.
• Database password: optional; held as a secret.
• Database Configuration: required; a JSON value; the field is pre-filled with the shape to complete.

Organization Level

• Organization IDs and names
• Organization members and their roles
• Pending invitations
• API keys metadata (not secrets)

Service Level

• Service names and IDs
• Service configurations
• Service providers (AWS, GCP, Azure)
• Service regions and endpoints

Metrics Level

• Prometheus metrics from services
• Service-to-service communication patterns
• Performance metrics
• Health status indicators

Database Level (if data inspection enabled)

• Database names
• Table schemas
• Column names and data types
• Sample data for PI classification
• Data volume statistics

Important:

Relyance AI only reads metadata and samples data. It does not store or transmit your actual business data.

Discovered Assets

After connecting ClickHouse, you'll see the following assets in Relyance AI:

In the Data Map

• ClickHouse Organizations: Top-level container assets
• ClickHouse Services: Individual database services
• Service Hosts: Cloud provider infrastructure
• Service Names: Logical service identifiers
• Data Flows: Service-to-service communication paths

In Asset Explorer

• Tables and columns from scanned databases
• Personal Information classifications
• Data category mappings
• Service dependencies

Permissions Reference

The ClickHouse API key requires the following permissions:

For database inspection, the database user needs:

Troubleshooting

Connection Fails

Error: "Invalid API credentials"

• Verify Key ID and Key Secret are correct
• Check that the API key hasn't been revoked
• Ensure the key has sufficient permissions

Error: "Organization access denied"

• Verify the API key has organization-level read permissions
• Check that your ClickHouse account is active

Database Inspection Fails

Error: "Authentication failed for database user"

• Verify database username and password are correct
• Test the credentials directly using a SQL client
• Check that the user hasn't been dropped or disabled

Error: "Permission denied on database"

• Verify the user has SELECT permissions
• Check database allow_list/block_list configuration
• Ensure SHOW permissions are granted

Metrics Collection Issues

Error: "Cannot retrieve Prometheus metrics"

• Verify services are running and healthy
• Check that Prometheus metrics are enabled on services
• Ensure API key has metrics read permissions

Slow Scan Performance

If the initial scan takes too long:

• Reduce the number of databases in allow_list
• Add large temporary databases to block_list
• Adjust sampling percentage in Asset Discovery settings
• Consider disabling data inspection for very large databases

Security Best Practices

1. Use Dedicated API Keys: Create a separate API key specifically for Relyance AI
2. Principle of Least Privilege: Only grant read permissions required for scanning
3. Rotate Credentials: Periodically rotate API keys and database passwords
4. Monitor Access: Review ClickHouse audit logs for Relyance AI access patterns
5. Restrict Database Access: Use allow_list to limit which databases are scanned
6. Secure Secrets: Never commit API keys or passwords to source control

Updating Configuration

To modify the integration settings:

1. Navigate to Settings→ Integrations
2. Find the ClickHouse integration
3. Click Edit
4. Update the desired fields
5. Click Re-Authenticate to apply updates

Changes take effect on the next scheduled scan or when you manually trigger a sync.

Manage this integration with Terraform

Connections for this integration can be managed as code with the Relyance Terraform provider. Non-secret fields go in auth.params; secret fields go in auth.secrets_wo, which is write-only — never stored in Terraform state. Rotate secrets by bumping auth.secrets_wo_version.

resource "relyance_integration_connection" "clickhouse" {
  vendor = "clickhouse"
  name   = "<your connection name>"

  auth = {
    method = "username-password"
    params = {
      product = "from_clickhouse"
      database_config = jsonencode({
        allow_list = [
          ""
        ]
        block_list = [
          ""
        ]
      })
      data_storage_location = "us"
    }
    # Secret fields are write-only: sent to Relyance, never stored in state.
    secrets_wo = {
      key_id = var.clickhouse_key_id
      key_secret = var.clickhouse_key_secret
      db_password = var.clickhouse_db_password
    }
    secrets_wo_version = 1
  }

  scans = { "data-inspection" = { enabled = true } }
}