ESP Platform Configuration

The esp-platform application is the core backend API for ESProfiler tenant deployments. It is built on Java and Spring Boot, managing tenant databases, distributed background tasks, integrations with central microservices, and AI-driven intelligence engines.

This guide provides a comprehensive breakdown of application.yml and its environment profiles (application-dev.yml, application-hop.yml), explaining how each configuration section works, how properties map to environment variables in AWS ECS / production deployments, and how individual tenants can be tailored for compliance, branding, feature entitlements, and security.

Configuration Architecture & Resolution Order

Spring Boot resolves configuration properties hierarchically. Values defined higher in the list override lower values:

┌────────────────────────────────────────────────────────┐
│  1. System Environment Variables (AWS ECS / Container) │  (Highest priority)
├────────────────────────────────────────────────────────┤
│  2. Active Profile Config (e.g. application-dev.yml)   │
├────────────────────────────────────────────────────────┤
│  3. Base Application Config (application.yml)          │  (Lowest priority)
└────────────────────────────────────────────────────────┘

Active Profiles

Tenant Customization & Configuration Levers

In production, each ESProfiler customer runs on a dedicated, isolated ECS service and database instance. You customize each tenant's behavior, branding, security boundaries, and feature set through environment variables injected into the tenant's ECS task definition.

                    ┌────────────────────────────────────────┐
                    │       Tenant ECS Task Definition       │
                    └───────────────────┬────────────────────┘
                                        │
     ┌──────────────────┬───────────────┴───────────────┬──────────────────┐
     ▼                  ▼                               ▼                  ▼
┌──────────────┐ ┌──────────────┐              ┌──────────────┐ ┌──────────────────┐
│   Identity   │ │  Compliance  │              │ Feature Set  │ │ Privacy & AI     │
│  & Routing   │ │  & Notices   │              │ Entitlements │ │ Observability    │
├──────────────┤ ├──────────────┤              ├──────────────┤ ├──────────────────┤
│Domain / Alias│ │Legal Banner  │              │Commercial    │ │Trace Redaction   │
│Client UUID   │ │Currency / VAT│              │Threat Engine │ │PostHog Analytics │
│TLS & Port    │ │S3 File Store │              │Deep Research │ │Gap Sensitivity   │
└──────────────┘ └──────────────┘              └──────────────┘ └──────────────────┘

1. Identity, Domain & CAS Single Sign-On

Each tenant has a unique hostname and OAuth2 client registration within the Central Authentication Service (CAS):

• ESP_EXTERNAL_DOMAIN: The vanity domain used by end users to access the platform (e.g., acme.own.esprofiler.com or custom enterprise domain security.acme.com).
• ESP_CLIENT_ID: The OAuth2 Client UUID registered in CAS representing this tenant instance.
• ESP_SECURITY_ORGANISATION_ALIAS: The organization slug in CAS used to route authentication to the customer's specific identity provider (e.g. Okta, Azure AD, Google Workspace).

2. UI Pinning & Custom Compliance Notices

The Single Page Application (SPA) delivery and login compliance banners can be tailored per tenant:

• ESP_SPA_TARGET_VERSION: Allows pinning a tenant to a specific frontend release (e.g., v3-9-12) for enterprise change-freeze windows, or leaving it blank/dynamic to track latest releases.
• ESP_SPA_SHOW_NOTICE (true/false): Controls whether a mandatory legal disclaimer is presented before users can interact with the system.
• ESP_SPA_NOTICE_MESSAGE: Custom legal text (e.g. UK Computer Misuse Act 1990, US DoD system banners, or company-specific Acceptable Use Policies).

3. Tenant Feature Entitlements & Module Toggles

Contract tiers dictate which platform modules are enabled for a tenant. Set any of these to false to restrict access:

ESP_MODULES_ENABLED_COMMERCIAL=true        # Vendor spend, contract renewal & cost analysis
ESP_MODULES_ENABLED_I_ENGINE=true          # AI Intelligence Engine & recommendations
ESP_MODULES_ENABLED_USER_ASSERTIONS=true   # User verification audits & control assertions
ESP_MODULES_ENABLED_VENDOR_REQUEST=true    # Vendor security assessment request workflows
ESP_MODULES_ENABLED_THREAT_ANALYTICS=true  # Threat intelligence & MITRE ATT&CK mappings
ESP_MODULES_ENABLED_DEEP_RESEARCH=true     # Autonomous agentic deep research capabilities

4. Localization & Currency Management

Multi-national tenants can define their base reporting currency:

• ESP_CURRENCY_TARGET: Target reporting currency for commercial spend aggregation (e.g., GBP, USD, EUR, AUD, CAD, JPY).
• ESP_CURRENCY_EXCHANGE_PROVIDERS: Feed providers for live exchange rate calculations (e.g. ECB, IMF).
• ESP_COST_CALCULATION_ANNUAL_DAYS_MULTIPLIER: Default is 365.25 for leap-year adjusted amortization, or set to 365.0 for standard non-leap calculations.

5. File Storage Isolation (Local vs S3)

Tenants can upload vendor documents, evidence files, and report attachments to dedicated S3 buckets:

• ESP_FILE_MODE: S3 (production) or LOCAL (development/testing).
• ESP_FILE_S3_BUCKET_NAME: S3 bucket name (e.g. customer.esprofiler.com or a dedicated tenant bucket esprofiler-tenant-acme-eu-west-2).
• ESP_FILE_S3_REGION: AWS Region hosting the bucket (e.g. eu-west-2, us-east-1).

6. Cryptographic Salt & Integration Secret Storage

The platform stores integration API keys (e.g., CrowdStrike, Splunk, Tenable, Sentinel) in an encrypted table using AES-256-GCM:

• ESP_SECRETS_PASSWORD: Master encryption key for the tenant's credential vault.
• ESP_SECRETS_SALT: A unique, per-tenant 32-character hexadecimal salt string.

7. AI Privacy, Telemetry & Intelligence Sensitivity

Depending on tenant data governance requirements (e.g., GDPR, HIPAA, banking secrecy), telemetry and LLM interaction capture can be adjusted:

• ESP_LLM_OBSERVATION_CONTENT_CAPTURE_MODE:
  • FULL: Standard mode — captures full prompt text, AI responses, and reasoning traces into Langfuse for analysis.
  • REDACTED: Automatically redacts personal identifying information (PII) and tenant identifiers from LLM traces.
  • METADATA_ONLY: Exports strictly execution latencies, token counts, and HTTP status codes without recording any prompt or completion text.
• ESP_NOTIFICATION_POSTHOG_ENABLED: Set to false for tenants requiring complete opt-out from frontend product telemetry.
• ESP_INTELLIGENCE_STACK_GAP_PHANTOM_THRESHOLD: Threshold (default 0.7 = 70%) below which a mapped capability with low product utilization is flagged as a "phantom gap".
• ESP_LANGFUSE_PROMPTS_LABEL: Selects the prompt bundle label to fetch from Langfuse (e.g., production, enterprise-v2, canary).

8. Custom GCP Project (Bring Your Own Cloud / BYOC AI)

For enterprise tenants who mandate running Gemini / Vertex AI through their own GCP billing project:

• ESP_GCP_PROJECT_ID: The customer-owned GCP Project ID.
• ESP_GCP_PRIVATE_KEY: Base64 encoded (B64:...) or PEM formatted Service Account Private Key.
• ESP_GCP_CLIENT_EMAIL: Service Account Email with Vertex AI User role.

Multi-Database Architecture

esp-platform employs a dual-database architecture: MariaDB / MySQL for core relational domain entities, and PostgreSQL with pgvector for embedding storage and semantic similarity search.

┌────────────────────────────────────────────────────────┐
│                      esp-platform                      │
└───────────────┬────────────────────────┬───────────────┘
                │                        │
       [ Relational JPA ]       [ Vector Data JPA ]
                │                        │
                ▼                        ▼
        ┌───────────────┐        ┌───────────────┐
        │ MySQL/MariaDB │        │  PostgreSQL   │
        │   (Port 3306) │        │   + PGVector  │
        │  Main Schema  │        │  (Port 25432) │
        └───────────────┘        └───────────────┘

Relational Database (MariaDB / MySQL / AWS Aurora)

spring:
  datasource:
    driverClassName: software.aws.rds.jdbc.mysql.Driver
    jdbc-url: jdbc:${esp.database.type}://${esp.database.host}:${esp.database.port}/${esp.database.name}
    username: ${esp.database.username}
    password: ${esp.database.password}
    name: ESPDataSource
    hikari:
      maximum-pool-size: 30
      connection-timeout: 10000
      max-lifetime: 1140000
  jpa:
    open-in-view: false
    generate-ddl: false
    hibernate:
      ddl-auto: none
    properties:
      hibernate:
        auto_quote_keyword: true
        jdbc:
          time_zone: UTC
        type:
          json_format_mapper: org.hibernate.type.format.jackson.Jackson3JsonFormatMapper
      org:
        hibernate:
          envers:
            store_data_at_delete: true
            audit_table_suffix: _aud

  liquibase:
    enabled: true
    change-log: /db/changelog/db.changelog-master.xml
    database-change-log-lock-table: DB_CHANGELOG_LOCK
    database-change-log-table: DB_CHANGELOG
    parameters:
      annualDaysMultiplier: ${esp.cost-calculation.annual-days-multiplier}

Configuration Details

• AWS RDS JDBC Driver: Uses software.aws.rds.jdbc.mysql.Driver to provide enhanced failover detection for Aurora MySQL clusters.
• Hikari Connection Pool: Capped at 30 connections with a max lifetime of 19 minutes (1140000 ms) to prevent stale connections across AWS NAT gateways.
• Hibernate Envers: Tracks audit history for security compliance using the _aud table suffix and records deleted records.
• Liquibase Migrations: Automatically runs schema migrations on startup against DB_CHANGELOG. The parameter annualDaysMultiplier is passed into changelog scripts for cost computation formulas.

Vector Database (PostgreSQL + PGVector)

Vector embeddings for semantic intelligence are managed under a separate pgvector datasource namespace:

pgvector:
  datasource:
    jdbc-url: jdbc:postgresql://${esp.database.pg.host}:${esp.database.pg.port}/${esp.database.pg.name}
    username: ${esp.database.pg.username}
    password: ${esp.database.pg.password}
    hikari:
      schema: esp_platform
    table-name: vector_store
  liquibase:
    enabled: true
    change-log: /db/changelog/postgres-db/db.changelog-master.xml
    database-change-log-lock-table: POSTGRES_DB_CHANGELOG_LOCK
    database-change-log-table: POSTGRES_DB_CHANGELOG
    default-schema: esp_platform

Server & Runtime Infrastructure

These settings control the embedded web server, thread pool management, and request constraints.

server:
  port: 8088
  shutdown: graceful
  http2:
    enabled: true
  max-http-request-header-size: 40KB

spring:
  application:
    name: platform-api
  threads:
    virtual:
      enabled: true
  lifecycle:
    timeout-per-shutdown-phase: 20s
  jackson:
    time-zone: UTC
  servlet:
    multipart:
      max-file-size: 100MB
      max-request-size: 100MB
  task:
    execution:
      propagate-context: true

• server.port (8088): The internal HTTP listening port for the container. In AWS ECS, ALB target groups map inbound container traffic to this port.
• server.shutdown: graceful & spring.lifecycle.timeout-per-shutdown-phase: 20s: Allows in-flight requests up to 20 seconds to complete gracefully before the container terminates during deployments.
• spring.threads.virtual.enabled: true: Enables Java Project Loom virtual threads for high-throughput concurrent I/O.
• spring.task.execution.propagate-context: true: Propagates Micrometer and LlmTraceContext across asynchronous @Async thread boundaries.

Distributed Task Scheduler (Quartz)

The platform runs clustered background jobs (e.g. periodic intelligence re-indexing, compliance sync, notification processing) using Quartz backed by the relational database.

spring:
  quartz:
    job-store-type: jdbc
    jdbc:
      initialize-schema: never
    properties:
      org:
        quartz:
          scheduler:
            instanceName: ESPClusteredScheduler
            instanceId: AUTO
          jobStore:
            driverDelegateClass: org.quartz.impl.jdbcjobstore.StdJDBCDelegate
            useProperties: false
            tablePrefix: QRTZ_
            isClustered: true
            clusterCheckinInterval: 20000
          threadPool:
            threadCount: 10

• isClustered: true: Allows multiple ECS task instances for the same tenant to safely share scheduled jobs without duplicate execution.
• clusterCheckinInterval: 20000: Instances heartbeat every 20 seconds to detect failed nodes and reassign jobs.

Central Services Mesh

esp-platform coordinates with other ESProfiler central backend services:

esp:
  mfs:
    url: https://api.esprofiler.com/mfs       # Module & Feature Service
  ces:
    url: https://api.esprofiler.com/ces       # Capability Engine Service
  cps:
    url: https://api.esprofiler.com/cps       # Customer Platform Service
  ccs:
    url: https://api.esprofiler.com/ccs       # Capability Catalog Service
  pes:
    url: https://api.esprofiler.com/pes       # Policy Engine Service
  task-hub:
    url: https://${esp.external-domain}/task-hub

In the dev profile, these are remapped to local development ports (localhost:8990 - 8997).

Complete Environment Variable Reference Matrix

The following matrix categorizes all supported environment variables into Tenant-Mandatory, Tenant-Customizable, and Shared Platform Defaults.

1. Tenant-Mandatory Variables (Must be unique for every tenant)

2. Tenant-Customizable Variables (Tailored per contract / tier)

3. Shared Platform Defaults (Common across standard deployments)

Tenant Configuration Archetypes (Examples)

Below are practical configuration examples illustrating how environment variables are set up for different customer profiles in AWS ECS.

Archetype A: Standard Commercial Tenant (UK / GBP)

{
  "name": "ESP_EXTERNAL_DOMAIN",
  "value": "fintech-uk.own.esprofiler.com"
},
{
  "name": "ESP_CLIENT_ID",
  "value": "7f83b2c1-4d5e-4a6b-9c8d-0e1f2a3b4c5d"
},
{
  "name": "ESP_SECURITY_ORGANISATION_ALIAS",
  "value": "fintech-uk"
},
{
  "name": "ESP_CURRENCY_TARGET",
  "value": "GBP"
},
{
  "name": "ESP_MODULES_ENABLED_COMMERCIAL",
  "value": "true"
},
{
  "name": "ESP_MODULES_ENABLED_THREAT_ANALYTICS",
  "value": "true"
},
{
  "name": "ESP_MODULES_ENABLED_DEEP_RESEARCH",
  "value": "true"
},
{
  "name": "ESP_LLM_OBSERVATION_CONTENT_CAPTURE_MODE",
  "value": "FULL"
},
{
  "name": "ESP_NOTIFICATION_POSTHOG_ENABLED",
  "value": "true"
}

Archetype B: Strict Compliance & Privacy Tenant (US Healthcare / USD)

Enforces prompt redaction in Langfuse, displays a mandatory HIPAA/Legal disclaimer modal, turns off product usage analytics, and aggregates spend in USD:

{
  "name": "ESP_EXTERNAL_DOMAIN",
  "value": "healthcorp.own.esprofiler.com"
},
{
  "name": "ESP_CLIENT_ID",
  "value": "3b2a1c0d-9e8f-4a5b-6c7d-8e9f0a1b2c3d"
},
{
  "name": "ESP_SECURITY_ORGANISATION_ALIAS",
  "value": "healthcorp-us"
},
{
  "name": "ESP_CURRENCY_TARGET",
  "value": "USD"
},
{
  "name": "ESP_SPA_SHOW_NOTICE",
  "value": "true"
},
{
  "name": "ESP_SPA_NOTICE_MESSAGE",
  "value": "WARNING: This system contains confidential patient and healthcare data. Unauthorized access or export is strictly prohibited and subject to legal prosecution under HIPAA and federal law."
},
{
  "name": "ESP_LLM_OBSERVATION_CONTENT_CAPTURE_MODE",
  "value": "REDACTED"
},
{
  "name": "ESP_NOTIFICATION_POSTHOG_ENABLED",
  "value": "false"
},
{
  "name": "ESP_INTELLIGENCE_STACK_GAP_PHANTOM_THRESHOLD",
  "value": "0.80"
}

Archetype C: Starter / Light Tier Tenant (Restricted Features)

Deploys a basic subscription where advanced modules like Autonomous Deep Research and Vendor Requests are disabled:

{
  "name": "ESP_EXTERNAL_DOMAIN",
  "value": "starter-co.own.esprofiler.com"
},
{
  "name": "ESP_CLIENT_ID",
  "value": "c8d9e0f1-2a3b-4c5d-6e7f-8a9b0c1d2e3f"
},
{
  "name": "ESP_MODULES_ENABLED_COMMERCIAL",
  "value": "true"
},
{
  "name": "ESP_MODULES_ENABLED_THREAT_ANALYTICS",
  "value": "true"
},
{
  "name": "ESP_MODULES_ENABLED_VENDOR_REQUEST",
  "value": "false"
},
{
  "name": "ESP_MODULES_ENABLED_DEEP_RESEARCH",
  "value": "false"
}

Troubleshooting Common Misconfigurations

1. Database Connection Timeout on Boot

• Symptom: HikariPool-1 - Connection is not available, request timed out after 10000ms.
• Remedy: Ensure the ECS Security Group allows outbound TCP on port 3306 (MySQL) and 5432 (PostgreSQL) to the RDS security group. Check that RDS instance endpoints resolve in the tenant VPC.

2. GCP Private Key Formatting Error

• Symptom: IllegalArgumentException: Invalid private key format or JSON parsing error.
• Remedy: When passing the GCP private key via ECS Task Definition environment variables, newline characters (\n) can get unescaped. Either encode the key in Base64 and prefix with B64: (e.g., B64:MIIEvQIBAD...), or store the multiline key directly in AWS Secrets Manager and reference it via ECS secret value ARN.

3. OpenTelemetry / Langfuse 4318 POST Error

• Symptom: Continuous Failed to export metrics: localhost:4318/v1/metrics connection refused.
• Remedy: Verify that management.otlp.metrics.export.enabled is set to false. Langfuse only accepts traces (/api/public/otel/v1/traces), not OpenTelemetry metrics.

4. Quartz Lock / Cluster Check-in Failure

• Symptom: ClusterManager: Error managing cluster: Failure obtaining db row lock: Table 'QRTZ_LOCKS' doesn't exist.
• Remedy: Verify that Liquibase successfully ran on startup and that the database user has permissions to create and lock Quartz tables.