Software Telemetry

Continuous Secure Monitoring for GRAX

GRAX monitors deployed software via egress-only log and telemetry streaming technologies. This allows the GRAX team to best ensure reliability without need for ingress connections or direct environment access, both of which have security implications. This document explains both of the methods in use today by GRAX and describes the data contained by both.

GRAX Metrics

Metrics are quantitative expressions of application performance, health, and configuration; their numerical nature assists in detection of failures. GRAX calculates and streams metrics continuously as the application runs. Telemetry regarding system resources (CPU, RAM, Disk, etc.) is streamed continuously, but the GRAX application will submit specialized telemetry events occassionally. These include, in part:

  • At time-of-boot
  • At time-of-update
  • Changing configuration of:
    • Auto Backup
    • Search
    • History Stream
    • General Settings
  • Processing tasks change status (created, started, ended, failed, etc.):
    • Archives
    • Restores

GRAX Logs

Logs are more detailed and structured than application metrics; they're traditionally used for investigating issues -- not detecting them. GRAX streams logs with an authenticated egress-only connection, making application logs available to GRAX Engineering for the sake of supportability and bug-fixing. Here are some related key details:

  • GRAX logging never contains your Salesforce records or any system secrets.
  • Logs utilize the same authenticated hq.grax.com connection as licensing and metrics.
  • Access to these logs is tightly controlled internally at GRAX.
  • Logs are only retained for 15 days by default.
  • This functionality cannot be disabled.

Logging uses a forward-only collector and will not transmit logs from an earlier point in time.

What Do GRAX Logs Contain?

Logs emitted by GRAX never contain customer CRM data, PII, or secrets. These logs are intentionally designed to be useful for GRAX engineers; as such, they contain:

  • Source Function Names
  • Source File Names
  • Function Timing Information
  • API Request Methods
  • API Request Paths
  • CPU Performance Metrics / Profiles
  • Memory Performance Metrics / Profiles
  • Storage Performance Metrics / Profiles
  • Function Metadata (Object Names, Batch Sizes, Record Counts, etc.)

As you can see, the data logged within the GRAX logging system is strictly related to operation and performance of the GRAX application with no exposure of protected data at any time.

As stated above, logs from GRAX are intended for consumption by GRAX engineers. We do not publish documentation nor provide training on understanding the internals of the GRAX application. This means that logs will not provide value to teams monitoring GRAX directly without the assistance of GRAX support.

Who Can View GRAX Logs?

Your logs are only visible to the engineers who directly support and manage operation of the GRAX application. For more information about security controls, audits, and compliance, see here.

Network Considerations

This functionality is required for the GRAX application to operate. As such, egress to hq.grax.com (3.232.229.75) is required at all times from the application. Without this access, the application will not boot or run; this is not configurable.

Data Security

GRAX takes security of customer data seriously. As such, none of your Salesforce data ever leaves the application environment. A breakdown of collected data follows:

  • Names of Salesforce objects (Standard and Custom) covered by backup and archive operations.
  • Number of records for Salesforce objects (Standard and Custom) covered by backup and archive operations.
  • GRAX backup/archive/restore configurations (schedule, start time, etc) and statuses.
  • Size, performance, and internal metrics for the proprietary GRAX storage layer in your storage bucket.
  • Size and performance metrics for the attached postgres database.
  • Total data size sent and received to/from Salesforce.
  • Feature enablement status (feature flags, feature access levels)
  • Structured application logging (optional)

No sensitive, classified, or restricted data or PII is included in telemetry communications. The content of backed up records is not inspected for telemetry, nor is it made available to any GRAX engineers.

All data is encrypted with HTTPS and TLS 1.2+ while in flight, and encrypted on disk when at rest. Access to the telemetry dataset is restricted within the GRAX team to only engineers whose roles require access.