Skip to main content

How we ensure production-safe tuning

DBtune prioritizes safety in production environments through built-in guardrails. These guardrails are designed to prevent downtime and performance degradation resulting from the tuning process. You control the thresholds that decide when a configuration is flagged as unsafe: the memory threshold is set in the agent, and the performance thresholds in the platform. These checks are performed both during and after tuning sessions.

Guardrails during a tuning session

These guardrails are in place to prevent unexpected issues arising from new configurations being actively tested by DBtune during a tuning session.

Native memory guardrail

During a tuning session, DBtune actively monitors total RAM usage using a built-in guardrail. By default, if a PostgreSQL configuration causes memory usage to exceed 90% of available RAM, DBtune automatically adjusts to avoid that configuration. This adjustment is a seamless part of the tuning process and requires no user intervention.

Users can customize this 90% memory threshold to match their specific safety requirements. To change it, add the following configuration in the dbtune.yaml agent config file:

guardrail_settings:
memory_threshold: 90

Or you can add the following environment variable:

DBT_MEMORY_THRESHOLD=90

When this memory usage guardrail is triggered (reaching the defining threshold), the agent immediately reverts to the default (baseline) configuration. To allow the system to stabilize and prevent production disruptions, the agent then waits for the duration of one "tuning iteration" before the backend suggests a new server configuration to test.

Native performance degradation guardrail

To ensure optimal performance and stability, DBtune includes a built-in guardrail that actively monitors the performance of each PostgreSQL server configuration being tested on a customer instance during the tuning session.

Each threshold is expressed as a multiplier of your baseline performance. Which guardrails apply depends on whether the session targets a Workload Fingerprint:

Session typeGuardrailTriggers whenDefault
StandardAverage query runtime (AQR)AQR rises above the multiple of baseline2 (AQR doubles)
StandardTransactions per second (TPS)TPS falls below the fraction of baseline0.05 (5% of baseline)
FingerprintFingerprint performanceFingerprint AQR rises above the multiple of baseline2
FingerprintAverage query runtime (AQR)Database-wide AQR rises above the multiple of baseline10
FingerprintTransactions per second (TPS)TPS falls below the fraction of baseline0.05 (5% of baseline)

Note how loose the TPS default is: at 0.05 it only fires if throughput collapses to a twentieth of baseline, so it catches a broken configuration rather than ordinary variation. Fingerprint sessions likewise allow database-wide AQR to reach 10 before tripping, while holding the fingerprint's own runtime to 2.

When this performance guardrail is triggered, DBtune will proceed by testing a new recommended configuration. Unlike the memory usage guardrail, which caused a reversion to the baseline configuration, this guardrail simply prompts DBtune to explore a different tuning path.

Adjusting the thresholds

The defaults above suit most workloads, but you can tighten or loosen each one:

  1. Navigate to your database's Tuning tab.
  2. Open the Advanced settings section.
  3. Adjust the guardrail thresholds for the session type you are running.

AQR multipliers accept values from 1 to 20; TPS multipliers accept values from 0 to 1. The form flags AQR multipliers below 1.5 as not recommended, since a tight threshold trips on ordinary workload variance.

Custom guardrails

For metrics DBtune does not watch natively, such as disk growth rate, you can add your own checks by writing a custom agent against the open-source DBtune agent.

Guardrails after a tuning session

Even after a tuning session ends, the DBtune agent remains active, continuously monitoring the database instance. Its purpose is to proactively alert users to any unhealthy server behavior that might emerge at any point in the future, even if it wasn’t apparent during the tuning process.

Native memory guardrail notification

Following the completion of a tuning session, DBtune continues to monitor total RAM usage with a dedicated guardrail. Unlike the active memory management during tuning, this post-tuning guardrail operates by notifying the user of any breaches in the defined threshold, without automatically altering server parameters. This aligns with DBtune's core principle of only modifying the PostgreSQL instance during active tuning sessions. As with the previous active tuning guardrail, this notification threshold is also adjustable. As soon as a guardrail threshold is reached, DBtune takes the following two actions:

  1. An email notification is sent to the user’s address.
  2. A notification is displayed to the user within the DBtune platform.
note

If you are using the self-hosted version of DBtune, you will need to configure your own SMTP email server to receive email notifications. You can do this by setting the appropriate environment variables in your deployment configuration. See the SMTP configuration example in the self-hosted documentation for more details.