Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 24 additions & 14 deletions products/feature-flagging/feature-flagging-api/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,22 +18,27 @@ The OpenFeature SDK (`dev.openfeature:sdk`) is included as a transitive dependen

### Evaluation metrics (optional)

To enable evaluation metrics (`feature_flag.evaluations` counter), add the OpenTelemetry SDK dependencies:
To enable evaluation metrics (`feature_flag.evaluations` counter), enable the Datadog Java agent's
OpenTelemetry metrics pipeline:

```shell
DD_METRICS_OTEL_ENABLED=true
```

The provider records metrics through the OpenTelemetry Metrics API. Add `opentelemetry-api` if your
application does not already use the OpenTelemetry API for custom metrics:

```xml
<dependency>
<groupId>io.opentelemetry</groupId>
<artifactId>opentelemetry-sdk-metrics</artifactId>
<version>1.47.0</version>
</dependency>
<dependency>
<groupId>io.opentelemetry</groupId>
<artifactId>opentelemetry-exporter-otlp</artifactId>
<artifactId>opentelemetry-api</artifactId>
<version>1.47.0</version>
</dependency>
```

Any OpenTelemetry API 1.x version is compatible. If these dependencies are absent, the provider operates normally without metrics.
The OpenTelemetry SDK and OTLP exporter are not required on the application classpath. The Datadog
Java agent collects the API metric and exports it through the same OTLP pipeline as other custom OTel
metrics.

## Usage

Expand All @@ -52,15 +57,20 @@ boolean enabled = client.getBooleanValue("my-feature", false,

## Evaluation metrics

When the OTel SDK dependencies are on the classpath, the provider records a `feature_flag.evaluations` counter via OTLP HTTP/protobuf. Metrics are exported every 10 seconds to the Datadog Agent's OTLP receiver.
When `DD_METRICS_OTEL_ENABLED=true` and the OpenTelemetry API is on the classpath, the provider
records a `feature_flag.evaluations` counter. The Datadog Java agent exports it to the Datadog
Agent's OTLP receiver using the configured OpenTelemetry metrics export interval.

### Configuration

| Environment variable | Description | Default |
|---|---|---|
| `OTEL_EXPORTER_OTLP_METRICS_ENDPOINT` | Signal-specific OTLP endpoint (used as-is) | — |
| `OTEL_EXPORTER_OTLP_ENDPOINT` | Generic OTLP endpoint (`/v1/metrics` appended) | — |
| (none set) | Default endpoint | `http://localhost:4318/v1/metrics` |
Configure the OTLP endpoint and protocol using the standard Datadog Java agent OpenTelemetry metrics
settings. For example, to export metrics over OTLP/gRPC:

```shell
DD_METRICS_OTEL_ENABLED=true
OTEL_EXPORTER_OTLP_ENDPOINT=http://<agent-host>:4317
OTEL_EXPORTER_OTLP_PROTOCOL=grpc
```

### Metric attributes

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -44,19 +44,13 @@ dependencies {
api("dev.openfeature:sdk:1.20.1")

compileOnly(project(":products:feature-flagging:feature-flagging-bootstrap"))
compileOnly(project(":utils:config-utils"))
compileOnly("io.opentelemetry:opentelemetry-api:1.47.0")
compileOnly("io.opentelemetry:opentelemetry-sdk-metrics:1.47.0")
compileOnly("io.opentelemetry:opentelemetry-exporter-otlp:1.47.0")

testImplementation(project(":products:feature-flagging:feature-flagging-bootstrap"))
testImplementation("io.opentelemetry:opentelemetry-api:1.47.0")
testImplementation("io.opentelemetry:opentelemetry-sdk-metrics:1.47.0")
testImplementation("io.opentelemetry:opentelemetry-exporter-otlp:1.47.0")
testImplementation(libs.bundles.junit5)
testImplementation(libs.bundles.mockito)
testImplementation(libs.moshi)
testImplementation("io.opentelemetry:opentelemetry-sdk-testing:1.47.0")
testImplementation("org.awaitility:awaitility:4.3.0")
}

Expand Down
21 changes: 1 addition & 20 deletions products/feature-flagging/feature-flagging-api/gradle.lockfile
Original file line number Diff line number Diff line change
Expand Up @@ -12,28 +12,14 @@ com.google.code.findbugs:jsr305:3.0.2=compileClasspath,spotbugs,testCompileClass
com.google.code.gson:gson:2.13.2=spotbugs
com.google.errorprone:error_prone_annotations:2.41.0=spotbugs
com.squareup.moshi:moshi:1.11.0=testCompileClasspath,testRuntimeClasspath
com.squareup.okhttp3:okhttp:4.12.0=testRuntimeClasspath
com.squareup.okio:okio-jvm:3.6.0=testRuntimeClasspath
com.squareup.okio:okio:1.17.5=testCompileClasspath
com.squareup.okio:okio:3.6.0=testRuntimeClasspath
com.squareup.okio:okio:1.17.5=testCompileClasspath,testRuntimeClasspath
com.thoughtworks.qdox:qdox:1.12.1=codenarc
commons-io:commons-io:2.20.0=spotbugs
de.thetaphi:forbiddenapis:3.10=compileClasspath
dev.openfeature:sdk:1.20.1=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.leangen.geantyref:geantyref:1.3.16=testRuntimeClasspath
io.opentelemetry:opentelemetry-api:1.47.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
io.opentelemetry:opentelemetry-context:1.47.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
io.opentelemetry:opentelemetry-exporter-common:1.47.0=testRuntimeClasspath
io.opentelemetry:opentelemetry-exporter-otlp-common:1.47.0=testRuntimeClasspath
io.opentelemetry:opentelemetry-exporter-otlp:1.47.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
io.opentelemetry:opentelemetry-exporter-sender-okhttp:1.47.0=testRuntimeClasspath
io.opentelemetry:opentelemetry-sdk-common:1.47.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
io.opentelemetry:opentelemetry-sdk-extension-autoconfigure-spi:1.47.0=testRuntimeClasspath
io.opentelemetry:opentelemetry-sdk-logs:1.47.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
io.opentelemetry:opentelemetry-sdk-metrics:1.47.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
io.opentelemetry:opentelemetry-sdk-testing:1.47.0=testCompileClasspath,testRuntimeClasspath
io.opentelemetry:opentelemetry-sdk-trace:1.47.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
io.opentelemetry:opentelemetry-sdk:1.47.0=testCompileClasspath,testRuntimeClasspath
jaxen:jaxen:2.0.0=spotbugs
net.bytebuddy:byte-buddy-agent:1.18.10=testCompileClasspath,testRuntimeClasspath
net.bytebuddy:byte-buddy:1.18.10=testCompileClasspath,testRuntimeClasspath
Expand Down Expand Up @@ -64,11 +50,6 @@ org.jacoco:org.jacoco.agent:0.8.14=jacocoAgent,jacocoAnt
org.jacoco:org.jacoco.ant:0.8.14=jacocoAnt
org.jacoco:org.jacoco.core:0.8.14=jacocoAnt
org.jacoco:org.jacoco.report:0.8.14=jacocoAnt
org.jetbrains.kotlin:kotlin-stdlib-common:1.9.10=testRuntimeClasspath
org.jetbrains.kotlin:kotlin-stdlib-jdk7:1.9.10=testRuntimeClasspath
org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.9.10=testRuntimeClasspath
org.jetbrains.kotlin:kotlin-stdlib:1.9.10=testRuntimeClasspath
org.jetbrains:annotations:13.0=testRuntimeClasspath
org.junit.jupiter:junit-jupiter-api:5.14.1=testCompileClasspath,testRuntimeClasspath
org.junit.jupiter:junit-jupiter-engine:5.14.1=testRuntimeClasspath
org.junit.jupiter:junit-jupiter-params:5.14.1=testCompileClasspath,testRuntimeClasspath
Expand Down
Original file line number Diff line number Diff line change
@@ -1,19 +1,13 @@
package datadog.trace.api.openfeature;

import datadog.trace.config.inversion.ConfigHelper;
import dev.openfeature.sdk.ErrorCode;
import io.opentelemetry.api.GlobalOpenTelemetry;
import io.opentelemetry.api.common.AttributeKey;
import io.opentelemetry.api.common.Attributes;
import io.opentelemetry.api.common.AttributesBuilder;
import io.opentelemetry.api.metrics.LongCounter;
import io.opentelemetry.api.metrics.Meter;
import io.opentelemetry.exporter.otlp.http.metrics.OtlpHttpMetricExporter;
import io.opentelemetry.sdk.metrics.SdkMeterProvider;
import io.opentelemetry.sdk.metrics.export.AggregationTemporalitySelector;
import io.opentelemetry.sdk.metrics.export.PeriodicMetricReader;
import io.opentelemetry.sdk.resources.Resource;
import java.io.Closeable;
import java.time.Duration;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

Expand All @@ -25,17 +19,6 @@ class FlagEvalMetrics implements Closeable {
private static final String METRIC_NAME = "feature_flag.evaluations";
private static final String METRIC_UNIT = "{evaluation}";
private static final String METRIC_DESC = "Number of feature flag evaluations";
private static final Duration EXPORT_INTERVAL = Duration.ofSeconds(10);

private static final String DEFAULT_ENDPOINT = "http://localhost:4318/v1/metrics";
// Signal-specific env var (used as-is, must include /v1/metrics path)
private static final String ENDPOINT_ENV = "OTEL_EXPORTER_OTLP_METRICS_ENDPOINT";
// Generic env var fallback (base URL, /v1/metrics is appended)
private static final String ENDPOINT_GENERIC_ENV = "OTEL_EXPORTER_OTLP_ENDPOINT";
// OTel standard env vars for service name
private static final String SERVICE_NAME_ENV = "OTEL_SERVICE_NAME";
private static final AttributeKey<String> SERVICE_NAME_KEY =
AttributeKey.stringKey("service.name");

private static final AttributeKey<String> ATTR_FLAG_KEY =
AttributeKey.stringKey("feature_flag.key");
Expand All @@ -48,75 +31,34 @@ class FlagEvalMetrics implements Closeable {
AttributeKey.stringKey("feature_flag.result.allocation_key");

private volatile LongCounter counter;
// Typed as Closeable to avoid loading SdkMeterProvider at class-load time
// when the OTel SDK is absent from the classpath
private volatile Closeable meterProvider;

FlagEvalMetrics() {
try {
String endpoint = ConfigHelper.env(ENDPOINT_ENV);
if (endpoint == null || endpoint.isEmpty()) {
String base = ConfigHelper.env(ENDPOINT_GENERIC_ENV);
if (base != null && !base.isEmpty()) {
endpoint = base.endsWith("/") ? base + "v1/metrics" : base + "/v1/metrics";
} else {
endpoint = DEFAULT_ENDPOINT;
}
}

OtlpHttpMetricExporter exporter =
OtlpHttpMetricExporter.builder()
.setEndpoint(endpoint)
.setAggregationTemporalitySelector(AggregationTemporalitySelector.deltaPreferred())
.build();

PeriodicMetricReader reader =
PeriodicMetricReader.builder(exporter).setInterval(EXPORT_INTERVAL).build();

// Build resource with service name from OTEL_SERVICE_NAME env var
// Resource.getDefault() only provides unknown_service:java, so we read env vars manually
Resource resource = buildResource();

SdkMeterProvider sdkMeterProvider =
SdkMeterProvider.builder().setResource(resource).registerMetricReader(reader).build();
meterProvider = sdkMeterProvider;

Meter meter = sdkMeterProvider.meterBuilder(METER_NAME).build();
Meter meter = GlobalOpenTelemetry.get().getMeterProvider().meterBuilder(METER_NAME).build();
counter =
meter
.counterBuilder(METRIC_NAME)
.setUnit(METRIC_UNIT)
.setDescription(METRIC_DESC)
.build();

log.debug("Flag evaluation metrics initialized, exporting to {}", endpoint);
log.debug("Flag evaluation metrics initialized");
} catch (NoClassDefFoundError e) {
log.error(
"OpenTelemetry SDK is not on the classpath — evaluation metrics disabled. Add"
+ " opentelemetry-sdk-metrics and opentelemetry-exporter-otlp to your dependencies to"
+ " enable flag evaluation metrics.",
"OpenTelemetry API is not on the classpath — evaluation metrics disabled. Add"
+ " opentelemetry-api to your dependencies and enable DD_METRICS_OTEL_ENABLED to"
+ " export flag evaluation metrics through the Datadog Java agent.",
e);
counter = null;
meterProvider = null;
} catch (Exception e) {
log.error("Failed to initialize flag evaluation metrics", e);
counter = null;
meterProvider = null;
}
}

/** Package-private constructor for testing with a mock counter. */
FlagEvalMetrics(LongCounter counter) {
this.counter = counter;
this.meterProvider = null;
}

/** Package-private constructor for integration testing with an injected SdkMeterProvider. */
FlagEvalMetrics(SdkMeterProvider sdkMeterProvider) {
meterProvider = sdkMeterProvider;
Meter meter = sdkMeterProvider.meterBuilder(METER_NAME).build();
counter =
meter.counterBuilder(METRIC_NAME).setUnit(METRIC_UNIT).setDescription(METRIC_DESC).build();
}

void record(
Expand Down Expand Up @@ -153,27 +95,5 @@ public void close() {

void shutdown() {
counter = null;
Closeable mp = meterProvider;
if (mp != null) {
meterProvider = null;
try {
mp.close();
} catch (Exception e) {
// Ignore shutdown errors
}
}
}

/**
* Builds a Resource with the service name from OTEL_SERVICE_NAME environment variable. Falls back
* to Resource.getDefault() if OTEL_SERVICE_NAME is not set.
*/
private static Resource buildResource() {
String serviceName = ConfigHelper.env(SERVICE_NAME_ENV);
if (serviceName != null && !serviceName.isEmpty()) {
return Resource.getDefault()
.merge(Resource.builder().put(SERVICE_NAME_KEY, serviceName).build());
}
return Resource.getDefault();
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -53,9 +53,8 @@ public Provider(final Options options) {
metrics = new FlagEvalMetrics();
hook = new FlagEvalHook(metrics);
} catch (LinkageError | Exception e) {
// FlagEvalMetrics logs the detailed error when it can load but OTel SDK init fails.
// This outer catch fires when the class itself can't load (OTel API absent entirely).
log.warn("Evaluation metrics unavailable — OTel classes not on classpath", e);
// This outer catch fires when the metrics helper itself can't load (OTel API absent).
log.warn("Evaluation metrics unavailable — OTel API classes not on classpath", e);
}
this.flagEvalMetrics = metrics;
this.flagEvalHook = hook;
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
package datadog.trace.api.openfeature;

import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.mockito.ArgumentMatchers.eq;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.verify;
Expand All @@ -9,13 +8,6 @@
import dev.openfeature.sdk.ErrorCode;
import io.opentelemetry.api.common.Attributes;
import io.opentelemetry.api.metrics.LongCounter;
import io.opentelemetry.sdk.metrics.SdkMeterProvider;
import io.opentelemetry.sdk.metrics.data.AggregationTemporality;
import io.opentelemetry.sdk.metrics.data.LongPointData;
import io.opentelemetry.sdk.metrics.data.MetricData;
import io.opentelemetry.sdk.metrics.export.AggregationTemporalitySelector;
import io.opentelemetry.sdk.testing.exporter.InMemoryMetricReader;
import java.util.Collection;
import org.junit.jupiter.api.Test;
import org.mockito.ArgumentCaptor;

Expand Down Expand Up @@ -146,35 +138,9 @@ void shutdownClearsCounter() {
}

@Test
void multipleRecordCallsAccumulateWithDeltaTemporality() {
// Use delta temporality to match the deltaPreferred() selector configured on the production
// OTLP exporter in FlagEvalMetrics. Delta temporality exports only increments since last
// collection, which is what OTLP receivers expect.
InMemoryMetricReader reader =
InMemoryMetricReader.builder()
.setAggregationTemporalitySelector(AggregationTemporalitySelector.deltaPreferred())
.build();
SdkMeterProvider provider = SdkMeterProvider.builder().registerMetricReader(reader).build();

try (FlagEvalMetrics metrics = new FlagEvalMetrics(provider)) {
for (int i = 0; i < 5; i++) {
metrics.record("count-flag", "on", "STATIC", null, "default-alloc");
}

Collection<MetricData> data = reader.collectAllMetrics();
MetricData metric =
data.stream()
.filter(m -> m.getName().equals("feature_flag.evaluations"))
.findFirst()
.orElseThrow(() -> new AssertionError("feature_flag.evaluations metric not found"));

assertEquals(
AggregationTemporality.DELTA,
metric.getLongSumData().getAggregationTemporality(),
"Exported metric must use DELTA temporality");

LongPointData point = metric.getLongSumData().getPoints().iterator().next();
assertEquals(5L, point.getValue(), "5 record() calls must produce a delta sum of 5");
void defaultConstructorUsesOpenTelemetryApiOnly() {
try (FlagEvalMetrics metrics = new FlagEvalMetrics()) {
metrics.record("count-flag", "on", "STATIC", null, "default-alloc");
}
}

Expand Down
Loading