Skip to content

Commit f2ec0ee

Browse files
api-clients-generation-pipeline[bot]ci.datadog-api-spec
api-clients-generation-pipeline[bot]
and
ci.datadog-api-spec
authored
Add OpenAPI for LLM Observability dataset draft state and versions (DataDog#3374)
Co-authored-by: ci.datadog-api-spec <packages@datadoghq.com>
1 parent 5ed2d8d commit f2ec0ee

20 files changed

Lines changed: 2039 additions & 0 deletions

‎.generator/schemas/v2/openapi.yaml‎

Lines changed: 439 additions & 0 deletions
Large diffs are not rendered by default.

‎examples/v2/llm-observability/GetLLMObsDatasetDraftState.rb‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
# Get LLM Observability dataset draft state returns "OK" response
2+
3+
require "datadog_api_client"
4+
DatadogAPIClient.configure do |config|
5+
config.unstable_operations["v2.get_llm_obs_dataset_draft_state".to_sym] = true
6+
end
7+
api_instance = DatadogAPIClient::V2::LLMObservabilityAPI.new
8+
p api_instance.get_llm_obs_dataset_draft_state("project_id", "dataset_id")

‎examples/v2/llm-observability/ListLLMObsDatasetVersions.rb‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
# List LLM Observability dataset versions returns "OK" response
2+
3+
require "datadog_api_client"
4+
DatadogAPIClient.configure do |config|
5+
config.unstable_operations["v2.list_llm_obs_dataset_versions".to_sym] = true
6+
end
7+
api_instance = DatadogAPIClient::V2::LLMObservabilityAPI.new
8+
p api_instance.list_llm_obs_dataset_versions("project_id", "dataset_id")

‎examples/v2/llm-observability/LockLLMObsDatasetDraftState.rb‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
# Lock LLM Observability dataset draft state returns "OK" response
2+
3+
require "datadog_api_client"
4+
DatadogAPIClient.configure do |config|
5+
config.unstable_operations["v2.lock_llm_obs_dataset_draft_state".to_sym] = true
6+
end
7+
api_instance = DatadogAPIClient::V2::LLMObservabilityAPI.new
8+
p api_instance.lock_llm_obs_dataset_draft_state("project_id", "dataset_id")

‎examples/v2/llm-observability/UnlockLLMObsDatasetDraftState.rb‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
# Unlock LLM Observability dataset draft state returns "OK" response
2+
3+
require "datadog_api_client"
4+
DatadogAPIClient.configure do |config|
5+
config.unstable_operations["v2.unlock_llm_obs_dataset_draft_state".to_sym] = true
6+
end
7+
api_instance = DatadogAPIClient::V2::LLMObservabilityAPI.new
8+
p api_instance.unlock_llm_obs_dataset_draft_state("project_id", "dataset_id")

‎features/scenarios_model_mapping.rb‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1054,6 +1054,18 @@
10541054
"dataset_id" => "String",
10551055
"body" => "LLMObsDatasetUpdateRequest",
10561056
},
1057+
"v2.GetLLMObsDatasetDraftState" => {
1058+
"project_id" => "String",
1059+
"dataset_id" => "String",
1060+
},
1061+
"v2.LockLLMObsDatasetDraftState" => {
1062+
"project_id" => "String",
1063+
"dataset_id" => "String",
1064+
},
1065+
"v2.UnlockLLMObsDatasetDraftState" => {
1066+
"project_id" => "String",
1067+
"dataset_id" => "String",
1068+
},
10571069
"v2.ListLLMObsDatasetRecords" => {
10581070
"project_id" => "String",
10591071
"dataset_id" => "String",
@@ -1076,6 +1088,10 @@
10761088
"dataset_id" => "String",
10771089
"body" => "LLMObsDeleteDatasetRecordsRequest",
10781090
},
1091+
"v2.ListLLMObsDatasetVersions" => {
1092+
"project_id" => "String",
1093+
"dataset_id" => "String",
1094+
},
10791095
"v2.ListLLMObsExperimentEvents" => {
10801096
"experiment_id" => "String",
10811097
"page_limit" => "Integer",

‎features/v2/llm_observability.feature‎

Lines changed: 108 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -417,6 +417,33 @@ Feature: LLM Observability
417417
When the request is sent
418418
Then the response status is 404 Not Found
419419

420+
@generated @skip @team:DataDog/ml-observability
421+
Scenario: Get LLM Observability dataset draft state returns "Bad Request" response
422+
Given operation "GetLLMObsDatasetDraftState" enabled
423+
And new "GetLLMObsDatasetDraftState" request
424+
And request contains "project_id" parameter from "REPLACE.ME"
425+
And request contains "dataset_id" parameter from "REPLACE.ME"
426+
When the request is sent
427+
Then the response status is 400 Bad Request
428+
429+
@generated @skip @team:DataDog/ml-observability
430+
Scenario: Get LLM Observability dataset draft state returns "Not Found" response
431+
Given operation "GetLLMObsDatasetDraftState" enabled
432+
And new "GetLLMObsDatasetDraftState" request
433+
And request contains "project_id" parameter from "REPLACE.ME"
434+
And request contains "dataset_id" parameter from "REPLACE.ME"
435+
When the request is sent
436+
Then the response status is 404 Not Found
437+
438+
@generated @skip @team:DataDog/ml-observability
439+
Scenario: Get LLM Observability dataset draft state returns "OK" response
440+
Given operation "GetLLMObsDatasetDraftState" enabled
441+
And new "GetLLMObsDatasetDraftState" request
442+
And request contains "project_id" parameter from "REPLACE.ME"
443+
And request contains "dataset_id" parameter from "REPLACE.ME"
444+
When the request is sent
445+
Then the response status is 200 OK
446+
420447
@generated @skip @team:DataDog/ml-observability
421448
Scenario: Get a custom evaluator configuration returns "Bad Request" response
422449
Given operation "GetLLMObsCustomEvalConfig" enabled
@@ -538,6 +565,33 @@ Feature: LLM Observability
538565
When the request is sent
539566
Then the response status is 200 OK
540567

568+
@generated @skip @team:DataDog/ml-observability
569+
Scenario: List LLM Observability dataset versions returns "Bad Request" response
570+
Given operation "ListLLMObsDatasetVersions" enabled
571+
And new "ListLLMObsDatasetVersions" request
572+
And request contains "project_id" parameter from "REPLACE.ME"
573+
And request contains "dataset_id" parameter from "REPLACE.ME"
574+
When the request is sent
575+
Then the response status is 400 Bad Request
576+
577+
@generated @skip @team:DataDog/ml-observability
578+
Scenario: List LLM Observability dataset versions returns "Not Found" response
579+
Given operation "ListLLMObsDatasetVersions" enabled
580+
And new "ListLLMObsDatasetVersions" request
581+
And request contains "project_id" parameter from "REPLACE.ME"
582+
And request contains "dataset_id" parameter from "REPLACE.ME"
583+
When the request is sent
584+
Then the response status is 404 Not Found
585+
586+
@generated @skip @team:DataDog/ml-observability
587+
Scenario: List LLM Observability dataset versions returns "OK" response
588+
Given operation "ListLLMObsDatasetVersions" enabled
589+
And new "ListLLMObsDatasetVersions" request
590+
And request contains "project_id" parameter from "REPLACE.ME"
591+
And request contains "dataset_id" parameter from "REPLACE.ME"
592+
When the request is sent
593+
Then the response status is 200 OK
594+
541595
@generated @skip @team:DataDog/ml-observability
542596
Scenario: List LLM Observability datasets returns "Bad Request" response
543597
Given operation "ListLLMObsDatasets" enabled
@@ -662,6 +716,33 @@ Feature: LLM Observability
662716
When the request is sent
663717
Then the response status is 200 OK
664718

719+
@generated @skip @team:DataDog/ml-observability
720+
Scenario: Lock LLM Observability dataset draft state returns "Bad Request" response
721+
Given operation "LockLLMObsDatasetDraftState" enabled
722+
And new "LockLLMObsDatasetDraftState" request
723+
And request contains "project_id" parameter from "REPLACE.ME"
724+
And request contains "dataset_id" parameter from "REPLACE.ME"
725+
When the request is sent
726+
Then the response status is 400 Bad Request
727+
728+
@generated @skip @team:DataDog/ml-observability
729+
Scenario: Lock LLM Observability dataset draft state returns "Not Found" response
730+
Given operation "LockLLMObsDatasetDraftState" enabled
731+
And new "LockLLMObsDatasetDraftState" request
732+
And request contains "project_id" parameter from "REPLACE.ME"
733+
And request contains "dataset_id" parameter from "REPLACE.ME"
734+
When the request is sent
735+
Then the response status is 404 Not Found
736+
737+
@generated @skip @team:DataDog/ml-observability
738+
Scenario: Lock LLM Observability dataset draft state returns "OK" response
739+
Given operation "LockLLMObsDatasetDraftState" enabled
740+
And new "LockLLMObsDatasetDraftState" request
741+
And request contains "project_id" parameter from "REPLACE.ME"
742+
And request contains "dataset_id" parameter from "REPLACE.ME"
743+
When the request is sent
744+
Then the response status is 200 OK
745+
665746
@generated @skip @team:DataDog/ml-observability
666747
Scenario: Push events for an LLM Observability experiment returns "Accepted" response
667748
Given operation "CreateLLMObsExperimentEvents" enabled
@@ -765,6 +846,33 @@ Feature: LLM Observability
765846
When the request is sent
766847
Then the response status is 200 OK
767848

849+
@generated @skip @team:DataDog/ml-observability
850+
Scenario: Unlock LLM Observability dataset draft state returns "Bad Request" response
851+
Given operation "UnlockLLMObsDatasetDraftState" enabled
852+
And new "UnlockLLMObsDatasetDraftState" request
853+
And request contains "project_id" parameter from "REPLACE.ME"
854+
And request contains "dataset_id" parameter from "REPLACE.ME"
855+
When the request is sent
856+
Then the response status is 400 Bad Request
857+
858+
@generated @skip @team:DataDog/ml-observability
859+
Scenario: Unlock LLM Observability dataset draft state returns "Not Found" response
860+
Given operation "UnlockLLMObsDatasetDraftState" enabled
861+
And new "UnlockLLMObsDatasetDraftState" request
862+
And request contains "project_id" parameter from "REPLACE.ME"
863+
And request contains "dataset_id" parameter from "REPLACE.ME"
864+
When the request is sent
865+
Then the response status is 404 Not Found
866+
867+
@generated @skip @team:DataDog/ml-observability
868+
Scenario: Unlock LLM Observability dataset draft state returns "OK" response
869+
Given operation "UnlockLLMObsDatasetDraftState" enabled
870+
And new "UnlockLLMObsDatasetDraftState" request
871+
And request contains "project_id" parameter from "REPLACE.ME"
872+
And request contains "dataset_id" parameter from "REPLACE.ME"
873+
When the request is sent
874+
Then the response status is 200 OK
875+
768876
@generated @skip @team:DataDog/ml-observability
769877
Scenario: Update LLM Observability dataset records returns "Bad Request" response
770878
Given operation "UpdateLLMObsDatasetRecords" enabled

‎features/v2/undo.json‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3943,6 +3943,24 @@
39433943
"type": "idempotent"
39443944
}
39453945
},
3946+
"GetLLMObsDatasetDraftState": {
3947+
"tag": "LLM Observability",
3948+
"undo": {
3949+
"type": "safe"
3950+
}
3951+
},
3952+
"LockLLMObsDatasetDraftState": {
3953+
"tag": "LLM Observability",
3954+
"undo": {
3955+
"type": "idempotent"
3956+
}
3957+
},
3958+
"UnlockLLMObsDatasetDraftState": {
3959+
"tag": "LLM Observability",
3960+
"undo": {
3961+
"type": "idempotent"
3962+
}
3963+
},
39463964
"ListLLMObsDatasetRecords": {
39473965
"tag": "LLM Observability",
39483966
"undo": {
@@ -3980,6 +3998,12 @@
39803998
"type": "unsafe"
39813999
}
39824000
},
4001+
"ListLLMObsDatasetVersions": {
4002+
"tag": "LLM Observability",
4003+
"undo": {
4004+
"type": "safe"
4005+
}
4006+
},
39834007
"ListLLMObsExperimentEvents": {
39844008
"tag": "LLM Observability",
39854009
"undo": {

‎lib/datadog_api_client/configuration.rb‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -228,18 +228,22 @@ def initialize
228228
"v2.get_llm_obs_annotated_interactions_by_trace_i_ds": false,
229229
"v2.get_llm_obs_annotation_queue_label_schema": false,
230230
"v2.get_llm_obs_custom_eval_config": false,
231+
"v2.get_llm_obs_dataset_draft_state": false,
231232
"v2.list_llm_obs_annotation_queues": false,
232233
"v2.list_llm_obs_dataset_records": false,
233234
"v2.list_llm_obs_datasets": false,
235+
"v2.list_llm_obs_dataset_versions": false,
234236
"v2.list_llm_obs_experiment_events": false,
235237
"v2.list_llm_obs_experiments": false,
236238
"v2.list_llm_obs_integration_accounts": false,
237239
"v2.list_llm_obs_integration_models": false,
238240
"v2.list_llm_obs_projects": false,
239241
"v2.list_llm_obs_spans": false,
242+
"v2.lock_llm_obs_dataset_draft_state": false,
240243
"v2.search_llm_obs_experimentation": false,
241244
"v2.search_llm_obs_spans": false,
242245
"v2.simple_search_llm_obs_experimentation": false,
246+
"v2.unlock_llm_obs_dataset_draft_state": false,
243247
"v2.update_llm_obs_annotation_queue": false,
244248
"v2.update_llm_obs_annotation_queue_label_schema": false,
245249
"v2.update_llm_obs_custom_eval_config": false,

‎lib/datadog_api_client/inflector.rb‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3796,6 +3796,11 @@ def overrides
37963796
"v2.llm_obs_dataset_data_attributes_response" => "LLMObsDatasetDataAttributesResponse",
37973797
"v2.llm_obs_dataset_data_request" => "LLMObsDatasetDataRequest",
37983798
"v2.llm_obs_dataset_data_response" => "LLMObsDatasetDataResponse",
3799+
"v2.llm_obs_dataset_draft_state_data" => "LLMObsDatasetDraftStateData",
3800+
"v2.llm_obs_dataset_draft_state_data_attributes" => "LLMObsDatasetDraftStateDataAttributes",
3801+
"v2.llm_obs_dataset_draft_state_response" => "LLMObsDatasetDraftStateResponse",
3802+
"v2.llm_obs_dataset_draft_state_type" => "LLMObsDatasetDraftStateType",
3803+
"v2.llm_obs_dataset_draft_state_user" => "LLMObsDatasetDraftStateUser",
37993804
"v2.llm_obs_dataset_record_data_response" => "LLMObsDatasetRecordDataResponse",
38003805
"v2.llm_obs_dataset_record_item" => "LLMObsDatasetRecordItem",
38013806
"v2.llm_obs_dataset_records_data_attributes_request" => "LLMObsDatasetRecordsDataAttributesRequest",
@@ -3815,6 +3820,10 @@ def overrides
38153820
"v2.llm_obs_dataset_update_data_attributes_request" => "LLMObsDatasetUpdateDataAttributesRequest",
38163821
"v2.llm_obs_dataset_update_data_request" => "LLMObsDatasetUpdateDataRequest",
38173822
"v2.llm_obs_dataset_update_request" => "LLMObsDatasetUpdateRequest",
3823+
"v2.llm_obs_dataset_version_data" => "LLMObsDatasetVersionData",
3824+
"v2.llm_obs_dataset_version_data_attributes" => "LLMObsDatasetVersionDataAttributes",
3825+
"v2.llm_obs_dataset_versions_response" => "LLMObsDatasetVersionsResponse",
3826+
"v2.llm_obs_dataset_version_type" => "LLMObsDatasetVersionType",
38183827
"v2.llm_obs_delete_annotation_queue_interactions_data_attributes_request" => "LLMObsDeleteAnnotationQueueInteractionsDataAttributesRequest",
38193828
"v2.llm_obs_delete_annotation_queue_interactions_data_request" => "LLMObsDeleteAnnotationQueueInteractionsDataRequest",
38203829
"v2.llm_obs_delete_annotation_queue_interactions_request" => "LLMObsDeleteAnnotationQueueInteractionsRequest",

0 commit comments

Comments
 (0)