OpenAPI::Client::OpenAI::Path::evals(3)

NAME
OpenAPI::Client::OpenAI::Path::evals - Documentation for the /evals path.

DESCRIPTION
This document describes the API endpoint at "/evals".

PATHS

  "GET /evals"
List evaluations for a project.

Operation ID

"listEvals"

    $client->listEvals( ... );

Parameters

•   "after" (in query) (Optional) - Identifier for the last eval from the previous pagination request.

    Type: "string"

•   "limit" (in query) (Optional) - Number of evals to retrieve.

    Type: "integer"

    Default: 20

•   "order" (in query) (Optional) - Sort order for evals by timestamp. Use `asc` for ascending order or `desc` for descending order.

    Type: "string"

    Allowed values: "asc, desc"

    Default: "asc"

•   "order_by" (in query) (Optional) - Evals can be ordered by creation time or last updated time. Use `created_at` for creation time or `updated_at` for last updated time.

    Type: "string"

    Allowed values: "created_at, updated_at"

    Default: "created_at"

Responses

Status Code: 200

A list of evals

Content Types:

•   "application/json"

    Example (See the OpenAI spec for more detail <https://github.com/openai/openai-openapi/blob/master/openapi.yaml>):

        {
          "object": "list",
          "data": [
            {
              "object": "eval",
              "id": "eval_67abd54d9b0081909a86353f6fb9317a",
              "data_source_config": {
                "type": "custom",
                "schema": {
                  "type": "object",
                  "properties": {
                    "item": {
                      "type": "object",
                      "properties": {
                        "input": {
                          "type": "string"
                        },
                        "ground_truth": {
                          "type": "string"
                        }
                      },
                      "required": [
                        "input",
                        "ground_truth"
                      ]
                    }
                  },
                  "required": [
                    "item"
                  ]
                }
              },
              "testing_criteria": [
                {
                  "name": "String check",
                  "id": "String check-2eaf2d8d-d649-4335-8148-9535a7ca73c2",
                  "type": "string_check",
                  "input": "{{item.input}}",
                  "reference": "{{item.ground_truth}}",
                  "operation": "eq"
                }
              ],
              "name": "External Data Eval",
              "created_at": 1739314509,
              "metadata": {},
            }
          ],
          "first_id": "eval_67abd54d9b0081909a86353f6fb9317a",
          "last_id": "eval_67abd54d9b0081909a86353f6fb9317a",
          "has_more": true
        }

  "POST /evals"
Create the structure of an evaluation that can be used to test a model's performance. An evaluation is a set of testing criteria and the config for a data source, which dictates the schema of the data used in the evaluation. After creating an evaluation, you can run it on different models and model parameters. We support several types of graders and datasources. For more information, see the Evals guide <https://platform.openai.com/docs/guides/evals>.

Operation ID

"createEval"

    $client->createEval( ... );

Parameters

Request Body

Content Type: "application/json"

Example:

    {
       "testing_criteria" : [
          null
       ]
    }

Responses

Status Code: 201

OK

Content Types:

•   "application/json"

    Example (See the OpenAI spec for more detail <https://github.com/openai/openai-openapi/blob/master/openapi.yaml>):

        {
          "object": "eval",
          "id": "eval_67abd54d9b0081909a86353f6fb9317a",
          "data_source_config": {
            "type": "custom",
            "item_schema": {
              "type": "object",
              "properties": {
                "label": {"type": "string"},
              },
              "required": ["label"]
            },
            "include_sample_schema": true
          },
          "testing_criteria": [
            {
              "name": "My string check grader",
              "type": "string_check",
              "input": "{{sample.output_text}}",
              "reference": "{{item.label}}",
              "operation": "eq",
            }
          ],
          "name": "External Data Eval",
          "created_at": 1739314509,
          "metadata": {
            "test": "synthetics",
          }
        }

SEE ALSO
OpenAPI::Client::OpenAI::Path

COPYRIGHT AND LICENSE
Copyright (C) 2023-2025 by Nelson Ferraz

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself, either Perl version 5.14.0 or, at your option, any later version of Perl 5 you may have available.