Skip to content

Prompty Specification

Prompty assets are represented by files with a .prompty extension and must follow the schema outlined in the Prompty Specification.

1. Specification: Schema

We can now revisit the Prompty Specification to get the full picture of the Prompty asset schema. Let's start by recongizing the top-level sections:

  • $schema - the schema version used for validation
  • $id - the unique identifier for the schema
  • title - the title of the schema
  • description - a brief description of the schema
  • type - the type of the schema (here, "object")
  • additionalProperties - whether additional properties are allowed (here, "false")
  • properties - the properties of the schema
  • definitions - the definitions used in the schema
Specificaton file: prompty.yaml
YAML
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
    # This schema represents the specification file for the prompty
# _frontmatter_, not the content section.

$schema: http://json-schema.org/draft-07/schema#  
$id: http://azureml/sdk-2-0/Prompty.yaml  
title: Prompty front matter schema specification  
description: A specification that describes how to provision a new prompty using definition frontmatter.
type: object 

additionalProperties: false

properties:
$schema:
    type: string
# metadata section
model:
    type: object
    additionalProperties: false
    properties:
    api:
        type: string
        enum: 
        - chat
        - completion
        description: The API to use for the prompty -- this has implications on how the template is processed and how the model is called.
        default: chat

    configuration:
        oneOf:
        - $ref: "#/definitions/azureOpenaiModel"
        - $ref: "#/definitions/openaiModel"
        - $ref: "#/definitions/maasModel"

    parameters:
        $ref: "#/definitions/parameters"

    response: 
        type: string
        description: This determines whether the full (raw) response or just the first response in the choice array is returned.
        default: first 
        enum:
        - first
        - full


name:
    type: string
    description: Name of the prompty
description:
    type: string
    description: Description of the prompty
version:
    type: string
    description: Version of the prompty
authors:
    type: array
    description: Authors of the prompty
    items:
    type: string
tags:
    type: array
    description: Tags of the prompty
    items:
    type: string

# not yet supported -- might add later
# base:
#   type: string
#   description: The base prompty to use as a starting point

sample: 
    oneOf:
    - type: object
        description: The sample to be used in the prompty test execution
        additionalProperties: true
    - type: string
        description: The file to be loaded to be used in the prompty test execution

# the user can provide a single sample in a file or specify the data inline
# sample:
#   messages: 
#     - role: user
#       content: where is the nearest coffee shop?
#     - role: system
#       content: I'm sorry, I don't know that. Would you like me to look it up for you?
# or point to a file
# sample: sample.json
# also the user can specify the data on the command line
# pf flow test --flow p.prompty --input my_sample.json
# if the user runs this command, the sample from the prompty will be used
# pf flow test --flow p.prompty   

inputs:
    type: object
    description: The inputs to the prompty

outputs:
    type: object
    description: The outputs of the prompty

# currently not supported -- might not be needed
# init_signature:
#   type: object
#   description: The signature of the init function

template:
    type: string
    description: The template engine to be used can be specified here. This is optional.
    enum: [jinja2]
    default: jinja2

definitions:

Here,

  • properties describe valid "keys" that can be present in a .prompty asset file,
  • definitions describe "reusable" schema definitions" for quick reference in properties

Let's take a look at Prompty schema properties and definitions in more detail.

1.1 Specification: Properties

The properties schema describes these valid "keys" that can be present in a .prompty asset file:

  • $schema - (string) the schema version used for validation
  • model - (object) the model configuration for the asset
    • api - (string) the API type (chat (default) or completion)
    • configuration - (object) one of definitions provided
    • parameters - (object) from definitions provided
    • response - (string) whether to return full or first response (default: first)
  • name - (string) the name of the asset
  • description - (string) a brief description of the asset
  • version - (string) the version of the asset
  • authors - (array) a list of authors who contributed to the asset
  • tags - (array) a list of tags for the asset
  • sample - test data for validation - can be object (inline) or string (filename)
  • inputs - (object) defining request properties for prompty asset
  • outputs - (object) defining response properties for prompty asset
  • template - (string) the template engine to be used (default: jinja2)

1.2 Specification: Definitions

The definitions section of the schema describes reusable schema sections that can then be referenced within the properties section with the $ref keyword. For example, this is in model properties:

YAML
1
2
3
4
5
6
7
8
    configuration:
        oneOf:
        - $ref: "#/definitions/azureOpenaiModel"
        - $ref: "#/definitions/openaiModel"
        - $ref: "#/definitions/maasModel"

    parameters:
        $ref: "#/definitions/parameters"

This lets us define complex schema once and reference it in multiple places without duplication of content - helping keep the specification readable and maintainable.

The specification has 3 definitions for Model: Configuration:

Definition: openaiModel configuration for Model
YAML
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
# vanilla openai models
openaiModel:
    type: object
    description: Model used to generate text
    properties:
    type:
        type: string
        description: Type of the model
        const: openai
    name:
        type: string
        description: Name of the model
    organization:
        type: string
        description: Name of the organization
    additionalProperties: false
Definition: azurOpenaiModel configuration for Model
YAML
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
# azure openai models
azureOpenaiModel:
    type: object
    description: Model used to generate text
    properties:
    type:
        type: string
        description: Type of the model
        const: azure_openai
    api_version:
        type: string
        description: Version of the model
    azure_deployment:
        type: string
        description: Deployment of the model
    azure_endpoint:
        type: string
        description: Endpoint of the model
    additionalProperties: false
Definition: maasModel configuration for Model
YAML
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
# for maas models
maasModel:
    type: object
    description: Model used to generate text
    properties:
    type:
        type: string
        description: Type of the model
        const: azure_serverless
    azure_endpoint:
        type: string
        description: Endpoint of the model
    additionalProperties: false

The specification has 1 definition for Model: Parameters. For now, it defines these as common to all models (where individual models may process each differently).

The paramters are: response_format, seed, max_tokens, temperature, tools_choice, tools, frequency_penalty, presence_penalty, stop, and top_p. Click to expand for details.

Definition: parameters for Model
YAML
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
# parameters for the model -- for now these are not per model but the same for all models
parameters:
    type: object
    description: Parameters to be sent to the model 
    additionalProperties: true
    properties: 
    response_format: 
        type: object
        description: >
        An object specifying the format that the model must output. Compatible with
        `gpt-4-1106-preview` and `gpt-3.5-turbo-1106`.
        Setting to `{ "type": "json_object" }` enables JSON mode, which guarantees the
        message the model generates is valid JSON.

    seed:
        type: integer
        description: > 
        This feature is in Beta. If specified, our system will make a best effort to
        sample deterministically, such that repeated requests with the same `seed` and
        parameters should return the same result. Determinism is not guaranteed, and you
        should refer to the `system_fingerprint` response parameter to monitor changes
        in the backend.

    max_tokens:
        type: integer
        description: The maximum number of [tokens](/tokenizer) that can be generated in the chat completion.

    temperature:
        type: number
        description: What sampling temperature to use, 0 means deterministic.

    tools_choice:
        oneOf:
        - type: string
        - type: object

        description: > 
        Controls which (if any) function is called by the model. `none` means the model
        will not call a function and instead generates a message. `auto` means the model
        can pick between generating a message or calling a function. Specifying a
        particular function via
        `{"type": "function", "function": {"name": "my_function"}}` forces the model to
        call that function.

        `none` is the default when no functions are present. `auto` is the default if
        functions are present.

    tools:
        type: array
        items:
        type: object

    frequency_penalty:
        type: number
        description: What sampling frequency penalty to use. 0 means no penalty.

    presence_penalty:
        type: number
        description: What sampling presence penalty to use. 0 means no penalty.

    stop:
        type: array
        items:
        type: string
        description: > 
        One or more sequences where the model should stop generating tokens. The model
        will stop generating tokens if it generates one of the sequences. If the model
        generates a sequence that is a prefix of one of the sequences, it will continue
        generating tokens.

    top_p:
        type: number
        description: > 
        What nucleus sampling probability to use. 1 means no nucleus sampling. 0 means
        no tokens are generated.

2. Asset: Example

To get a practical sense for specification usage, let's look at an example Prompty asset file. Use the Prompty Visual Studio Code extension to generate the default .prompty starter file as follows:

  • open the file-explorer window (left) in Visual Studio Code
  • right-click on the folder where you want to create the .prompty file
  • select New Prompty File from the context menu

You should see something like this - we'll demysystify the contents, next.

Asset file: basic.prompty
Markdown
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
---
name: ExamplePrompt
description: A prompt that uses context to ground an incoming question
authors:
- Seth Juarez
model:
    api: chat
    configuration:
        type: azure_openai
        azure_endpoint: ${env:AZURE_OPENAI_ENDPOINT}
        azure_deployment: <your-deployment>
        api_version: 2024-07-01-preview
    parameters:
        max_tokens: 3000
sample:
    firstName: Seth
    context: >
        The Alpine Explorer Tent boasts a detachable divider for privacy, 
        numerous mesh windows and adjustable vents for ventilation, and 
        a waterproof design. It even has a built-in gear loft for storing 
        your outdoor essentials. In short, it's a blend of privacy, comfort, 
        and convenience, making it your second home in the heart of nature!
    question: What can you tell me about your tents?
---

system:
You are an AI assistant who helps people find information. As the assistant, 
you answer questions briefly, succinctly, and in a personable manner using 
markdown and even add some personal flair with appropriate emojis.

# Customer
You are helping {{firstName}} to find answers to their questions.
Use their name to address them in your responses.

# Context
Use the following context to provide a more personalized response to {{firstName}}:
{{context}}

user:
{{question}}

2.1 Asset: Frontmatter

The frontmatter section of the asset occurs between the --- delimiters (lines 1-24 above) and contains metadata about the asset. Let's take a look at just this segment below.

We see the following properties used in the frontmatter:

  • name - default name is "ExamplePrompt" (change it)
  • description - default description given (change it)
  • authors - default author given (change it)
  • model - the model configuration for the asset
    • api - set to default ("chat")
    • configuration - set to default ("azure_openai")
    • parameters - sets max_tokens to 3000
  • sample - set to inline object (not filename) with 3 properties
    • firstName - default requestor name (change it)
    • context - gives example "grounding" data (change it)
    • question - default user question (change it)

The frontmatter is used by Prompty tooling and runtime to understand the asset and its requirements for execution. The sample is key to allowing us to iteratively build the "shape" of our data with local testing, before integrating with external services to fetch real data that matches this shape.

Asset Frontmatter: basic.prompty
Markdown
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
---
name: ExamplePrompt
description: A prompt that uses context to ground an incoming question
authors:
- Seth Juarez
model:
    api: chat
    configuration:
        type: azure_openai
        azure_endpoint: ${env:AZURE_OPENAI_ENDPOINT}
        azure_deployment: <your-deployment>
        api_version: 2024-07-01-preview
    parameters:
        max_tokens: 3000
sample:
    firstName: Seth
    context: >
        The Alpine Explorer Tent boasts a detachable divider for privacy, 
        numerous mesh windows and adjustable vents for ventilation, and 
        a waterproof design. It even has a built-in gear loft for storing 
        your outdoor essentials. In short, it's a blend of privacy, comfort, 
        and convenience, making it your second home in the heart of nature!
    question: What can you tell me about your tents?
---

2.2 Asset: Template

The template section of the asset occurs below the second --- delimiters (lines 24-end in example above) and provides the content for the prompt template. Let's take a look at just this segment below.

A prompt template typically defines the persona, instructions and primary content (e.g., cues, examples) for the inference task that we want the model to execute. We see the following properties used in the template section for this purpose:

  • system - establishes default persona (AI assistant to help ..) and instructions (answer briefly ...)
    • Customer - data shape for "Customer" with instructions ("address them by name")
    • Context - data shape for "Grounding Context" with instructions ("personalize response")
  • user - defines the user prompt (actual incoming request)

The {{<variable-name>}} syntax used in the template denotes placeholders for data that will be bound in, when the template is instantiated at runtime. Note how each of those variables typically has a value defined in the sample object, for testing purposes. This allows us to get a sense for the shape of data we expect to retrieve and augment (from third-party services or from flow orchestration) as we iterate and engineer the prompt.

Asset Template: basic.prompty
Markdown
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
system:
You are an AI assistant who helps people find information. As the assistant, 
you answer questions briefly, succinctly, and in a personable manner using 
markdown and even add some personal flair with appropriate emojis.

# Customer
You are helping {{firstName}} to find answers to their questions.
Use their name to address them in your responses.

# Context
Use the following context to provide a more personalized response to {{firstName}}:
{{context}}

user:
{{question}}