Retrieve results

When a processing request is completed successfully you can proceed in fetching the results. The results are provided in a JSON object format and are described in more detail in this page.

🌐 cURL Example

An example of fetching the results via cURL:

curl --request GET \
     --url https://api.behavioralsignals.com/v5/clients/your-client-id/processes/pid/results \
     --header 'X-Auth-Token: your-api-token' \
     --header 'accept: application/json'

The GET results method requires the client ID (long: {cid}) and the process ID (long: {pid}) to be passed as path parameters.

On invocation, it returns the result of the processing in JSON format. In case the specified cid or pid is not found or the status of the job is not set to completed, a corresponding error response is sent to the user.

Optionally, add the focus and sensitivity query params to also get Focus results, e.g. .../results?focus=angry,low_confidence&sensitivity=high.

🐍 Python SDK Example

from behavioralsignals import Client
from behavioralsignals.utils import print_results

client = Client()

process = client.behavioral.upload_audio(file_path="audio.wav")
result = client.behavioral.wait_for_result(
    pid=process.pid,
    timeout=600,
)

print_results(result.results)

Client() reads your credentials from the BEHAVIORALSIGNALS_CID and BEHAVIORALSIGNALS_API_KEY environment variables (see setup).

If the process has already finished, client.behavioral.get_result(pid=...) returns the result without waiting.

Response schema

The response is a JSON with the following structure:

{
  "pid": 0,
  "cid": 0,
  "code": 0,
  "message": "string",
  "results": [
    {
      "id": "0",
      "startTime": "0.209",
      "endTime": "7.681",
      "task": "<task>",
      "prediction": [
        {
          "label": "<label_1>",
          "posterior": "0.9576"
        },
        {
          "label": "<label_2>",
          "posterior": "0.0377"
        },
        ...
      ],
      "finalLabel": "<label_1>",
      "level": "utterance"
    },
    ...
  ]
}

results is an array, where each element corresponds to a prediction for a specific task and utterance/segment. The available tasks are the following:

• diarization: Contains the speaker label of the utterance, e.g: SPEAKER_00, SPEAKER_01, .... If the embeddings query param is defined, the speaker embeddings are also returned.
• asr: Contains the verbal content of the utterance
• language: The detected language
• gender: The sex of the speaker, estimated from the voice, one value per speaker for the whole file
• age: The age range of the speaker, estimated from the voice, one value per speaker for the whole file
• emotion: The detected emotion. Class labels: happy, angry, neutral, sad
• strength: The detected arousal of speech. Class labels: weak, neutral, strong
• positivity: The detected valence (sentiment) of speech. Class labels: negative, neutral, positive
• speaking_rate: How fast/slow the speaker talks. Class labels: slow, normal, fast
• hesitation: Whether there are signs of hesitation in speech. Class labels: no, yes
• engagement: Whether the speaker expresses engagement in their tone. Class labels: engaged, neutral, withdrawn
• whispering: Whether the speech is whispered. Class labels: non-whispering, whispering
• shouting: Whether the speech is shouted. Class labels: non-shouting, shouting
• intensity: The intensity of the expressed emotion, as a score in (0,1). It does not say which emotion is intense: for a score of anger, use the angry probability of emotion.
• breathiness: How breathy the voice sounds, as a score in (0,1)
• monotony: How monotone the delivery is, as a score in (0,1), where 1 is monotone
• fluency: How fluent the speech sounds to a listener, as a score in (0,1), where 1 is fully fluent
• signal_quality.snr, signal_quality.c50, signal_quality.sdr, signal_quality.overall: The quality of the recording, as scores in (0,1) where 1 is clean: background noise, reverberation, clipping, and an overall score that is close to the worst of the three
• features: This task is only present when the embeddings query param is defined. It contains the behavioral embeddings of the speaker.
• focus: This task is only present when the focus query param is defined, or when the process was streamed with focus. It flags utterances or segments where a label you asked for is likely but is not the finalLabel. See Focus.

The class labels of gender, age and the eight behavioral tasks (emotion to shouting) add up to 28 signals: each label is its own signal with its own probability. See Definition of behaviors for what each one means.

The id of each result is used to indicate the utterance/segment id. The startTime, endTime indicate the start/end of the utterance/segment. All results of one utterance share the same id, startTime and endTime. To split results by speaker, use the finalLabel of the utterance's diarization result.

Each result has a prediction array. This includes the values of each class for the specific task. For example in case of the emotion task, an example prediction object would be:

"prediction": [
  {
    "label": "sad",
    "posterior": "0.7969"
  },
  {
    "label": "neutral",
    "posterior": "0.1931"
  },
  {
    "label": "happy",
    "posterior": "0.007"
  },
  {
    "label": "angry",
    "posterior": "0.0029"
  }
]

In case of a score task (intensity, breathiness, monotony, fluency, signal_quality.*), an example prediction object would be:

"prediction": [
  {
    "score": "0.2989"
  }
]

The posterior indicates the probability of this class label being present in the utterance/segment. Posteriors and scores are strings, so convert them to numbers first.

For score tasks, the prediction object contains a score field (no label or posterior). The score is a value in (0,1).

The finalLabel in the result object is the label with the largest posterior. dominantInSegments, when present, is an internal field that you can ignore. Score tasks have no finalLabel, since they produce a continuous score rather than a class.

focus results are different: their finalLabel is suspected_<label>, and their posterior can be a combined score. See Focus.

The level field indicates whether this result corresponds to a "segment" or an "utterance". This field will always be "utterance" for this endpoint - the distinction between "segment" and "utterance" is more relevant for our streaming API.

The embedding field contains the speaker or behavioral embedding. This field is empty in most tasks except two:

• diarization: Here the embedding field corresponds to the speaker embedding
• features: The embedding field corresponds to the behavioral features

This field is present only when the embeddings query param is present.