Veritone Benchmark provides the resources and tools necessary to evaluate and monitor the performance of AI Models. Being able to effectively select the right model from the hundreds available, as well as integrate it with existing human resources in a way that cultivates trust in your digital workforce is an important requirement for all businesses.

For AIWare developers, this allows you to compare your engine against other engines so that you can refine your engines to be the best in class.

For AIWare users, the Benchmark application allows you to figure out which engines generate the best combination of speed, accuracy, and performance all while minimizing cost -- allowing AIWare to deliver more business value to you versus individual AI providers.

Main Benefits:

One of the main benefits of AIWare is the ability to run engines from different providers to process your media. With this choice comes the problem:

How do I know which engines will provide the best results?

Veritone Benchmark is designed to make it easy for you to compare the results of engines against each other, allowing you to visualize engine performance over time, and see apples-to-apples comparisons of engines against one another.

Features:

  • Dataset Creation and Management

  • Model Comparison and Evaluation

  • Annotation and Labeling

  • Model Performance Reports and Sharing


Cognitive Capabilities Supported:

The current version of the application supports the following cognitive categories:

Transcription

Transcription engines — often known as "speech-to-text" or natural language processing (NLP) — take recorded speech audio and output the words that were said. Depending on the engine's capabilities, the output could be a simple sequence of words or a "lattice of confidence" expressing multiple options for how the words were spoken.

Facial Detection

Face detection engines can detect human faces in media assets, and locate them (within the visual frame) in terms of a bounding polygon. Unlike a face recognition engine, a face detection engine merely determines whether a face (any face) was detected. It does not try to identify the face or match it to other data.

Logo detection

Logo detection engines can detect the logo in media assets, and locate them (within the visual frame) in terms of a bounding polygon. Unlike a logo recognition engine, a logo detection engine merely determines whether a logo (any logo) was detected. It does not try to identify the logo or match it to other data.

Object Detection

Object detection engines can detect the Object in media assets, and locate them (within the visual frame) in terms of a bounding polygon. Unlike an object recognition engine, an Object detection engine merely determines whether an object (any object) was detected. It does not try to identify the object or match it to other data.

Speaker Separation

Speaker Separation engines can view phonetic transcription as a conversation in the media. Unlike a transcription engine, Speaker separation engines display transcripts in the media as a conversation.

Did this answer your question?