If you are looking for the quickest, easiest way to compare AI model metrics the New Benchmark Creation Wizard allows you to benchmark a single or multiple media files across supported Classes and Capabilities. This can be done with existing Data Center media files or by importing new media files directly from the application.

Creating a New Benchmark

Within your Benchmark application, select the Add New button at the top left of your browser and choose New Benchmark. This will open the New Benchmark creation wizard and guide you through the process.

Step 1 - Choose Your Data:

The first step to create a new benchmark is to choose from existing or new media files you would like to use to compare Cognitive engine metrics.

Using Existing Media Files:

If you have already imported Media data into your Data Center, you can leverage these files and the AI Model output that has been run against them for comparison. Using the navigation interface, locate the folder and files you wish to use for your benchmark by selecting each file using the checkbox.

  • Choose up to 10 media files to benchmark

Once you have selected all files you would like to benchmark, choose the Next button.

Importing New Media Files:

Selecting the Upload Files to Benchmark option will allow you to choose new files from your local machine and import them directly into your account for benchmarking.

  • Choose up to 10 media files to benchmark

You will be able to choose video, audio, or image data from your local machine to import and process into your Benchmark application.

  • Imported data will be stored within your Data Center for future access and benchmarks.

Choosing AI Models for Evaluation

After your new data has been successfully uploaded, you will need to select the AI Models you would like to use for your benchmark for each file selected.

  • Note that the AI Model availability will be based on the file data type. As an example, a Face Recognition Cognitive Engine can only processing Video and Image-based data and will not be available for Audio.

Choose a Baseline or Upload Groundtruth

Once you have selected the AI Models you would like to benchmark for your Media Files, you will need to include ground truth or define what models to use as a baseline on each file.

  • Helpful Tip: When choosing a baseline, select an AI model that is known to perform well against your media files. This will be used as a point of comparison when evaluating other AI model performance.

Did this answer your question?