Datasets are collections of data that are used to conduct experiments in the Benchmark application to evaluate AI models. They are represented in a tabular form, with each column containing unique variables, such as location, color or quality. Each row of your dataset will contain a single asset, its unique variables, as well as the Ground Truth data that will be used to measure the accuracy of the models you want to evaluate.

Creating a Dataset is easy and fast and can be done in one of two ways:

  • Importing from a CSV

  • Manual creation by asset


Importing a Dataset (Recommended)

Begin by clicking the New button and selecting the Import Dataset option

The Import Dataset popup will now open and guide you through importing your dataset.

Download a CSV Template:

Note: You will need to download the CSV template in order to import a dataset into the application. You can do so by clicking this option on the page

Dataset Content:

You must now configure the type of data your dataset will contain as well as the Cognitive Class and Capability you wish to evaluate.

Note: that the Class and Capability you select will determine the AI Models and Annotation you can use within this Dataset.

Select Submit to create your dataset


Creating a Dataset Manually:

Manually creating a dataset gives you the ultimate control over each asset and feature your dataset contains.

Begin by clicking the New button and selecting the New Dataset option

The creation process is broken into three steps:

Basic Info

Contains primary information on your dataset, such as the name, description, and tags.

Dataset Contents

You must now configure the type of data your dataset will contain as well as the Cognitive Class and Capability you wish to evaluate.

Note: that the Class and Capability you select will determine the AI Models and Annotation you can use within this Dataset.

Features

Features are the attributes that are associated with each asset in your dataset. Each new feature you add will allow you to define custom attributes on your asset which can be used to filter and discover more granular insights in post Experiment Analytics charts.

As an example, a Dataset that is being used to evaluate a Facial recognition model can include features such as:

  • Image Quality

  • Noise

  • Gender

  • Race

  • Age

each feature can act as a filter in order to gain insight into asset contents that can affect accuracy.

Add Assets

The final step in the process is to add assets to your Dataset. Each asset will include a name, the data asset, and the ground truth file that corresponds to the asset.

Using the Add New Asset section on the page begin adding each asset with the required fields specified. When you are done, click the Add Asset button. This will add this asset into your dataset and will appear below. Continue this process until your dataset is complete. Click the Done button on the bottom right corner of your screen to create your new dataset.

Did this answer your question?