Benchmark for Anonymous Video Analytics - Evaluation Tool

Evaluate

We provide an online tool that allows one to evaluate Anonymous Video Analytics (AVA) algorithms on the proposed dataset.

Before uploading any file, please carefully read the instructions below.

How to use the evaluation tool?

  • Download the dataset
  • Generate the estimations using your AVA algorithm. One CSV file should be outputed per video. Each CSV file should have the same name than the video (e.g. for the video A.mp4, the estimation file should be A.csv)
  • Compress the CSV files into a single zip file (e.g. my_AVA_estimations.csv. Make sure of compressing the CSV files and not a directory with the CSV files.
  • Submit the zip file using the tool above
  • Download the per-video performance scores. Please be patient, the evaluation process might take few minutes depending on the number of videos to evaluate.


Data format

For each video, the algorithm should save the estimations in a CSV file following the format described below. The CSV file must have T rows, where the columns represent:

  • time: float that indicates the time, in seconds, the algorithm took to generate the results for the current frame
  • person_j: eight floats (or integers) that indicate the x0, y0, x1, y1 (x0, y0, x1, y1) coordinates of the bounding box (the origin of the reference system is the top-left corner of the frame) of the j-th person (face) with Opportunity to See (OTS) the signage, as defined in the paper, detected by the algorithm in frame,
    • x0: horizontal coordinate of the top-left corner of the person/face
    • y0: vertical coordinate of the top-left corner of the person/face
    • x1: horizontal coordinate of the bottom-right corner of the person/face
    • y1: vertical coordinate of the bottom-right corner of the person/face
  • three integers that indicate the values of the identity/age/gender attributes of the j-th person.
    • id: integer that indicates the person identity
    • age: integer that indicates the estimated person age
    • gender: 0=male; 1=female

For localization (face or person), identity, age, and gender the outputs shall be -1 if the estimation is unknown (e.g. the face is not visible); or -2 if the algorithm does not provide an output for that attribute.

This resultsPDF CSV file is an example of estimation file for a sample 30-frame video, T=30, with no person detected until frame 6; a person, with identity 0, detected from frame 6 onward; and two people, with identity 0 and 1, detected from frame 20 onward.


Intel is committed to respecting human rights and avoiding complicity in human rights abuses. See Intel's Global Human Rights Principles. Intel's products and software are intended only to be used in applications that do not cause or contribute to a violation of an internationally recognized human right.