RRE-Enterprise: Evaluation Explore/Compare Dashboard

This blog tutorial is intended for advanced users of RRE-Enterprise.

We covered the OVERVIEW dashboard already, the first approach to the results of a search quality evaluation.

If you are a software engineer working on a Search project, you want to go deeper in the exploration of the search quality evaluation results.

The Explore functionality gives you the ability to investigate why a certain metric returned a certain score:

 

Once selected an evaluation, all the collections belonging to the evaluation are selectable.
For each collection you have the set of metrics chosen at evaluation time, each metric shows the score and the difference from the previous evaluation:

When exploring the results you can focus on one or more metrics, so let’s see what happens if you select some of them:

The first view shows the average score for the metric in each of the topics defined in your rating set.
This gives you an idea of how each group of queries performs.
The expand and collapse functionality allows you to navigate the results even deeper, viewing the performance of each individual query:

And clicking the individual query, you see all the details, including the search result list and the comparison with the previous evaluation:

This gives you the ability to deeply debug why certain queries potentially improved/got worse, checking the Black Box API query, the corresponding search engine query (potentially automatically discovered by RRE-Enterprise), and the results list.

N.B. the Explore dashboard automatically compares the latest evaluation with the previous one(if available).

Compare

The Compare functionality is identical to the Explore, with one main difference:

You can select any two evaluations to compare.
N.B. they must be compatible in terms of the Rating set used to run the evaluation.

• The Target Iteration is the current evaluation you want to compare with a past one
• The Base Iteration is the past evaluation to use as the control

The user interface to compare the two evaluations remains the same.

You have now completed the basic tutorial to use RRE-Enterprise.

The next blog series will explore the internals of RRE-Enterprise and how it can achieve some of its wonders!

Stay tuned!