This blog tutorial is intended for advanced users of RRE-Enterprise.
We covered the OVERVIEW dashboard already, the first approach to the results of a search quality evaluation.
If you are a software engineer working on a Search project, you want to go deeper in the exploration of the search quality evaluation results.
The Explore functionality gives you the ability to investigate why a certain metric returned a certain score:
Once selected an evaluation, all the collections belonging to the evaluation are selectable.
For each collection you have the set of metrics chosen at evaluation time, each metric shows the score and the difference from the previous evaluation:
When exploring the results you can focus on one or more metrics, so let’s see what happens if you select some of them:
The first view shows the average score for the metric in each of the topics defined in your rating set.
This gives you an idea of how each group of queries performs.
The expand and collapse functionality allows you to navigate the results even deeper, viewing the performance of each individual query:
And clicking the individual query, you see all the details, including the search result list and the comparison with the previous evaluation:
This gives you the ability to deeply debug why certain queries potentially improved/got worse, checking the Black Box API query, the corresponding search engine query (potentially automatically discovered by RRE-Enterprise), and the results list.
N.B. the Explore dashboard automatically compares the latest evaluation with the previous one(if available).
Compare
The Compare functionality is identical to the Explore, with one main difference:
You can select any two evaluations to compare.
N.B. they must be compatible in terms of the Rating set used to run the evaluation.
- The Target Iteration is the current evaluation you want to compare with a past one
- The Base Iteration is the past evaluation to use as the control
The user interface to compare the two evaluations remains the same.
You have now completed the basic tutorial to use RRE-Enterprise.
The next blog series will explore the internals of RRE-Enterprise and how it can achieve some of its wonders!
Stay tuned!
Shameless plug for our training and services!
Did I mention we do a Search Quality Evaluation training (in two versions: for product managers and software engineers)?
We also provide consulting on these topics, get in touch if you want to set up the search quality evaluation pipeline for your search project!
Subscribe to our newsletter
Did you like this post about Drop constant features: a real-world Learning to Rank scenario? Don’t forget to subscribe to our Newsletter to stay always updated from the Information Retrieval world!
