- This event has passed.
Dense Retrieval with Apache Solr Neural Search [Melbourne Search and Recommendation]
June 29 @ 8:00 am - 9:00 amFree
Melbourne Search and Recommendation’s aim is to make the Meetup more welcoming to all search and recommendation developers, data scientists and researchers who are facing similar challenges — while maintaining the level of technical depth that we’ve come to enjoy.
APACHE LUCENE/SOLR COMMITTER
APACHE SOLR PMC MEMBER
Alessandro has been involved in designing and developing search-relevant solutions from the early stages of Apache Solr 1.4 and edismax query parser in 2010. Over the years he has worked on various projects aiming to build search solutions able to satisfy the user information needs, often integrating such solutions with machine learning and artificial intelligence technologies.
Dense Retrieval with Apache Solr Neural Search
Neural Search is an industry derivation from the academic field of Neural information Retrieval. More and more frequently, we hear about how Artificial Intelligence (AI) permeates every aspect of our lives and this includes also software engineering and Information Retrieval.
In particular, the advent of Deep Learning introduced the use of deep neural networks to solve complex problems that could not be solved simply by an algorithm. Deep Learning can be used to produce a vector representation of both the query and the documents in a corpus of information. Search, in general, comprises of performing four primary steps:
– generate a representation of the query that describes the information need – generate a representation of the document that captures the information contained in it
– match the query and the document representations from the corpus of information
– assign a score to each matched document in order to establish a meaningful document ranking by relevance in the results
With the Neural Search module, Apache Solr is introducing support for neural network based techniques that can improve these four aspects of search.
This talk explores the first official contribution of Neural Search capabilities available from Apache Solr 9.0(may 2022): Approximate K-Nearest Neighbor Vector Search for matching and ranking.
You will learn:
– how Approximate Nearest Neighbor (ANN) approaches work, with a focus on Hierarchical Navigable Small World Graph (HNSW)
– how the Apache Lucene implementation works
– how the Apache Solr implementation works, with the new field type and query parser introduced
– how to run KNN queries and how to use it to rerank a first stage pass Join us as we explore this new Apache Solr feature!