Jon Brassey
Director @ TRIP Database Ltd
We have worked with Sease over a number of years and we keep returning, a sign they are doing something right! Most recently they have supported us with two main projects. The RAG clinical question answering was the most exciting for us and it was great to see that Sease worked closely with us, listening to our feedback, to ensure a really helpful proof of concept. We can move ahead, operationalising this approach, with huge optimism. The other recent project was supporting our move to the cloud. This was a really big move for us after so many years on dedicated servers. Having Sease independently look at our current system and proposed architecture was reassuring to say the least.
TripDatabase is a clinical search engine that allows users to quickly and easily find and use high-quality research evidence to support their practice and/or care. Trip has been online since 1997 and has developed into the internet’s premier source of evidence-based content. Their motto is ‘Find evidence fast’ and this is something they aim to deliver for every single search.
With technology advancing at a rapid pace, TripDatabase recognised the need to enhance its search capabilities to stay ahead of the curve. The impending migration to the latest version of Apache Solr provided an opportune moment to reassess and innovate. With a desire to incorporate advanced natural language search capabilities, TripDatabase sought expert advice intending to understand the feasibility and implications of adopting vector-based search and large language models within their platform.
In response to TripDatabase’s need to improve its search capabilities, Sease took a multi-angle approach to providing an analysis and prototypes of integrating advanced neural technologies. This solution involved a thorough process of exploration, experimentation, and refinement.
Over the years we have refined how to face these scenarios and the formula we have designed is a set of online and offline activities with the scope of understanding the client search system and giving immediate value through a systematic audit.
The main focus of the Exploration & Discovery phase is to give immediate value to the client, highlighting what they are doing well and what can be done better in terms of processes, architecture, configuration and usage.
Once that team dynamics, communication protocols, and software engineering practices are clarified it’s time to get technical, so we reviewed the system architecture and Apache Solr schema and solrconfig configurations.
Analysing these aspects is strictly tied to the client’s requirements and how they expect their users to interact with the search engine.
For the specific case of Trip, we dedicated extra care to studying their architecture and configurations in perspective of the upcoming migration: a deep check of breaking and not breaking changes was performed, both referring to the official Solr documentation and the detailed release notes and Jira issues tracking the Solr code changes.
We studied the data and the queries, identifying and recommending any best practices in data modelling and building the best Solr query for each of their task.
The outcome of this first package was a report structured with the following paragraphs:
Architecture & Deployment
Configurations SolrConfig.xml
Data Model (Schema.xml)
Query Approach
Search Quality Evaluation
Future Roadmap
Potential Challenges of a Migration to Solr 9.x
Each chapter has a short summary with the main takeaways and a list of paragraphs, each paragraph has an additional summary with the key takeaways and the long explanation.
It’s important to make the report readable, along with the Exploration & Discovery, we carry various calls to communicate the most important takeaways, but it’s vital to put them down, in a clear and usable presentation that is nice for both the business and the technical reader
Building upon the insights gleaned from the Exploration & Discovery phase, Sease embarked on the development of a Vector-based search prototype.
The work started analysing a paper from a team of researchers who fine-tuned a Large Language Model on top of data made available by Trip for academic usage.
The intent was to use such a model, fine-tuned for sentence similarity to encode Trip’s queries and textual documents to numerical vectors.
We decided on an Apache Solr implementation and a Python REST server to process the queries, encode the embeddings and run the vector Solr queries.
The outcome of the POC was both the code and a report highlighting the promising aspects and the weaknesses of the proposed approach and implementation.
We never run a Proof Of Concept with the sole scope of convincing a client to go in that direction or chasing the mainstream: we genuinely design and implement the best solution in budget and evaluate the promising aspects and weaknesses, offering a detailed plan of what it takes to bring it to production, if interesting enough
When starting with this project we were already familiar with Trip’s domain and use cases so we focused on designing a nice architecture to fetch data from their corpora of information and pre-process them to make them more appealing to Large Language Model fruition.
The idea was to design a Hybrid RAG framework, leveraging both lexical and vector-based search to identify the context for the LLM to generate their response.
As usual for these kinds of projects, the starting point is an analysis of the state of the art, for large language modelling applied to the language and domain of interest.
Choosing the model to use shouldn’t be taken as an easy task: it requires in-depth studies of how the model was pre-trained, the data used and how the model was fine-tuned (and the data used).
Once that’s solved (at least at the POC level), the content must be adapted: you don’t embed hundreds of pages of content to a single vector if the embedding model was trained to encode 5-10 terms at the time!
Chunking was the natural solution for this problem, and we iteratively improved the solution starting from simple “number of characters” truncation up to using machine learning for better candidate delimiters estimation.
Once the retrieval part was up to a decent standard, the following step was to frame a prompt to pass the retrieved context to a second large language model, fine-tuned for instruction-following.
The output for the client was also in this case a Python REST server, configurable to take in input the original query and return the generated snippet and citation.
The deliverable was paired with a report highlighting the pros, cons and steps necessary to bring the solution to production.
Throughout the project lifecycle, Sease maintained a structured workflow, characterized by clear communication channels (web-call and e-mail) and iterative refinement. Regular updates and consultations ensured seamless collaboration between Sease and TripDatabase, fostering alignment with project objectives and milestones.
Thanks to the collaboration with Sease for integrating advanced neural search functionality, TripDatabase now has all the necessary tools to embark on a transformation path to improve its natural language search capabilities. Comprehensive evaluation and prototype implementations have provided a clear roadmap to increase the functionality and relevance of the search, responding to the dynamic needs of the medical community. With an improved architecture and innovative features, TripDatabase is poised to continue efficiently delivering high-quality research evidence, consolidating its position as a leader in evidence-based healthcare.
Contact us today to learn how Sease’s consulting services can help your organization integrate advanced neural search solutions. Whether you’re looking to explore prototypes, assess feasibility, or implement neural search, our expert team is here to guide you every step of the way.