Anyscale, a leading AI application platform, has announced a collaboration with MongoDB to improve multi-modal search capabilities, according to Anyscale. This partnership aims to address the limitations of traditional search systems and provide a more sophisticated search experience for enterprises dealing with large volumes of multi-modal data.

Challenges with Legacy Search Systems

Enterprises often struggle with legacy search systems that are not equipped to handle the complexities of multi-modal data, which includes text, images, and structured data. Traditional systems typically rely on lexical search methods that match text tokens, resulting in poor recall and irrelevant search results.

For instance, an e-commerce platform searching for a “green dress” might return items like “Bio Green Apple Shampoo” due to the limitations of lexical search. This is because the search system only matches text tokens and does not understand the semantic meaning behind the query.

Innovative Solution Using Anyscale and MongoDB

The collaboration between Anyscale and MongoDB aims to overcome these limitations by leveraging advanced AI models and scalable data indexing pipelines. The solution involves:

• Using Anyscale to run multi-modal large language models (LLMs) to generate product descriptions from images and names.

• Generating embeddings for product names and descriptions, which are then indexed into MongoDB Atlas Vector Search.

• Creating a hybrid search backend that combines legacy text matching with advanced semantic search capabilities.

This approach enhances the search relevance and user experience by understanding the semantic context of queries and returning more accurate results.

Use Case: E-commerce Platform

An example use case is an e-commerce platform with a large catalog of products. The platform aims to improve its search capabilities by implementing a scalable multi-modal search system that can handle both text and image data. The dataset used for this implementation is the Myntra dataset, which contains images and metadata of products for Myntra, an Indian fashion e-commerce company.

The legacy search system only matched text tokens, resulting in irrelevant search results. By using Anyscale and MongoDB, the platform can now return more relevant results by understanding the semantic meaning of queries and using images to enrich the search context.

System Architecture

The system is divided into two main stages: an offline data indexing stage and an online search stage. The data indexing stage processes, embeds, and upserts text and images into MongoDB, while the search stage handles search requests in real-time.

Data Indexing Stage

This stage involves:

• Metadata enrichment using multi-modal LLMs to generate product descriptions and metadata fields.

• Embedding generation for product names and descriptions.

• Data ingestion into MongoDB Atlas Vector Search.

Search Stage

The search stage combines legacy text matching with advanced semantic search. It involves:

1. Sending a search request from the frontend.

2. Processing the request at the ingress deployment.

3. Generating embeddings for the query text.

4. Performing a vector search on MongoDB.

5. Returning the search results to the frontend.

Conclusion

The collaboration between Anyscale and MongoDB represents a significant advancement in multi-modal search technology. By integrating advanced AI models and scalable data indexing pipelines, enterprises can now offer a more relevant and efficient search experience. This solution is particularly beneficial for e-commerce platforms looking to improve their search capabilities and user experience.

For more information, visit the Anyscale blog.

Image source: Shutterstock