Today, instead of talking about ElasticSearch/Solr, we’re going to talk about some of the new small and beautiful search engine frames.
MeiliSearch vs. Elasticsearch
Elasticsearch was designed as a back-end search engine, and although it was not initially suitable for this purpose, it was often used to build search bars for end users. Unlike Elasticsearch, which is a general-purpose search engine, MeiliSearch focuses on providing a specific feature.
Elasticsearch handles searches through massive amounts of data and text analysis. To make it effective for end-user searches, you need to take the time to understand the inner workings of Elasticsearch so that you can customize and adapt it to suit your needs. MeiliSearch is designed to provide a high-performance real-time search experience for end users. However, it is not possible to handle complex queries or analyze very large data sets.
Elasticsearch can sometimes be too slow if you want to provide a full real-time search experience. Most of the time, it returns significantly slower search results than MeiliSearch. MeiliSearch is the perfect choice if you need a simple tool to deploy a typos tolerance search bar, provide prefix search functionality, allow users to search intuitively, and return their results instantly with near-perfect relevance.
MeiliSearch vs. Algolia
MeiliSearch was inspired by Algolia’s product and the algorithm behind it. We did look at most of the algorithms and data structures described in their blog post in order to implement our own. So MeiliSearch is a new search engine based on Algolia’s work and recent research papers. It offers similar functionality and quickly achieves the same degree of relevance as its predecessor.
Unlike Algolia, MeiliSearch is open source and written in Rust, a modern system-level programming language that allows functionality to be built quickly. Rust also enables portability and flexibility, which makes deployment of our search engine within virtual machines, containers, and even Lambda@Edge a seamless operation.
One of Algolia’s key assets is the strong global infrastructure they provide to their customers. MeiliSearch is currently offering a search engine that does not yet provide a competitive infrastructure. However, our goal is to make it much simpler to deploy and maintain than Algolia’s.
Open source
Lucene
Apache Lucene is a free and open source search library written in Java for full-text indexing and searching of documents. The project was first founded in 1999 by Doug Cutting, who had previously written search engines at Xerox’s Palo Alto Research Centre (PARC) and Apple. Since Lucene was developed to build web search applications like Google, you can see DuckDuckGo still uses it for some specific searches.
Lucene was later split into several projects.
Lucene itself: full-text search library. Solr: An enterprise search server with powerful REST apis. Nutch: a scalable and scalable web crawler that relies on Apache Hadoop. Because Lucene is the technology behind many open or closed source search engines, it is considered the reference search library.
Sonic
Sonic is a lightweight and patternless search indexing server written in Rust. Sonic cannot be considered an out-of-the-box solution and does not guarantee relevance ranking compared to MeiliSearch. In fact, it does not store any documents, but consists of an inverted index with levenstein automata, which means that any application that queries Sonic must retrieve the search results from an external database using the returned ID, and then apply some relevance ranking. Its ability to run on a few Megabytes of RAM makes it a minimalist and resource-efficient alternative to database tools that may be too heavy to scale.
#Typesense Like MeiliSearch, Typesense is a lightweight open source search engine optimized for speed. We are currently reevaluating its features and functionality to better understand how it compares to MeiliSearch.
Lucene derivatives
Lucene-Solr
Solr, a sub-project of Apache Lucene, was founded by Yonik Seeley in 2004 and is now one of the most widely used search engines in the world. Solr is a search platform written in Java and built on Top of Lucene. In other words, Solr is an HTTP wrapper around Lucene’s Java API, which means you can use it to take advantage of all of Lucene’s capabilities. In addition, Solr server is combined with Solr Cloud to provide distributed indexing and search capabilities, ensuring high availability and scalability. Data is shared, but also automatically replicated. Moreover, Solr is more than just a search engine; It is often used as a NoSQL database for document structures. Documents are stored in collections, comparable to tables in a relational database.
Thanks to its extensible plug-in architecture and customizable capabilities, Solr is a search engine with endless use cases, although it is particularly popular in enterprise search because it can index and search documents and E-mail attachments.
Bleve & Tantivy
Bleve and Tantivy are search engine projects, written in Golang and Rust respectively, inspired by Apache Lucene and its algorithms (e.g., TF-IDF, short for the term frequency – Reverse document frequency). Like Lucene, both are libraries that can be used for any search project; But they don’t have a ready-made API that works right out of the box.
Elasticsearch
Elasticsearch is a Lucene library based search engine that is the most popular for full text search. It provides a REST API that is accessed via HTTP JSON. One of its key options, called index sharding, gives you the ability to partition indexes into physical Spaces to improve performance and ensure high availability. Both Lucene and Elasticsearch are designed to handle large data streams, analyze logs, and run complex queries. You can manipulate and analyze documents that match a specified query (for example, calculate the average age of all users named “Thomas “).
Today Lucene and Elasticsearch are the dominant players in the open source search engine space. They are solid solutions for many different use cases in the search world, as well as for building your own recommendation engine. They are good general-purpose products, but need to be configured correctly to get similar results to MeiliSearch or Algolia.
Business closed source
Algolia
Algolia is a SaaS search engine company. Its software is closed source. In its early stages, Algolia offered a mobile search engine that could be embedded into applications and faced the challenge of implementing search algorithms from scratch. From the very beginning, it was decided to build a search engine that was aimed directly at the end user, that is, within a mobile app or website. Algolia has successfully demonstrated over the years how tolerating typos can be critical to improving the user experience, as well as its impact on lower bounce rates and higher conversion rates.
In addition to Algolia, there are a large number of SaaS products available in the search engine market. Most of them use Elasticsearch and fine-tune their Settings to have a customized and personalized solution.
Swiftype
Swiftype is a search service provider specializing in web search and analytics. Swiftype was founded by Matt Riley and Quin Hoxie in 2012 and has been owned by Elastic since November 2017. It’s an end-to-end solution built on top of Elasticsearch, which means it has the ability to leverage Elastic Stack.
Doofinder
Doofinder is a paid onsite search service that was developed to integrate into any web site with minimal configuration. Doofinder is used by online shops to increase sales and is designed to facilitate the buying process.
conclusion
Each search scheme is best suited to the constraints of a particular use case. Because each type of search engine offers a unique set of features, comparing their performance is not easy or relevant. For example, speed comparisons between Product-based databases between Elasticsearch and Algolia are unfair. The same is true for a very large full-text based database.
Therefore, we cannot compare ourselves to Lucene-based or other task-specific search engines.
Of the specific use cases we covered, the solution most similar to MeiliSearch was Algolia.
While Algolia offers the most advanced and powerful search capabilities, this efficiency comes with an expensive price tag. In addition, their services are targeted at large companies.
MeiliSearch is dedicated to serving all types of developers. Our goal is to provide a developer-friendly tool that is easy to install and deploy. Because it’s important to us to provide end-users with a great search experience right out of the box, we want to make it possible for everyone to have the best search experience with minimal effort and with minimal capital investment.