Search Engineer#
Profile: Search Engineer
The search engineer is responsible for enhancing the search experience for end users.
This page provides information on the scope of the search engineer role and links to relevant parts of Squirro’s documentation.
Introduction#
Squirro Cognitive Search is a core component of the Squirro platform.
Its purpose is to connect end users with the data they’re looking for as quickly and accurately as possible through an Enterprise search experience. In some cases, it means exposing end users to data they didn’t know they were looking for, but which is relevant to them.
The search engineer is responsible for configuring and enhancing the technical components of the search experience.
The Project Creator then uses the search engineer’s work to create a search experience that is tailored to the end user’s needs by configuring search settings within projects.
Query Processing#
Search engineers can configure the query processing pipeline and should be familiar with the components of the pipeline.
The query processing pipeline is a series of steps that are applied to a user’s query before it is sent to the search engine.
The figure below illustrates how query processing fits into Squirro’s overall architecture.
Reference: To learn more, see Query Processing.
Note
Search engineers should also be familiar with Squirro’s Pipeline Editor, which is the drag-and-drop visual editor used to add, remove, and edit steps within Squirro’s Data Processing Pipeline.
Query-processing how-to guides include the following:
libNLP#
Squirro’s natural language processing library (libNLP) is a text-processing library that is used to extract entities and concepts from text.
It powers Query Processing, as well as other Squirro applications, including the no-code Machine Learning (ML) AI Studio.
Reference: To learn more, see libNLP.
Custom libNLP Steps#
Part of Squirro’s power is its extensibility. Search engineers can create custom steps to add to the query processing pipeline to enhance the search experience using boosters, classifiers, expanders, language detectors, and other tools.
Reference: For a step-by-step guide, see How to Create Custom Query-Processing Steps.
Document Relevancy#
Squirro’s baseline full-text search provides a document relevancy score of BM25 out of the box.
But with different domains, users, and preferences, search engineers can influence the relevancy of documents in search results in multiple ways.
Reference: To learn more, see Document Relevancy.
Semantic and Hybrid Search#
The next generation of search is semantic search, which uses machine learning to understand the meaning of a query and the content of a document.
To learn more about Squirro’s implementation of semantic search, including Hybrid Search, see Semantic and Hybrid Search.
Recommendations#
Squirro can make recommendations to end users based on various types of inputs, including search queries.
As a search engineer, you can customize these recommendations to suit project needs.
Recommendations in Squirro work based on the following:
Correlated labels
Non-correlated labels
Machine learning
Reference: To learn more, see Recommendations.