Semantic Search

In order to use semantic search you must first generate semantic indexes. We say that a semantic index is a searchable index that is the result of converting your searchable documents to vectors, called embeddings, based on their content. Once a semantic index is set up, new documents that are passed to TellusR via the API will automatically be added to it, so long as the documents contain relevant fields for the index.

When checking out queries in the dashboard, the normalized semantic scores are displayed as:

You can manage your semantic indexes under Admin -> Indexing. Here you can configure new semantic indexes and see an overview of ongoing and completed reindexing tasks.

Configure new semantic indexes

• Project: the project from which to take documents.
• Index tag: preferably a short descriptive tag of the index you are about to create. You will need this tag if you will configure search components manually. See Advanced Configuration.
• Language: Select a language. It is recommended to select a language that fits the language of your documents.
• Fields to use in index: Here you can specify which fields should be treated as the documents content. Their content is then joined and converted to an embedding. E.g. if your documents have title and content_segment you can select both fields to create embeddings based on joining title with description.
• Filter fields for index: Select fields to be indexed for each embedding in addition to the embedding. E.g. let’s say your documents have category and you want to perform semantic searches filtered by category. Then category needs to be supplied here.

Only use fields whose content as text is descriptive of the document. Avoid numeric fields and attributes that do not carry any semantically meaningful content.