Pipelines :: TellusR − Documentation

Pipelines

This page provides a detailed description of the available pipelines in TellusR and their supported parameters.

A pipeline is a retrieval algorithm that belongs to a project. It takes input parameters and typically returns TellusR documents from a project.

flowchart TD
  Q["query parameters (JSON payload)"] --> P["pipeline"]
  I["project indexes / collections"] --> P
  P --> R["list of results with metadata"]

tellusrSearch

Implements hybrid search (keyword and semantic). Returns a result list with optional highlights, facets, and filters.

Parameter	Type	Description
`q`	String	The free text query to perform.
`hl`	Boolean	If set to true, the response will contain regular and semantic highlights for the hits. Default value: `false`.
`limit`	Integer	Determines how many results to show. Default value: `10`.
`start`	Integer	A pagination parameter. Determines where in the result list to start showing results. Default value: `0`.
`proj`	String	Project onto certain document fields. E.g., `author,title`.
`semanticWeight`	Number	Control the balance between semantic and regular results by providing a number between 0 (regular) and 1 (semantic). Default value: `0.5`.
`focus`	Number	Parametrizes the transition between focusing on the best matching part of a document (`focus=0`) vs. the best matching overall document (`focus=1`) when querying. This effect is only noticeable for large documents. Default value: `0.5`.
`fq`	String	The filter to apply. It follows the Lucene query parser spec.
`facet`	Object	Defines facets to be returned. Maps field names to configurations. Supported types: Value: `{"type": "value", "limit": 10}` Range: `{"type": "range", "ranges": [{"from": 0, "to": 100, "name": "low"}]}` or `{"type": "range", "rangeFactory": {"from": 0, "to": 100, "step": 10}}` Prefix: `{"type": "prefix", "prefixes": ["root/history", "/root/math", "/root/science"], "limit": 10}`
`sort`	String	A sorting specification. By default, documents are sorted by score. To sort by a different field, e.g., `date:desc`, `date:asc`. In both cases, documents with absent field values are placed last. Sorting can also be done lexicographically, e.g., `date:desc,score:desc`.

Example

POST /tellusr/api/v1/${YOUR_PROJECT}/compute/tellusrSearch
{
  "q": "search term",
  "semanticWeight": 0.5,
  "facet": {
    "category": { "type": "value", "limit": 10 },
    "price": { "type": "range", "rangeFactory": { "from": 0, "to": 1000, "step": 100 } }
  }
}

Response Example

{
  "params": {
    "hl": true,
    "limit": 5,
    "proj": [],
    "q": "roof",
    "start": 0
  },
  "info": {
    "combined.cacheHit": false,
    "contextparsing": {},
    "count.results.regular": 0,
    "count.results.semantic": 66,
    "count.results.total": 66,
    "time.millis.semantic.main": 55,
    "time.millis.semantic.title": 343,
    "time.millis.total": 454
  },
  "results": [
    {
      "id": "doc_001",
      "title": "Building Regulations - Chapter 10",
      "chapter": "Construction Safety",
      "score": 1,
      "highlights": [
        {
          "field": "content_segment",
          "highlight": "<em>Roofs</em> with sufficient slope for snow to slide must be secured in their entirety with snow guards.",
          "metadata": {
            "chunk_index": 1,
            "chunks": 3
          },
          "matchType": "regular"
        }
      ],
      "url": "https://example.com/regs/10-3"
    },
    {
      "id": "doc_002",
      "title": "Energy Efficiency Standards",
      "chapter": "Energy",
      "score": 0.91,
      "highlights": [
        {
          "field": "content_segment",
          "highlight": "Minimum requirements for <em>roof</em> insulation U-values must be met for all new constructions.",
          "metadata": {
            "chunk_index": 0,
            "chunks": 2
          },
          "matchType": "semantic"
        }
      ],
      "url": "https://example.com/regs/14-3"
    }
  ]
}

topChunkRag

Used for RAG (Retrieval-Augmented Generation) workflows, focusing on retrieving the most relevant text chunks.

Parameter	Type	Description
`q`	String	The query to find relevant chunks for.
`limit`	Integer	Determines how many results to show. Default value: `10`.
`start`	Integer	A pagination parameter. Determines where in the result list to start showing results. Default value: `0`.
`semanticWeight`	Number	Control the balance between semantic and regular results by providing a number between 0 (regular) and 1 (semantic). Default value: `0.5`.
`fq`	String	The filter to apply. It follows the Lucene query parser spec.
`highlightWindow`	Integer	The number of surrounding chunks to fetch for each hit, providing additional context. Default value: `3`.
`rag.hl.metadata`	String	A comma-separated list of metadata fields to include for each chunk. Can be any field name present in the chunks (e.g., `page_origin`, `headings`, `csv_as_json`, `category_paths`).
`rag.simplify`	Boolean	If set to true, simplifies the response by merging relevant chunks into a single `text` field and reducing metadata overhead.

Example

POST /tellusr/api/v1/${YOUR_PROJECT}/compute/topChunkRag
{
  "q": "What is the capital of France?"
}

Response Examples

Standard (Unsimplified) Response

By default (rag.simplify: false), the response contains a detailed list of hits, each with its relevant segments and hit information. This format is useful when you need precise page references or specific metadata for each retrieved chunk.

{
  "params": {
    "limit": 1,
    "proj": [],
    "q": "torus"
  },
  "results": [
    {
      "doc_id": "6873b11e1ea2be273fdc3afe78323a14c3eba57f163568f3baffbbca62ae9d9d",
      "title": "Topology",
      "docHits": [
        {
          "bestSegment": {
            "field": "content_segment",
            "value": "| Manifold ...",
            "chunk_index": 10,
            "metadata": {
              "page_origin": 5,
              "headings": ["Manifold", ...]
            }
          },
          "relevantSegments": [
            {
              "field": "content_segment",
              "value": "Further contributions ...",
              "chunk_index": 9,
              "metadata": { "page_origin": 4, "headings": [] }
            },
            {
              "field": "content_segment",
              "value": "| Manifold ...",
              "chunk_index": 10,
              "metadata": { "page_origin": 5, "headings": [...] }
            },
            {
              "field": "content_segment",
              "value": "Unifying the work ...",
              "chunk_index": 11,
              "metadata": { "page_origin": 5, "headings": [] }
            }
          ]
        }
      ],
      "score": 1
    }
  ]
}

Simplified Response

When rag.simplify is set to true, the response structure is flattened, merging relevant chunks into a single text field. This is ideal for passing directly to an LLM.

{
  "params": {
    "q": "torus",
    "highlightWindow": 3,
    "limit": 1,
    "rag.simplify": true
  },
  "results": [
    {
      "doc_id": "6873b11e1ea2be273fdc3afe78323a14c3eba57f163568f3baffbbca62ae9d9d",
      "title": "Topology",
      "docHits": [
        {
          "text": "... [merged text from chunks] ...",
          "metadata": {
            "page_origin": 5,
            "headings": []
          }
        }
      ],
      "score": 1
    }
  ]
}

topDocRag

Used for RAG workflows, focusing on retrieving the most relevant documents.

Parameter	Type	Description
`q`	String	The query to find relevant documents for.
`limit`	Integer	Determines how many documents are considered. Default value: `10`.
`start`	Integer	A pagination parameter. Determines where in the result list to start showing results. Default value: `0`.
`semanticWeight`	Number	Control the balance between semantic and regular results by providing a number between 0 (regular) and 1 (semantic). Default value: `0.5`.
`fq`	String	The filter to apply. It follows the Lucene query parser spec.
`subResultsN`	Integer	The number of additional chunk hits to find within each top document. Default value: `3`.
`highlightWindow`	Integer	The number of surrounding chunks to fetch for each hit, providing additional context. Default value: `3`.
`rag.hl.metadata`	String	A comma-separated list of metadata fields to include for each chunk. Can be any field name present in the chunks (e.g., `page_origin`, `headings`, `csv_as_json`, `category_paths`).
`rag.simplify`	Boolean	If set to true, simplifies the response by merging relevant chunks into a single `text` field and reducing metadata overhead.

Example
POST /tellusr/api/v1/${YOUR_PROJECT}/compute/topDocRag
{
  "q": "company policy on remote work"
}
Response Examples
The response structure for topDocRag follows the same format as topChunkRag. See the Response Examples section above for both standard and simplified formats.

documentStream

An iterator that allows extracting all documents in a project. It returns a batch of documents and an iterator token to fetch the next batch.

Parameter	Type	Description
`iterator`	String	(Optional) The iterator token provided in the previous response’s `info.iterator` field. Omit for the first call to start from the beginning.
`filter`	String	(Optional) To retrieve documents filtered by the value of a field. Format: `<field>:<value>`.

The pipeline returns a JSON object with two main fields:

results: An array of documents in the current batch.
info: Contains an iterator string used to retrieve the next batch.

To iterate through all documents, continue calling the endpoint with the iterator parameter until the results array is empty.

Example

Initial call:

GET /tellusr/api/v1/${YOUR_PROJECT}/compute/documentStream

Filtered call:

GET /tellusr/api/v1/${YOUR_PROJECT}/compute/documentStream?filter=category:news

Subsequent calls:

GET /tellusr/api/v1/${YOUR_PROJECT}/compute/documentStream?iterator=ITERATOR_TOKEN_FROM_PREVIOUS_RESPONSE

autocorrect

Corrects misspellings and provides completions for common phrases.

Parameter	Type	Description
`q`	String	The text to be corrected or completed.

Example

POST /tellusr/api/v1/${YOUR_PROJECT}/compute/autocorrect
{
  "q": "tellusrr"
}

suggestDocs

Suggests documents whose titles match the query.

Parameter	Type	Description
`q`	String	The query to match against document titles.

Example

POST /tellusr/api/v1/${YOUR_PROJECT}/compute/suggestDocs
{
  "q": "introduction"
}

inspectDocument

Fetches a full document by its unique ID.

Parameter	Type	Description
`id`	String	The unique identifier of the document.

Example

POST /tellusr/api/v1/${YOUR_PROJECT}/compute/inspectDocument
{
  "id": "doc_123"
}