> ## Documentation Index
> Fetch the complete documentation index at: https://docs.clinia.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Highlighting

> How to highlight search results

It is important to be able to show your user *why* a certain result is relevant to their search,
highlighting can fill this role in a few different ways.

## Vector

Highlights in a vector search context only make sense if the search is done on array properties;
the highlight will return the most relevant array items along with their scores, while the resource property
will return the entire property content. This allows you to show the user which part of the array
matched the query.

Highlights are returned for the three most relevant items for every *matched* resource. *Matched*
resources are found through HNSW (Hierarchical Navigable Small World) search.

<Note>
  Consider the threshold an implementation detail, it may change from model to model and Clinia takes
  care of fine-tuning it to return relevant results.

  The score will vary between 0 and 1. It is the cosine similarity between the passage and the knn operator value.
</Note>

```json theme={null}
// Example highlighting property of a query hit where path.to.property is an array property
{
  "path.to.property": [
    {
      "data": "passage matched by the query",
      "path": "path.to.property.0", // where 0 is the index of the passage when highlighting a segmented property
      "score": 0.9, // score range can vary from model to model, use this in relation to other highlights
      "type": "vector" // discriminator against textual highlights
    }
  ]
}
```

## Textual

Highlights for `match` operators directly return the matched text with `<em>` tags around the matched terms.

```json theme={null}
{
  "path.to.property": [
    {
      "highlight": "There is a matched <em>word</em> in this passage",
      "type": "textual"
    }
  ]
}
```

## Examples

The following setup is required to run the subsequent code snippets, but you can skip it and still
follow along with the rest of the tutorial if you prefer!

In short, it sets up and ingests data for a profile with three properties:

* `title`: a `symbol` property
* `abstract`: a `symbol` property that is vectorized
* `content`: an array of objects, each object having two `symbol` properties: `sectionTitle` and `text`.
  The `text` property is vectorized.

<Accordion title="Required setup">
  To correctly showcase highlights, we need to set up a registry source with an embedding pipeline.
  For the sake of this tutorial, we will be using the `prestigious-journal` source and the `articles` collection.

  <Steps>
    <Step title="Create a source">
      ```bash {collectForTests="source"} theme={null}
      curl -X PUT "https://$CLINIA_WORKSPACE/catalog/v1/sources/prestigious-journal" \
        -H "X-Clinia-API-Key: $CLINIA_TOKEN" \
        -H "Content-Type: application/json" \
        -d '{
        "type": "registry"
      }'
      ```
    </Step>

    <Step title="Create a collection">
      ```bash {collectForTests="profile"} theme={null}
      curl -X PUT "https://$CLINIA_WORKSPACE/sources/prestigious-journal/v1/collections/articles" \
        -H "X-Clinia-API-Key: $CLINIA_TOKEN" \
        -H "Content-Type: application/json" \
        -d '{
        "type": "resources",
        "profile": {
          "properties": {
            "title": {
              "type": "symbol"
            },
            "abstract": {
              "type": "symbol"
            },
            "content": {
              "type": "array",
              "items": {
                "type": "object",
                "properties": {
                  "text": {
                    "type": "symbol"
                  },
                  "sectionTitle": {
                    "type": "symbol"
                  }
                }
              }
            }
          }
        }
      }'
      ```
    </Step>

    <Step title="Create an embedding pipeline">
      ```bash {collectForTests="pipeline"} theme={null}
      curl -X PUT "https://$CLINIA_WORKSPACE/sources/prestigious-journal/v1/collections/articles/pipeline" \
        -H "X-Clinia-API-Key: $CLINIA_TOKEN" \
        -H "Content-Type: application/json" \
        -d '{
        "steps": [
          {
            "type": "VECTORIZER",
            "vectorizer": {
              "inputProperty": "abstract",
              "modelId": "text-embedding-004",
              "propertyKey": "vector",
              "provider": "google",
              "dimensions": 768
            }
          },
          {
            "type": "VECTORIZER",
            "vectorizer": {
              "inputProperty": "content.text",
              "modelId": "text-embedding-004",
              "propertyKey": "vector",
              "provider": "google",
              "dimensions": 768
            }
          }
        ]
      }'
      ```
    </Step>

    <Step title="Add sample data">
      ```bash {collectForTests="sampleData"} theme={null}
      curl -X POST "https://$CLINIA_WORKSPACE/sources/prestigious-journal/v1/resources/bulk" \
        -H "X-Clinia-API-Key: $CLINIA_TOKEN" \
        -H "Content-Type: application/json" \
        -d '{
        "operations": [
          {
            "action": "CREATE",
            "create": {
              "type": "articles",
              "data": {
                "title": "Metabolic Resilience Index: Continuous Multi-Sensor Signatures for Early Detection of Dysmetabolic Risk",
                "abstract": "Researchers proposing an integrated \"Metabolic Resilience Index\" argue that subtle shifts in glucose variability precede overt fasting hyperglycemia. In their conceptual framework, circadian misalignment amplifies low‑grade inflammation through maladaptive cortisol rhythms. They describe how wearable sensor data—heart rate variability, peripheral temperature, and sleep fragmentation—can triangulate emerging autonomic imbalance. The model asserts that postprandial spikes combined with elevated nocturnal glucose plateaus predict mitochondrial oxidative stress. Mitochondrial efficiency is inferred indirectly via delayed recovery of resting heart rate after mild exertion. The authors layer in gut microbiome diversity metrics, noting decreased short‑chain fatty acid proxy scores in tandem with rising inflammatory cytokine panels. They suggest that composite biomarker clustering outperforms any single lab value for early metabolic syndrome detection. Beta cell \"whisper distress\" is depicted as a phase where insulin pulsatility dampens before fasting labs appear abnormal. Subjective fatigue ratings correlate with sleep efficiency dips on days of higher glycemic excursions. A proposed dashboard flags when rolling 7‑day variability in glucose exceeds a personalized threshold. Cortisol awakening response flattening is labeled a sentinel of hypothalamic–pituitary axis strain. The narrative links microglial priming to systemic inflammatory tone during chronic circadian disruption. A feedback loop is illustrated where inflammation impairs mitochondrial turnover, worsening energetic flexibility. They emphasize that early intervention windows are missed when clinicians focus solely on annual fasting labs. Proposed interventions include light timing hygiene and meal distribution realignment rather than immediate pharmacology. The concept paper also mentions that residual post-lunch glycemic tails predict evening cravings. A pilot simulation shows that reducing late eating narrows nocturnal glucose variability bands. The framework highlights that continuous metrics enable detection of inflection points, not just static abnormalities. An uncertainty layer is added to prevent overconfidence in noisy wearable signals. Finally, the authors call for federated learning to refine the composite inflammation and glucose variability signatures across diverse populations.",
                "content": [
                  {
                    "sectionTitle": "Methods",
                    "text": "We conducted a longitudinal study involving 500 participants monitored over 12 months using wearable sensors that tracked heart rate variability, glucose levels, sleep patterns, and physical activity."
                  },
                  {
                    "sectionTitle": "Methods",
                    "text": "Data were collected in real-time and analyzed using machine learning algorithms to identify patterns indicative of metabolic resilience or vulnerability."
                  },
                  {
                    "sectionTitle": "Results",
                    "text": "Preliminary findings indicate that individuals with lower MRI scores exhibited higher variability in glucose levels and reduced heart rate variability, correlating with increased inflammatory markers."
                  },
                  {
                    "sectionTitle": "Conclusion",
                    "text": "Notably, these changes were detectable weeks before traditional clinical indicators of metabolic syndrome appeared."
                  }
                ]
              }
            }
          }
        ]
      }'
      ```
    </Step>

    <Step title="Wait for ingestion to complete">
      Look at the [Task API guide](/guides/managing-data/tasks#checking-task-status) to better understand how
      to poll for task status. This is not done here since it cannot be expressed as a single curl command.
    </Step>
  </Steps>
</Accordion>

### Vector search on a [Symbol](/explanation/data-types/base-types#text-and-string-types) property

Vector search using the `knn` operator on the vectorized `abstract.vector` property.
Vector highlighting requires a property that is an array or nested inside one. Since abstract is a flat symbol, the response includes no highlighting field.

```bash {collectForTests=query} theme={null}
curl -X POST "https://$CLINIA_WORKSPACE/sources/prestigious-journal/v1/collections/articles/query" \
  -H "X-Clinia-API-Key: $CLINIA_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
  "query": {
    "knn": {
      "abstract.vector": {
        "value": "circadian inflammation"
      }
    }
  }
}'
```

### Textual search on a vectorized [Symbol](/explanation/data-types/base-types#text-and-string-types) property

Even though `abstract` is vectorized, you can still request textual highlights on it using a `match`
operator. Textual and vector highlights on the same property are independent — one highlight entry is
returned per match occurrence, with the matched term wrapped in `<em>` tags.

```bash {collectForTests=query2} theme={null}
curl -X POST "https://$CLINIA_WORKSPACE/sources/prestigious-journal/v1/collections/articles/query" \
  -H "X-Clinia-API-Key: $CLINIA_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
  "query": {
    "match": {
      "abstract": {
        "value": "glucose",
        "type": "word"
      }
    }
  },
  "highlighting": [
    "abstract"
  ]
}'
```

```json {collectForTests=response2} theme={null}
// response shortened to .hits[0].highlighting
{
  "highlighting": {
    "abstract": [
      {
        "type": "textual",
        "highlight": "Researchers proposing an integrated \"Metabolic Resilience Index\" argue that subtle shifts in <em>glucose</em>"
      },
      {
        "type": "textual",
        "highlight": "The model asserts that postprandial spikes combined with elevated nocturnal <em>glucose</em> plateaus predict"
      },
      {
        "type": "textual",
        "highlight": "A proposed dashboard flags when rolling 7‑day variability in <em>glucose</em> exceeds a personalized threshold"
      },
      {
        "type": "textual",
        "highlight": "A pilot simulation shows that reducing late eating narrows nocturnal <em>glucose</em> variability bands."
      },
      {
        "type": "textual",
        "highlight": "Finally, the authors call for federated learning to refine the composite inflammation and <em>glucose</em> variability"
      }
    ]
  }
}
```

### Textual search on a non-vectorized [Symbol](/explanation/data-types/base-types#text-and-string-types) property

Textual search works the same on properties without a vectorizer. One highlight entry is returned
per match occurrence, with the matched term wrapped in `<em>` tags.

```bash {collectForTests=query3} theme={null}
curl -X POST "https://$CLINIA_WORKSPACE/sources/prestigious-journal/v1/collections/articles/query" \
  -H "X-Clinia-API-Key: $CLINIA_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
  "query": {
    "match": {
      "title": {
        "value": "metabolic",
        "type": "word"
      }
    }
  },
  "highlighting": [
    "title"
  ]
}'
```

```json {collectForTests=response3} theme={null}
// response shortened to .hits[0].highlighting
{
  "highlighting": {
    "title": [
      {
        "highlight": "<em>Metabolic</em> Resilience Index: Continuous Multi-Sensor Signatures for Early Detection of Dysmetabolic Risk",
        "type": "textual"
      }
    ]
  }
}
```

### Multiple textual queries on the same [Symbol](/explanation/data-types/base-types#text-and-string-types) property

Textual highlights combine all `match` queries on a same property.

```bash {collectForTests=query4} theme={null}
curl -X POST "https://$CLINIA_WORKSPACE/sources/prestigious-journal/v1/collections/articles/query" \
  -H "X-Clinia-API-Key: $CLINIA_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
  "query": {
    "and": [
      {
        "match": {
          "title": {
            "value": "metabolic",
            "type": "word"
          }
        }
      },
      {
        "match": {
          "title": {
            "value": "Dysmetabolic",
            "type": "word"
          }
        }
      }
    ]
  },
  "highlighting": [
    "title"
  ]
}'
```

```json {collectForTests=response4} theme={null}
// response shortened to .hits[0].highlighting
{
  "highlighting": {
    "title": [
      {
        "highlight": "<em>Metabolic</em> Resilience Index: Continuous Multi-Sensor Signatures for Early Detection of <em>Dysmetabolic</em> Risk",
        "type": "textual"
      }
    ]
  }
}
```

### Vector search on an [Array](/explanation/data-types/base-types#array) property

When your data contains arrays, you can use the `knn` operator on the
vectorized property to retrieve the most semantically relevant passages as highlights.

In this example, the `content` property is an array of objects, each object having a `text` symbol
property vectorized at `content.text.vector`. The highlights return the top matching passages with
their similarity scores.

```bash {collectForTests=query5} theme={null}
curl -X POST "https://$CLINIA_WORKSPACE/sources/prestigious-journal/v1/collections/articles/query" \
  -H "X-Clinia-API-Key: $CLINIA_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
  "query": {
    "knn": {
      "content.text.vector": {
        "value": "glucose"
      }
    }
  },
  "highlighting": [
    "content.text.vector"
  ]
}'
```

```json {collectForTests=response5} theme={null}
// response shortened to .hits[0].highlighting
{
  "highlighting": {
    "content": [
      {
        "data": "We conducted a longitudinal study involving 500 participants monitored over 12 months using wearable sensors that tracked heart rate variability, glucose levels, sleep patterns, and physical activity.",
        "path": "content.0.text",
        "score": 0.8712,
        "type": "vector"
      },
      {
        "data": "Preliminary findings indicate that individuals with lower MRI scores exhibited higher variability in glucose levels and reduced heart rate variability, correlating with increased inflammatory markers.",
        "path": "content.2.text",
        "score": 0.8547,
        "type": "vector"
      }
    ]
  }
}
```

### Multiple vector queries on the same [Array](/explanation/data-types/base-types#array) property

<Warning>
  The following query *does not work*. You cannot request highlights for a vector property that is queried
  by two different `knn` operators since there would be no way to know which highlight corresponds to
  which `knn` operator.
</Warning>

```bash {collectForTests=query6} theme={null}
curl -X POST "https://$CLINIA_WORKSPACE/sources/prestigious-journal/v1/collections/articles/query" \
  -H "X-Clinia-API-Key: $CLINIA_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
  "query": {
    "or": [
      {
        "knn": {
          "content.text.vector": {
            "value": "glucose"
          }
        }
      },
      {
        "knn": {
          "content.text.vector": {
            "value": "MRI scores"
          }
        }
      }
    ]
  },
  "highlighting": [
    "content.text.vector"
  ]
}'
```

### Textual search on an [Array](/explanation/data-types/base-types#array) of objects

You can also request textual highlights on an array of objects. A separate highlight entry is returned
for each array item that matches the query.

```bash {collectForTests=query7} theme={null}
curl -X POST "https://$CLINIA_WORKSPACE/sources/prestigious-journal/v1/collections/articles/query" \
  -H "X-Clinia-API-Key: $CLINIA_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
  "query": {
    "match": {
      "content.text": {
        "value": "glucose",
        "type": "word"
      }
    }
  },
  "highlighting": [
    "content.text"
  ]
}'
```

```json {collectForTests=response7} theme={null}
// response shortened to .hits[0].highlighting
{
  "highlighting": {
    "content.text": [
      {
        "highlight": "participants monitored over 12 months using wearable sensors that tracked heart rate variability, <em>glucose</em>",
        "type": "textual"
      },
      {
        "highlight": "Preliminary findings indicate that individuals with lower MRI scores exhibited higher variability in <em>glucose</em>",
        "type": "textual"
      }
    ]
  }
}
```
