Embed Text from Your Own OpenSearch
The sentence encoder serves POST /v1/embeddings with the request and response shape of OpenAI's embeddings API. An OpenSearch cluster can call it through an ML Commons connector, at ingest and at query time. OpenAI clients can call it too.
The endpoint has no authentication. Reach it over the cluster network only.
This guide applies to a self-hosted Zeta Alpha installation with OpenSearch running in the same cluster. In the examples, <encoder-service> is the always-on encoder Service for your model, for example sentence-encoder-api-arctic-embed-2-m, and <namespace> is the namespace it runs in.
The endpoint
| Field | Type | Description |
|---|---|---|
input | string or list of strings | Required. Text longer than the model's maximum length is truncated. |
model | string | Optional and ignored. A deployment serves one model, and the response names it. |
encoding_format | float or base64 | Default float. base64 returns the little-endian float32 bytes of each embedding. |
input_type | query or passage | Default passage. Not part of the OpenAI API. |
dimensions | integer | Optional. Must equal the dimension of the deployed model. |
Models such as E5 and Snowflake Arctic encode a search query differently from a passage. Send "input_type": "query" for the text of a search and leave it out for the text you index.
curl -X POST http://<encoder-service>.<namespace>.svc.cluster.local:8080/v1/embeddings \
--header 'Content-Type: application/json' \
--data '{"input": ["first text", "second text"]}'
{
"object": "list",
"data": [
{"object": "embedding", "index": 0, "embedding": [0.0172, 0.0217, -0.0072]},
{"object": "embedding", "index": 1, "embedding": [0.0231, 0.0104, -0.008]}
],
"model": "snowflake-arctic-embed-m-v2.0",
"usage": {"prompt_tokens": 0, "total_tokens": 0}
}
Embeddings come back in the order of input. usage is always zero. Whether embeddings are normalized is a setting of the deployment.
Call it from OpenSearch
These steps follow OpenSearch's semantic search with OpenAI tutorial and were run on OpenSearch 3.9.0. Every data node and ML node of the cluster must be able to reach the encoder.
Step 1: Allow the encoder URL
PUT /_cluster/settings
{
"persistent": {
"plugins.ml_commons.trusted_connector_endpoints_regex": [
"^http://<encoder-service>\\.<namespace>\\.svc\\.cluster\\.local:8080/v1/embeddings$"
],
"plugins.ml_commons.connector.private_ip_enabled": true
}
}
trusted_connector_endpoints_regex replaces the built-in list of trusted endpoints. Add the ones you already use. Set private_ip_enabled before you deploy a model. A model deployed earlier keeps rejecting private addresses until you undeploy and deploy it again.
Step 2: Create the connector for passages
This is the tutorial's connector with the URL and the model name changed. OpenSearch requires credential, and the encoder ignores it.
POST /_plugins/_ml/connectors/_create
{
"name": "Zeta Alpha encoder, passages",
"description": "Embeds text to index",
"version": "1",
"protocol": "http",
"parameters": {
"model": "snowflake-arctic-embed-m-v2.0"
},
"credential": {
"openAI_key": "unused"
},
"actions": [
{
"action_type": "predict",
"method": "POST",
"url": "http://<encoder-service>.<namespace>.svc.cluster.local:8080/v1/embeddings",
"headers": {
"Authorization": "Bearer ${credential.openAI_key}"
},
"request_body": "{ \"input\": ${parameters.input}, \"model\": \"${parameters.model}\" }",
"pre_process_function": "connector.pre_process.openai.embedding",
"post_process_function": "connector.post_process.openai.embedding"
}
]
}
Step 3: Create the connector for queries
Create a second connector with the same body, a different name, and this request_body:
"request_body": "{ \"input\": ${parameters.input}, \"model\": \"${parameters.model}\", \"input_type\": \"query\" }"
Skip this step for a model that encodes queries and passages the same way, and use the model from step 2 for both.
Step 4: Register and deploy a model for each connector
POST /_plugins/_ml/models/_register?deploy=true
{
"name": "Zeta Alpha encoder, passages",
"function_name": "remote",
"description": "Embeds text to index",
"connector_id": "<passage_connector_id>"
}
Repeat with the query connector. Keep the two model_id values from the responses.
Step 5: Test the models
POST /_plugins/_ml/models/<passage_model_id>/_predict
{
"parameters": {
"input": ["hello world"]
}
}
The response holds one sentence_embedding tensor per input, with data_type FLOAT32.
Step 6: Embed at ingest
PUT /_ingest/pipeline/zeta-alpha-embedding-pipeline
{
"description": "text embedding pipeline",
"processors": [
{
"text_embedding": {
"model_id": "<passage_model_id>",
"field_map": {
"text": "text_knn"
}
}
}
]
}
PUT /my-index
{
"settings": {
"index.knn": true,
"default_pipeline": "zeta-alpha-embedding-pipeline"
},
"mappings": {
"properties": {
"text": {"type": "text"},
"text_knn": {"type": "knn_vector", "dimension": 768, "space_type": "cosinesimil"}
}
}
}
Set dimension to the dimension of the deployed model.
Step 7: Search
Name the query model in the neural query:
POST /my-index/_search
{
"query": {
"neural": {
"text_knn": {
"query_text": "which fruit has potassium",
"model_id": "<query_model_id>",
"k": 10
}
}
}
}
To leave model_id out of every query, set it once in a search pipeline:
PUT /_search/pipeline/zeta-alpha-query-pipeline
{
"request_processors": [
{
"neural_query_enricher": {
"default_model_id": "<query_model_id>"
}
}
]
}
Then search with ?search_pipeline=zeta-alpha-query-pipeline.
Call it from an OpenAI client
from openai import OpenAI
client = OpenAI(
base_url="http://<encoder-service>.<namespace>.svc.cluster.local:8080/v1",
api_key="unused",
)
passages = client.embeddings.create(
input=["first text", "second text"], model="snowflake-arctic-embed-m-v2.0"
)
query = client.embeddings.create(
input="which fruit has potassium",
model="snowflake-arctic-embed-m-v2.0",
extra_body={"input_type": "query"},
)