Content Source Enhancement

The content source enhancement connector stamps editable, content-source-level custom_metadata onto every document of a target content source. Unlike static content source custom metadata, which is applied only while a document is being crawled, this connector reaches already-indexed documents — so you can change the metadata at any time and have it applied across the whole target source without re-crawling it.

Use it when a source needs metadata you expect to edit after ingestion — for example a department, a region, or a project tag — and re-crawling the source to change that metadata would be slow or expensive.

How It Works

1. You create a content source with connector: "content_source_enhancement" and point it at a target content source via target_content_source_id.
2. When the enhancement runs, it stamps the configured custom_metadata onto every document the target currently has, merging it into each document's metadata in the index.
3. Editing the custom_metadata and letting the connector run again re-stamps every current target document with the new values.
4. A document the target no longer has keeps no enhancement — its stamp is removed on the next run.

The stamp is layered over a document's own metadata: a per-source value set here takes precedence over a value extracted at crawl time, while a manual document edit still wins over both.

What Triggers a Run

A run is never started by creating or editing the enhancement source itself. It runs when:

• The target finishes a crawl — whenever the target content source completes an ingestion, its enhancements are refreshed automatically.
• You trigger it manually — start an ingestion job for the enhancement content source.
• A schedule is due — if you configure a periodic schedule on the enhancement source, it runs as a catch-all that re-stamps the current target documents and removes any that are gone.

Example Configuration

The following content source attaches department and region to every document of the target content source identified by target_content_source_id:

{
    "name": "Team Metadata",
    "connector": "content_source_enhancement",
    "connector_configuration": {
        "content_source_enhancement": {
            "enhancement_id": "content_source",
            "target_content_source_id": "<id of the target content source>",
            "custom_metadata": {
                "department": "research",
                "region": "emea"
            }
        }
    }
}

The keys you stamp must be configured in the index's document_fields_configuration, the same as for any other metadata; otherwise they are dropped.

Editing and Retargeting

To change the metadata, edit the custom_metadata object on the enhancement content source. The new values are applied to every current target document on the connector's next run.

The target is fixed at creation. To point the enhancement at a different content source, delete the enhancement source and create a new one — changing target_content_source_id on an existing one is rejected. (Editing custom_metadata while keeping the same target is always allowed.) This is because a retarget would otherwise strand the old target's stamp on its still-live documents until they were re-crawled.

Scheduling a Periodic Refresh

To make the enhancement re-run on its own — the "a schedule is due" trigger above — add an enhancement_full entry to scheduled_tasks. This acts as a catch-all: on each due run it re-stamps the target's current documents and removes the stamp from any that are gone, even if a crawl-completion event was missed.

enhancement_full is the only task_type this connector supports — it re-applies the enhancement to existing documents; it does not crawl content or sync access rights. Scheduling any other task type on a content source enhancement source is invalid and the run fails when it starts.

scheduled_tasks is a top-level field on the content source (alongside name and connector), not part of connector_configuration. Each entry is a task_type and a cron schedule:

{
    "name": "Team Metadata",
    "connector": "content_source_enhancement",
    "scheduled_tasks": [
        { "task_type": "enhancement_full", "schedule": "0 3 * * *" }
    ],
    "connector_configuration": {
        "content_source_enhancement": {
            "enhancement_id": "content_source",
            "target_content_source_id": "<id of the target content source>",
            "custom_metadata": { "department": "research" }
        }
    }
}

The 0 3 * * * above runs a full refresh every day at 03:00. You can set or change scheduled_tasks at any time by editing the content source (sending an empty list removes the schedule); a background poller starts the run when it is due. See How a content source runs for the general scheduling model, and Task types for the full set of task_type values.

Content Source Custom Metadata vs. Content Source Enhancement

Scenario	Recommended approach
Same metadata for all documents, set once at crawl time, changed only on the next full crawl	custom_metadata on the connector configuration
Same metadata for all documents of a source, edited after ingestion and applied without re-crawling	Content source enhancement (this connector)
Different metadata per document based on document content	Metadata Extractor Enhancement or Agent Processor Enhancement
Metadata from an external system, matched per document	Join Enhancement

A Note on Safety

Because a run removes the stamp from any document the target no longer has, a target that shrinks sharply (for example after a misconfiguration or a transient source outage) could remove a large share of enhancements at once. To guard against that, if a run would remove more than a configured fraction of the source's enhancements (the platform default is 20%), the removal is blocked — no enhancements are deleted — and the ingestion job finishes as incomplete with a warning reporting how many would have been removed. The rest of the run (re-stamping the target's current documents) still applies. When a large removal is expected, set max_delete_fraction_override on the connector configuration to raise the limit (a value of 1.0 disables the guard) and run again.

API Reference

See Content Source Enhancement in the API reference for the full configuration specification.