Content Source Custom Metadata

Content source custom metadata lets you attach static key-value pairs to every document ingested by a content source. This is useful when you need all documents from a source to carry the same metadata - for example, a department, a region, or a project tag - without setting up an enhancement connector.

The metadata is injected at ingestion time by the connector SDK. It works with all standard connectors (SharePoint, Confluence, S3, Google Drive, Slack, GitHub, etc.).

How It Works

1. You add a custom_metadata object to the connector configuration of your content source.
2. On every ingestion run, each crawled document receives the key-value pairs from custom_metadata.
3. If a document already has a field with the same key (from the connector's crawled data), the document's value is kept - content-source-level metadata never overwrites document-level metadata.

Avoid using reserved document field names (title, authors, uri, created_at, last_updated_at, content_source_id, allow_access_rights, deny_access_rights, logo_url) as keys in custom_metadata - these are treated as top-level document fields and may produce unexpected results.

Example Configuration

The following SharePoint connector configuration attaches department and region to every ingested document:

{
    "name": "Product Documentation",
    "description": "SharePoint document library for product docs",
    "is_indexable": true,
    "connector": "sharepoint",
    "schedule": "0 0 * * *",
    "connector_configuration": {
        "sharepoint": {
            "is_document_owner": true,
            "content_source_name": "Product Docs",
            "access_credentials": {
                "client_id": "my_client_id",
                "client_secret": "my_client_secret",
                "tenant_id": "my_tenant_id"
            },
            "site_paths": [
                {
                    "collection_hostname": "company.sharepoint.com",
                    "site_relative_path": "sites/product-docs"
                }
            ],
            "custom_metadata": {
                "department": "research",
                "region": "emea"
            }
        }
    }
}

After ingestion, every document from this content source will have department and region in its metadata. These fields can be used for filtering and faceting at query time, provided they are configured in the index's document_fields_configuration.

Updating Custom Metadata

Content source custom_metadata is applied at crawl time, so editing it does not change already-indexed documents on its own — the previous values remain until the source is re-crawled. To apply an update, modify the custom_metadata object on the content source and trigger a re-crawl (manually or via the next scheduled run); the new values are applied to all documents during that ingestion.

If you need content-source-level metadata that you can edit at any time and have applied to existing documents without re-crawling, use the Content Source Enhancement Connector instead.

When to Use Custom Metadata vs. Enhancements

Scenario	Recommended approach
Same metadata for all documents in a content source, applied at crawl time	custom_metadata on the connector configuration
Same metadata for all documents, editable and applied to existing documents without re-crawling	Content Source Enhancement Connector
Different metadata per document based on document content	Metadata Extractor Enhancement or Agent Processor Enhancement
Metadata from an external system, matched per document	Join Enhancement or Custom Enhancement
User-defined tags on individual documents	Tags Enhancement

API Reference

See BaseConnectorConfiguration.custom_metadata in the API reference for the full field specification.