Gen AI Assist Settings

The Gen AI Assist Settings page allows administrators to configure integrations with Large Language Models (LLMs) and Vector Stores. These configurations enable the platform to provide AI-assisted features such as intelligent query generation, contextual recommendations, and vector-based semantic search.

Administrators can set up:

  • LLM Settings: Configure LLM providers (e.g., Azure OpenAI) for generating responses and embeddings.

  • Vector Store Settings: Configure backends such as ClickHouse or Qdrant for storing and retrieving embeddings.

Configure LLM Settings

The LLM Settings section is divided into two parts: LLM (Completion Models) and Embed Settings (Embedding Models).

Steps

  1. Navigate to Admin > Configuration > Gen AI Assist Settings.

  2. In the LLM Settings section:

    • Select LLM: Choose the LLM provider from the drop-down (e.g., azure_openai).

    • API Key: Enter the API key for authentication.

    • Model: Specify the model name (e.g., gpt-4o for completions or text-embedding-3-small for embeddings).

    • Endpoint: Provide the endpoint URL of the deployment.

    • API Version: Enter the version of the API (e.g., 2025-01-01-preview or 2024-02-01).

  3. Repeat the same for Embed Settings, ensuring the correct embedding model and endpoint are configured.

  4. Click Save to apply changes.

Configure Vector Store Settings

The Vector Store Settings section allows administrators to configure where embeddings will be stored and queried from. Supported options include ClickHouse and Qdrant.

Steps

  1. In the Vector Store Settings section, select a backend:

    • ClickHouse

    • Qdrant (selected in the example image).

  2. Provide the required connection details:

    • URL: Enter the vector store endpoint (e.g., http://qdrant.qdrant:6333).

    • Host: Specify the host name (e.g., qdrant.qdrant).

    • Port: Provide the port number (e.g., 6333).

    • API Key: Enter the API key for authentication.

  3. Click Save to confirm the configuration.

Key Notes

  • Ensure that the LLM API keys and endpoints are active and accessible before saving settings.

  • The vector store must be running and reachable at the configured host and port.

  • Choose embedding models that are optimized for your use case (e.g., semantic search, classification).

PreviousForm SettingsNextKeycloak Settings

Last updated