OpenSearch Setup

This guide explains how to set up OpenSearch locally using Docker Compose. OpenSearch is an open-source search engine that is compatible with Elasticsearch and can be used as a drop-in replacement for the vector search capabilities in TextLayer Core.

Prerequisites

Connection Details

When connecting to OpenSearch from TextLayer Core, use the following environment variables:
OPENSEARCH_HOST: localhost
OPENSEARCH_USER: admin
OPENSEARCH_PASSWORD: `{your-secure-password}`
OPENSEARCH_PORT: 9200

Dashboard Access

You can access the OpenSearch Dashboard at http://localhost:5601/app/dev_tools/console with the following credentials:
  • Username: admin
  • Password: {your-secure-password}

Docker Compose Configuration

Create a docker-compose.yml file with the following configuration:
services:
  opensearch:
    image: opensearchproject/opensearch:2.13.0
    container_name: opensearch
    environment:
      - discovery.type=single-node
      - bootstrap.memory_lock=true
      - "OPENSEARCH_JAVA_OPTS=-Xms1g -Xmx1g"
      - plugins.security.ssl.http.enabled=false
      - plugins.security.allow_default_init_securityindex=true
      - plugins.security.authcz.admin_dn=CN=admin,OU=Ops,O=Example Com,DC=example,DC=com
      - OPENSEARCH_INITIAL_ADMIN_PASSWORD=`{your-secure-password}`
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - opensearch-data:/usr/share/opensearch/data
    ports:
      - 9200:9200
      - 9600:9600
    networks:
      - opensearch-net

  opensearch-dashboards:
    image: opensearchproject/opensearch-dashboards:2.13.0
    container_name: opensearch-dashboards
    ports:
      - 5601:5601
    environment:
      - OPENSEARCH_HOSTS=http://opensearch:9200
      - OPENSEARCH_USERNAME=admin
      - OPENSEARCH_PASSWORD=`{your-secure-password}`
      - OPENSEARCH_SSL_VERIFICATIONMODE=none
    depends_on:
      opensearch:
        condition: service_started
    networks:
      - opensearch-net

volumes:
  opensearch-data:

networks:
  opensearch-net:
    driver: bridge

Starting OpenSearch

To start OpenSearch and the dashboard, run the following command in the directory containing your docker-compose.yml file:
docker-compose up -d
This will start the services in detached mode. You can view the logs with:
docker-compose logs -f
To stop the services:
docker-compose down

Connecting from Python

When connecting to OpenSearch from your Python application, make sure to set use_ssl=False for local development:
session = OpenSearch(
    hosts=[{"host": os_host, "port": os_port}],
    http_auth=(os_user, os_password),
    use_ssl=False,
    verify_certs=True,
    ssl_assert_hostname=False,
    ssl_show_warn=False,
    timeout=30,
    max_retries=3,
    retry_on_timeout=True,
)

Verification

To verify that OpenSearch is running correctly, you can make a request to the health endpoint:
curl -u admin:`{your-secure-password}` http://localhost:9200/_cluster/health
The response should indicate that the cluster is in a “green” or “yellow” state.