# Elasticsearch

{% hint style="info" %}
Elasticsearch is an SIEM integration. SIEM integrations are used to perform analysis of any SIEM generated alerts, and/or to use generated data as part of investigation analysis.
{% endhint %}

The Dropzone platform integrates with the [Elasticsearch](https://www.elastic.co/elasticsearch) security SIEM. Many customers ingest other alert sources into Elasticsearch (e.g. IDPs) and integrate Dropzone into Elasticsearch rather than the source systems. Dropzone supports both Cloud deployments and On-premise deployments.

## Create an API Key and Obtain a Cloud ID

Elasticsearch requires an API Key and an Elasticsearch Cloud ID to enable.

{% hint style="info" %}
If you are using the Elasticsearch Serverless Projects-Based Model or an On-premise Elasticsearch using the Dropzone connector, you will not need to provide a Cloud ID.
{% endhint %}

To obtain an API Key, do the following:

* Navigate to your [Elastic Cloud home page](https://cloud.elastic.co/home) or deployment
* Under the Hosted Deployments section, locate the deployment you wish Dropzone.AI to be able to access
* Click "Open"

<figure><img src="https://435022081-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FmBVcKuiytGCsIDpL70BC%2Fuploads%2Fgit-blob-0d8989dced05bb928bef74d23abc19673a644c67%2Felasticsearch-integration-1.png?alt=media" alt=""><figcaption><p>Click Manage</p></figcaption></figure>

* In the Deployment overview page, click "Management" in the bottom left corner
* Click the icon next to Stack Management
* Navigate to API keys

<figure><img src="https://435022081-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FmBVcKuiytGCsIDpL70BC%2Fuploads%2Fgit-blob-f9bf3d80de56ae40c3d18b66ba0e81c9e818517c%2Felasticsearch-integration-3.png?alt=media" alt=""><figcaption><p>Navigate to API keys</p></figcaption></figure>

* Click "Create an API key"

<figure><img src="https://435022081-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FmBVcKuiytGCsIDpL70BC%2Fuploads%2Fgit-blob-68b47fd4003d2014244a24d3f4033b4600adb879%2Felasticsearch-integration-4.png?alt=media" alt=""><figcaption><p>Click "Create an API key"</p></figcaption></figure>

* Name the API key something memorable, such as Dropzone.AI
* Under type, select User API key
* Click "Create API Key"

<figure><img src="https://435022081-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FmBVcKuiytGCsIDpL70BC%2Fuploads%2Fgit-blob-733821b4086bf7d0a14e0c1ae89b72cf2de03b6a%2Felasticsearch-integration-5.png?alt=media" alt=""><figcaption><p>Create an API key></p></figcaption></figure>

* Copy the API key generated for use later in the Dropzone UI, where it is called "API Key"

<figure><img src="https://435022081-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FmBVcKuiytGCsIDpL70BC%2Fuploads%2Fgit-blob-f964d8d94be39c7e7e0bbd083a1cc265a40c2b52%2Felasticsearch-integration-6.png?alt=media" alt=""><figcaption><p>Copy the key</p></figcaption></figure>

To obtain your Elasticsearch Cloud ID, do the following:

* Navigate to your [Elastic Cloud home page](https://cloud.elastic.co/home)
* Under the Hosted Deployments section, locate the deployment you wish Dropzone.AI to be able to access
* Click "Open"

<figure><img src="https://435022081-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FmBVcKuiytGCsIDpL70BC%2Fuploads%2Fgit-blob-0d8989dced05bb928bef74d23abc19673a644c67%2Felasticsearch-integration-1.png?alt=media" alt=""><figcaption><p>Click Open</p></figcaption></figure>

* In the upper right of the Overview page, click "Endpoint & API Keys"

<figure><img src="https://435022081-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FmBVcKuiytGCsIDpL70BC%2Fuploads%2Fgit-blob-d24e82d0e80b51f7385b4970ef70478c783ba152%2Felasticsearch-integration-2.png?alt=media" alt=""><figcaption><p>Click Endpoint &#x26; API Keys</p></figcaption></figure>

* Check "Show Cloud ID"
* Copy the value shown for use later in the Dropzone UI, where it is called "Elasticsearch Cloud ID"

<figure><img src="https://435022081-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FmBVcKuiytGCsIDpL70BC%2Fuploads%2Fgit-blob-ada277487a3f8e33c2b1d2784c83b476e4e81086%2Felasticsearch-integration-7.png?alt=media" alt=""><figcaption><p>Copy the Elasticsearch Cloud ID</p></figcaption></figure>

## Enable Elasticsearch

To enable the Alert Source integration, you will need the following information:

| Dropzone Field         | Source                                                                                              |
| ---------------------- | --------------------------------------------------------------------------------------------------- |
| Elasticsearch Cloud ID | The cloud ID value copied earlier. Only necessary if you have an Elastic Cloud Hosted deployment    |
| Elasticsearch Server   | The server for your Elasticsearch project, e.g. <https://my-project.es.us-west-2.aws.elastic.cloud> |
| API Token              | The API token value generated earlier                                                               |

To enable the Alert Source integration, do the following:

* Navigate to your Dropzone AI tenant home page e.g. https\://*mycompany*.dropzone.app
* In the bottom left hand corner, navigate to Settings > Integrations

<figure><img src="https://435022081-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FmBVcKuiytGCsIDpL70BC%2Fuploads%2Fgit-blob-b3f07f902b1402dadc7abbd8bb62f9c204547390%2Fui-integrations-dropdown.png?alt=media" alt=""><figcaption><p>Integrations Dropdown</p></figcaption></figure>

* Click "Available"

<figure><img src="https://435022081-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FmBVcKuiytGCsIDpL70BC%2Fuploads%2Fgit-blob-434641ec6d4e45051842f86164f485d6bd289424%2Fapp_system_integrations_available.png?alt=media" alt=""><figcaption><p>Click Available</p></figcaption></figure>

* In the Search bar, search Elasticsearch, then click "Configure"

<figure><img src="https://435022081-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FmBVcKuiytGCsIDpL70BC%2Fuploads%2Fgit-blob-1d9380768be4d9986b67f4ec218c0490b7042c35%2Fapp_system_integrations_available_elasticsearch.png?alt=media" alt=""><figcaption><p>The Elasticsearch Tile</p></figcaption></figure>

* Under the Alert Source heading, if your Elasticsearch integration is behind an [On-premise Dropzone Connector](https://docs.dropzone.ai/platform/settings/connector), select your connector from the dropdown
* If you have a Cloud deployment, check the box labeled "Connect with Elastic Cloud ID," then input the Elasticsearch Cloud ID and API Key

<figure><img src="https://435022081-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FmBVcKuiytGCsIDpL70BC%2Fuploads%2Fgit-blob-ed378e72ca691bf33f876f21ad48bd778527f19c%2Fapp_system_integrations_available_elasticsearch_config_1.png?alt=media" alt=""><figcaption><p>The Elasticsearch Alert Cloud ID Configuration</p></figcaption></figure>

* Otherwise, input the Elasticsearch Server, Port, and API Key

<figure><img src="https://435022081-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FmBVcKuiytGCsIDpL70BC%2Fuploads%2Fgit-blob-fa078c1eb956cd428989c91f1b7a215db2bf078e%2Fapp_system_integrations_available_elasticsearch_config_2.png?alt=media" alt=""><figcaption><p>The Elasticsearch Alert Configuration (pt 1)</p></figcaption></figure>

* If you wish to enable namespace-based multitenancy, check the box labeled "Utilize namespace for multi-tenant environment." Otherwise, leave blank. See the bottom of this documentation for further information

<figure><img src="https://435022081-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FmBVcKuiytGCsIDpL70BC%2Fuploads%2Fgit-blob-631aa0e73d53c09a9c5d64816472d8d456d8fea3%2Felasticsearch_namespace_config.png?alt=media" alt=""><figcaption><p>The Elasticsearch Multi-tenant Namespace Configuration</p></figcaption></figure>

* Under the heading Elasticsearch Alert Queries, click "Add item" to add Elasticsearch Alert Queries for Dropzone to investigate

<figure><img src="https://435022081-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FmBVcKuiytGCsIDpL70BC%2Fuploads%2Fgit-blob-1280a1d24611d68a7c85f20add90b9da9a5cd4d2%2Fapp_system_integrations_available_elasticsearch_config_3.png?alt=media" alt=""><figcaption><p>The Elasticsearch Alert Configuration (pt 2)</p></figcaption></figure>

* You may use the Elasticsearch [Kibana alerts](https://www.elastic.co/guide/en/kibana/current/alerting-getting-started.html), or create your own custom index and query string
* To use your own custom index and query string, uncheck the box labeled "Use Kibana Alerts." Input your custom index and query string into the areas labeled "Custom Index" and "Query String", then click "Add Item"

<figure><img src="https://435022081-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FmBVcKuiytGCsIDpL70BC%2Fuploads%2Fgit-blob-37149908d6a9f53e81335a2a18ea0f82bcd23a8c%2Fapp_system_integrations_available_elasticsearch_config_4.png?alt=media" alt=""><figcaption><p>The Elasticsearch Kibana Alert Configuration (pt 3)</p></figcaption></figure>

* To use Elasticsearch Kibana Alerts, check the box labeled "Use Kibana Alerts"
  * In the "Kibana Alert Index" section, input an Index pattern for Kibana security alerts
  * You may choose to allow Dropzone to inject only specific Kibana alert [schema](https://www.elastic.co/guide/en/security/current/alert-schema.html) under the heading "Kibana Alert Status Allowlist". If you do, click "Add Item" to add specific Kibana alert statuses. Otherwise, leave blank

<figure><img src="https://435022081-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FmBVcKuiytGCsIDpL70BC%2Fuploads%2Fgit-blob-54e54f334b1baa69e640180235898d3929cc8676%2Fapp_system_integrations_available_elasticsearch_config_5.png?alt=media" alt=""><figcaption><p>The Elasticsearch Alert Configuration (pt 4)</p></figcaption></figure>

* Under the heading "Kibana Alert Severities", check the box for each severity level you want Dropzone to investigate alerts for

<figure><img src="https://435022081-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FmBVcKuiytGCsIDpL70BC%2Fuploads%2Fgit-blob-4a89129e5f9257f5854355919bea60971712efec%2Fapp_system_integrations_available_elasticsearch_config_6.png?alt=media" alt=""><figcaption><p>The Elasticsearch Alert Configuration (pt 5)</p></figcaption></figure>

* You may choose to allow Dropzone to allow or exclude select Kibana Alert [Rules](https://www.elastic.co/guide/en/kibana/current/alerting-getting-started.html#_rules)
  * If you wish to include an Alert Rule, click "Add item" under the heading "Kibana Alert Rule Allowlist". If you wish to exclude a rule, click "Add Item" under the section labeled "Kibana Alert Rule Exclusion List". Otherwise, leave blank

<figure><img src="https://435022081-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FmBVcKuiytGCsIDpL70BC%2Fuploads%2Fgit-blob-1642027c5e022b76cb1ca0432a2f2183151ed29a%2Fapp_system_integrations_available_elasticsearch_config_7.png?alt=media" alt=""><figcaption><p>The Elasticsearch Alert Configuration (pt 6)</p></figcaption></figure>

* Once you have finished adding your Elasticsearch Alert Queries, click "Add item" at the end of the Kibana Alerts section

<figure><img src="https://435022081-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FmBVcKuiytGCsIDpL70BC%2Fuploads%2Fgit-blob-204df163a1e425d2d7daa479224aeacb598d3b70%2Fapp_system_integrations_available_elasticsearch_config_8.png?alt=media" alt=""><figcaption><p>Click Add Item</p></figcaption></figure>

* Input your desired poll interval and lookback

<figure><img src="https://435022081-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FmBVcKuiytGCsIDpL70BC%2Fuploads%2Fgit-blob-4c7ea3bed79ba239af37678eecadaeb16c8d0b85%2Fapp_system_integrations_available_elasticsearch_config_9.png?alt=media" alt=""><figcaption><p>The Elasticsearch Alert Configuration (pt 7)</p></figcaption></figure>

* Click "Test & Save" to finish

## Multitenancy Configuration Using Namespaces

The Elasticsearch integration supports multitenancy based on the Elastic data stream naming scheme. This approach allows you to leverage Elasticsearch's built-in data organization capabilities while maintaining proper tenant separation within Dropzone.

### Understanding Elastic Namespaces

Elastic data streams follow a structured naming convention: **{type}-{dataset}-{namespace}**

The namespace component is a user-configurable arbitrary grouping that provides flexibility in organizing data. For example, you might have data streams like `logs-nginx.access-production` or `logs-nginx.access-staging`, where `production` and `staging` are different namespaces.

For more details on the Elastic data stream naming scheme, see [An introduction to the Elastic data stream naming scheme](https://www.elastic.co/blog/an-introduction-to-the-elastic-data-stream-naming-scheme).

### Enabling Namespace-Based Multitenancy

When the multitenant configuration is selected and the multitenant map is enabled in Dropzone, you can map Elasticsearch namespaces to tenants within the multitent map. This enables alert investigation to only search for data within a specific namespace related the alert.

To configure namespace-based multitenancy:

1. Define your namespace-to-tenant mappings to specify which Elasticsearch namespaces should be associated with which Dropzone tenants
2. Enable the multitenant configuration option in your Elasticsearch integration settings

<figure><img src="https://435022081-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FmBVcKuiytGCsIDpL70BC%2Fuploads%2Fgit-blob-631aa0e73d53c09a9c5d64816472d8d456d8fea3%2Felasticsearch_namespace_config.png?alt=media" alt=""><figcaption><p>The Elasticsearch Multi-tenant Namespace Configuration</p></figcaption></figure>

3. Ensure your Elasticsearch data streams follow the standard naming convention with appropriate namespace values
4. (Optional) Specify namespaces with alerts that are desired to be investigated otherwise all alerts with a namespace will be ingested.

{% hint style="warning" %}
If this configuration is enabled and an alert is ingested without the namespace specified, the alert will be dropped and will not be investigated.
{% endhint %}

If you have any errors engage your Dropzone AI support representative.
