# Migrator from elasticsearch to opensearch

# Introduction

**Ver**. 131123

CLI utility to migrate an elastic search environment to opensearch

Download link: [link](https://firefly.pandorafms.com/pandorafms/utils/elastic_to_opensearch)

Type: utility

# Compatibility matrix

<table id="bkmrk-sistemas-donde-se-ha"><tbody><tr><td>**Versión de opensearch probada**

</td><td>2.11.0

</td></tr><tr><td>**Tested elastic version**

</td><td>6.4.3

8.10.3

</td></tr><tr><td>**OS systems where tested**

</td><td>CentOS 7

Rocky 8

Ubuntu 22.04

</td></tr><tr><td>**Systems where it should work**

</td><td>Any Linux operating system with a Pandora FMS server installed

</td></tr></tbody></table>

# Pre requisites

In order to run this utility you need to :

- Connectivity to the elastic search environment.
- Connectivity to the opensearch environment.
- To have opensearch authentication credentials (in case you have basic authentication configured).

# Previous configuration

Check that you have access to Elasticsearch.  
From the machine where the migrator will run, run a test with curl to verify connectivity.

```
curl http://<hostname_or_ip>:9200
```

[![image.png](https://pandorafms.com/guides/public/uploads/images/gallery/2023-11/scaled-1680-/fUkimage.png)](https://pandorafms.com/guides/public/uploads/images/gallery/2023-11/fUkimage.png)

Perform the same check, but this time to the opensearch environment, it can be done with the same command and changing the hostname or ip in the case of not having basic authentication.

In case of having authentication you must execute:

```bash
curl -X GET -ku 'user:pass' https://<hostname_or_ip>:9200
```

[![image.png](https://pandorafms.com/guides/public/uploads/images/gallery/2023-11/scaled-1680-/TKWimage.png)](https://pandorafms.com/guides/public/uploads/images/gallery/2023-11/TKWimage.png)

If the check is correct and we have access to both environments, we can now use the viewpoint.

# Parameters and configuration

**Parameters**

<table border="1" id="bkmrk---conf-ruta-al-archi" style="width: 100%; height: 348.36px;"><tbody><tr style="height: 29.7969px;"><td style="width: 48.0734%; height: 29.7969px;">-s SOURCE\_URL, --source\_url SOURCE\_URL</td><td style="width: 51.9019%; height: 29.7969px;">URL of elastic source environment to be migrated</td></tr><tr style="height: 46.5938px;"><td style="width: 48.0734%; height: 46.5938px;">-d DESTINATION\_URL, --destination\_url DESTINATION\_URL</td><td style="width: 51.9019%; height: 46.5938px;">URL of the opensearch target environment</td></tr><tr style="height: 29.7969px;"><td style="width: 48.0734%; height: 29.7969px;">-P INDEX\_PATTERN, --index\_pattern INDEX\_PATTERN</td><td style="width: 51.9019%; height: 29.7969px;">Index pattern to be migrated, by default: pandorafms\*.</td></tr><tr style="height: 29.7969px;"><td style="width: 48.0734%; height: 29.7969px;"> -a, --use\_auth</td><td style="width: 51.9019%; height: 29.7969px;">Flag, use basic authentication</td></tr><tr style="height: 29.7969px;"><td style="width: 48.0734%; height: 29.7969px;">-u USER, --user USER</td><td style="width: 51.9019%; height: 29.7969px;">User for basic authentication, by default: admin</td></tr><tr style="height: 29.7969px;"><td style="width: 48.0734%; height: 29.7969px;"> -p PASSWORD, --password PASSWORD</td><td style="width: 51.9019%; height: 29.7969px;">Basic authentication password, default: admin</td></tr><tr style="height: 29.7969px;"><td style="width: 48.0734%; height: 29.7969px;">-t TIMEOUT, --timeout TIMEOUT</td><td style="width: 51.9019%; height: 29.7969px;">Time to wait between index migrations</td></tr><tr style="height: 46.5938px;"><td style="width: 48.0734%; height: 46.5938px;">-f, --force</td><td style="width: 51.9019%; height: 46.5938px;">Deletes the index at the destination if it exists and forces migration</td></tr><tr style="height: 29.7969px;"><td style="width: 48.0734%; height: 29.7969px;"> -C, --compare</td><td style="width: 51.9019%; height: 29.7969px;">Compares the source and destination indexes and the status of each.</td></tr><tr style="height: 46.5938px;"><td style="width: 48.0734%; height: 46.5938px;">-v, --verbose</td><td style="width: 51.9019%; height: 46.5938px;">Flag, shows step by step all migration actions, for debugging.</td></tr></tbody></table>

[![image.png](https://pandorafms.com/guides/public/uploads/images/gallery/2023-11/scaled-1680-/wboimage.png)](https://pandorafms.com/guides/public/uploads/images/gallery/2023-11/wboimage.png)

# Manual execution

Migration execution:

```
./elastic_to_opensearch -s 192.168.80.44:9200 -d 192.168.50.92:9200 -a \
-u admin -p admin
```

Migration execution filtering by a single index.

```
./elastic_to_opensearch -s 192.168.80.44:9200 -d 192.168.50.92:9200 -a \
-u admin -p admin -f \
-P pandorafms-b8ef1b65799a11ee94810242ac100002-2023.11.02
```

[![image.png](https://pandorafms.com/guides/public/uploads/images/gallery/2023-11/scaled-1680-/MdAimage.png)](https://pandorafms.com/guides/public/uploads/images/gallery/2023-11/MdAimage.png)

Execution check:

```
./elastic_to_opensearch -s 192.168.80.44:9200 -d 192.168.50.92:9200 -a \
-u admin -p admin -C
```

[![image.png](https://pandorafms.com/guides/public/uploads/images/gallery/2023-11/scaled-1680-/Q6fimage.png)](https://pandorafms.com/guides/public/uploads/images/gallery/2023-11/Q6fimage.png)

The check can also be performed by filtering by indexes using a search pattern:

```
./elastic_to_opensearch -s 192.168.80.44:9200 -d 192.168.50.92:9200 \
-a -u admin -p admin -C \
-P "pandorafms-b8ef1b65799a11ee94810242ac100002*"
```

[![image.png](https://pandorafms.com/guides/public/uploads/images/gallery/2023-11/scaled-1680-/Senimage.png)](https://pandorafms.com/guides/public/uploads/images/gallery/2023-11/Senimage.png)

# Recommendations

To perform a migration it is important to take into account the size of the environments to be migrated and the amount of indexes and documents we have stored. Migration is a demanding process that transfers a lot of data at high speed.

In general we indicate certain recommendations for environments with a large amount of data:

- Perform the migration before connecting the pandorafms environment to the new opensearch.
- Perform a staggered migration, year by year, month by month or even index by index, using the pattern filter. Depending on the amount of data stored.
- Right-size the environment.
- For a large amount of data, use an opensearch cluster instead of a single node.