Skip to main content
Version: 0.0.1

Tarball Install and Upgrade

This topic describes how to install and upgrade Dremio using a Tarball package.

Installing Dremio​

Perform these steps on each node of your Dremio cluster:

  1. Create a dremio group.

    {{< codeheader "Create dremio group" >}}

    sudo groupadd -r dremio

  2. Create a dremio user.

    {{< codeheader "Create dremio user" >}}

    sudo useradd -r -g dremio -d /var/lib/dremio -s /sbin/nologin dremio

  3. Create Dremio directories.

    {{< codeheader "Create dremio directories" >}}

    sudo mkdir /opt/dremio
    sudo mkdir /var/run/dremio && sudo chown dremio:dremio /var/run/dremio
    sudo mkdir /var/log/dremio && sudo chown dremio:dremio /var/log/dremio
    sudo mkdir /var/lib/dremio && sudo chown dremio:dremio /var/lib/dremio

  4. Unpack Dremio into the /opt/dremio directory.

    {{< codeheader "Unpack dremio into /opt/dremio" >}}

    sudo tar xvf dremio-<VERSION>.tar.gz -C /opt/dremio --strip-components=1

  5. Create a symbolic link for the configuration folder.

    {{< codeheader "Create symbolic link for configuration folder" >}}

    sudo ln -s /opt/dremio/conf /etc/dremio

  6. (Optional) Set up your service configuration (SystemD setup).

    a. Copy the unit file for the dremio daemon from <DREMIO_HOME>/share/dremio.service to /etc/systemd/system/dremio.service.

    {{< codeheader "Copy and move unit file for dremio daemon" >}}

    sudo cp <DREMIO_HOME>/share/dremio.service /etc/systemd/system/dremio.service

    b. Restart systemd.

    {{< codeheader "Restart systemd" >}}

    sudo systemctl daemon-reload

    c. (Optional) Enable dremio to start at boot.

    {{< codeheader "Enable dremio to start at boot (optional)" >}}

    sudo systemctl enable dremio

  7. Configure Dremio. See Dremio Configuration for more information.

Sample Service Configuration​

The following snippet shows the configuration in the <DREMIO_HOME>/share/dremio.service file:

{{< codeheader "Sample service configuration" >}}

    #
    # Installation is assumed to be under /opt/dremio
    #
    [Unit]
    Description=Dremio Daemon Server
    Documentation=https://docs.dremio.com
    Wants=network-online.target
    After=network-online.target
    
    [Service]
    User=dremio
    Group=dremio
    RuntimeDirectory=dremio
    ExecStart=/opt/dremio/bin/dremio start-fg
    Restart=on-failure
    # Set maximum number of file descriptors Dremio can open.
    # Dremio requires a large number to work correctly.
    LimitNOFILE=65536
    
    
    [Install]
    WantedBy=multi-user.target

Upgrading Dremio​

This topic provides upgrade instructions for Tarball-based installations.

Before you begin, do the following:

To upgrade your Dremio installation:

  1. Ensure that your existing Dremio cluster is backed up. See the Create a backup section.

  2. Shut down all Dremio provisioned nodes. For example: <DREMIO-HOME>/bin/dremio stop

  3. On each Dremio node in your cluster, move your old Dremio installation to another location. For example:

    {{< codeheader "Move Dremio installation" >}}

    mv /opt/dremio /opt/dremio_bak

  4. Download and install the new Dremio version.

    • For YARN deployments, install on each coordinator node in your Dremio cluster.
    • For non-YARN deployments, install on all nodes in your Dremio cluster.

    Perform the following steps on each Dremio node depending on your deployment (YARN vs non-YARN):

    Note
    Important: Be sure the create the new directories with the same path as the original installation. In particular, the /data directory and associated data files must be in the same path as the original installation.

    1. Create a new Dremio directory. For example:

      {{< codeheader "Create new dremio directory" >}}

      sudo mkdir /opt/dremio

    2. Unpack Dremio into the /opt/dremio directory. For example:

      {{< codeheader "Unpack dremio into /opt/dremio" >}}

      sudo tar xvf dremio-`<version>`.tar.gz -C /opt/dremio --strip-components=1

    3. Create a symbolic link for the configuration folder. For example:

      {{< codeheader "Create symbolic link for configuratin folder" >}}

      sudo ln -s /opt/dremio/conf /etc/dremio

  5. Copy the /conf directory and associated configuration files from your backup-copy location (from Step# 3) to the new installation location. As a result, the default files from the fresh install are overwritten. For example: 

    {{< codeheader "Copy and move /conf directory and files" >}}
    ```shell
    cp /opt/dremio_bak/conf/* /opt/dremio/conf
    ```
    * For YARN deployments, perform this step on each _coordinator_ node in your Dremio cluster.
    * For non-YARN deployments, perform this step on _all_ nodes in your Dremio cluster.

  6. On the coordinator node, copy the /data directory and associated data files from your backup-copy location (from Step# 3) to the new installation location. For example: 

    {{< codeheader "Copy and move /data directory and files on coordinator node" >}}
    ```shell
    cp -r /opt/dremio_bak/data /opt/dremio/data
    ```

    > **<i class="fa fa-exclamation-circle"></i>Note**  
    Important: The new **/data** directory and files must be in the same directory path as the original installation. 
    A different directory path results in the loss of all reflections 
    because they are accessed by their full path.
    >
    > If your original data directory is in a location different from **/opt/dremio/data**, 
    then this step can be skipped.

  7. On the coordinator node, execute the upgrade script as the user running the Dremio Daemon service. For example, as "dremio": 

    {{< codeheader "Execute upgrade script on coordinator node" >}}
    ```shell
    <DREMIO-HOME>/bin/dremio-admin upgrade
    ```

  8. Start up all Dremio provisioned nodes in the Dremio cluster. For example:

    {{< codeheader "Start Dremio provisioned nodes" >}}

    <DREMIO-HOME>/bin/dremio start

  9. Backup your new Dremio cluster. See Create a backup for more information.