Skip to main content

Schedules Preview

Dremio Arctic enables you to schedule optimization jobs to help you manage the accumulation of the data files that occurs through DML operations. This API allows you to add, modify, retrieve, list, and delete schedules.

Schedules Object
{
  "type": "OPTIMIZE",
  "catalogId": "1a234bc5-6789-46d7-b9cb-d236e195b34a",
  "id": "123ab456-cdef-78ab-c901-23de4f5ab6c7",
  "schedule": "@daily",
  "username": "dremio_user@company.com",
  "config": {
    "tableId": "sample-nyc-taxi-data",
    "reference": "main",
    "maxFileSize": "460 MB",
    "minFileSize": "192 MB",
    "minFiles": 5,
    "targetFileSize": "256 MB"
  }
}

Schedule Attributes​

{{< api-section file="data/api/arctic-schedules.json" data="schedules-attributes" >}}

config Object Attributes​

{{< api-section file="data/api/arctic-schedules.json" data="config-object" >}}

Listing All Schedules​

List all the job schedules available for the specified Arctic catalog.

Method and URL
GET /v0/arctic/catalogs/{catalogId}/schedules

Parameters​

{{< api-section file="data/api/arctic-schedules.json" data="listingAllSchedules" >}}

filter Object Attributes​

{{< api-section file="data/api/arctic-schedules.json" data="filter-object" >}}


Example Request
curl -X GET 'https://api.dremio.cloud/v0/arctic/catalogs/1a234bc5-6789-46d7-b9cb-d236e195b34a/schedules' \
-H 'Authorization: Bearer <personal access token>' \
-H 'Content-Type: application/json'
Example Response
[
  {
    "data": [
    {
      "type": "OPTIMIZE",
      "id": "123ab456-cdef-78ab-c901-23de4f5ab6c7",
      "schedule": "@weekly",
      "username": "dremio_user@company.com",
      "config": {
        "maxFileSize": "460 MB",
        "minFileSize": "192 MB",
        "minFiles": 5,
        "reference": "etl",
        "tableId": "sample-NYC-taxi-trips",
        "targetFileSize": "256 MB"
      }
    },
    {
      "type": "OPTIMIZE",
      "id": "abc123d4-ab12-345cd-1234-abc123def456",
      "schedule": "@monthly",      
      "username": "dremio_user@company.com",
      "config": {
        "maxFileSize": "920 MB",
        "minFileSize": "384 MB",
        "minFiles": 5,
        "reference": "dev",
        "tableId": "sample-SF-taxi-trips",
        "targetFileSize": "512 MB"
      }
    }
   ]
  }
]

Responses​

{{< api-responses file="data/api/arctic-schedules.json" data="listingAllSchedules-Responses" >}}

Creating a Schedule​

Creates a job schedule for the specified Arctic catalog.

Method and URL
POST /v0/arctic/catalogs/{catalogId}/schedules

Parameters​

{{< api-section file="data/api/arctic-schedules.json" data="addingSchedule" >}}

config Object Attributes​

{{< api-section file="data/api/arctic-schedules.json" data="config-object" >}}


Example Request to Create a Weekly Schedule
curl -X POST 'https://api.dremio.cloud/v0/arctic/catalogs/1a234bc5-6789-46d7-b9cb-d236e195b34a/schedules \
-H 'Authorization: Bearer <personal access token>' \
-H 'Content-Type: application/json' \
-d '{
    "type": "OPTIMIZE",
    "schedule": "@weekly",
    "config": {
       "reference": "etl",
       "tableID": "sample-NYC-taxi-trips",
       "maxFileSize": "460 MB",
       "minFileSize": "192 MB",
       "minFiles": 5,
       "targetFileSize": "70 MB"
      }
    }'
Example Response
{
  "type": "OPTIMIZE",
  "id": "123ab456-cdef-78ab-c901-23de4f5ab6c7",
  "schedule": "@weekly",
  "username": "dremio_user@company.com",
  "config": {
     "maxFileSize": "460 MB",
     "minFileSize": "192 MB",
     "minFiles": 5,
     "targetFileSize": "70 MB"
  }
}

Responses​

{{< api-responses file="data/api/arctic-schedules.json" data="addingSchedule-Responses" >}}

Retrieving a Schedule​

Retrieve the specified schedule.

Method and URL
GET /v0/arctic/catalogs/{catalogId}/schedules/{scheduleId}

Parameters​

{{< api-section file="data/api/arctic-schedules.json" data="retrievingSchedule" >}}

Example Request
curl -X GET 'https://api.dremio.cloud/v0/catalogs/1a234bc5-6789-46d7-b9cb-d236e195b34a/schedules/9876a54b-681b-49fe-afff-0d753ed5ad85' \
-H 'Authorization: Bearer <personal access token>' \
-H 'Content-Type: application/json'
Example Response
{
  "type": "OPTIMIZE",
  "id": "673cd876-abcd-1234-b478-91ca5e9cc8f6",
  "schedule": "@daily",
  "username": "analyst@company.com",
  "config": {
    "reference": "dev",
    "tableId": "sales_2022",
    "maxFileSize": "2048 MB",
    "minFileSize": "768 MB",
    "minFiles": 10,
    "targetFileSize": "1024 MB"
    },
}

Responses​

{{< api-responses file="data/api/arctic-schedules.json" data="retrievingSchedule-Responses" >}}

Modifying a Schedule​

Modify an Arctic job schedule.

Method and URL
PUT /v0/arctic/catalogs/{catalogId}/schedules/{scheduleId}

Parameters​

{{< api-section file="data/api/arctic-schedules.json" data="modifyingSchedule" >}}

config Object Attributes​

{{< api-section file="data/api/arctic-schedules.json" data="config-object" >}}


Example Request to Modify a Weekly Schedule
curl -X GET 'https://api.dremio.cloud/v0/catalogs/1a234bc5-6789-46d7-b9cb-d236e195b34a/schedules/9876a54b-681b-49fe-afff-0d753ed5ad85' \
-H 'Authorization: Bearer <personal access token>' \
-H 'Content-Type: application/json' \
-d '{
    "type": "OPTIMIZE",
    "schedule": "@weekly",
    "config": {
       "reference": "compare",
       "tableID": "customer-sales",
       "maxFileSize": "460 MB",
       "minFileSize": "192 MB",
       "minFiles": 3,
       "targetFileSize": "70 MB"
      }
    }'
Example Response
{
  "type": "OPTIMIZE",
  "id": "9876a54b-681b-49fe-afff-0d753ed5ad85",
  "schedule": "@weekly",
  "username": "marketing-admin@company.com",
  "config": {
     "reference": "compare",
     "tableID": "customer-sales",
     "maxFileSize": "460 MB",
     "minFileSize": "192 MB",
     "minFiles": 3,
     "targetFileSize": "70 MB"
  },
}

Responses​

{{< api-responses file="data/api/arctic-schedules.json" data="modifyingSchedule-Responses" >}}

Deleting a Schedule​

Delete an Arctic job schedule.

Method and URL
DELETE /v0/arctic/catalogs/{catalogId}/schedules/{scheduleId}

Parameters​

{{< api-section file="data/api/arctic-schedules.json" data="deletingSchedule" >}}

Example Request
curl -X DELETE 'https://api.dremio.cloud/v0/catalogs/1a234bc5-6789-46d7-b9cb-d236e195b34a/schedules/9876a54b-681b-49fe-afff-0d753ed5ad85' \
-H 'Authorization: Bearer <personal access token>' \
-H 'Content-Type: application/json'
Example Response
{
  "code": 200,
  "id": "e04bd884-fa37-4d10-bb4e-153443b8234c",
  "message": "Successfully found and deleted the schedule"
}

Responses​

{{< api-responses file="data/api/arctic-schedules.json" data="deletingSchedule-Responses" >}}