Observation API

The Observation API is used to manage observations.

FIND

Required role: Read
Required privilege: Observations.READ on the home domain of the calling user

Executes an Elasticsearch 5.6 query to find observations.

API Endpoint: [POST]: /observations/find

Request

Headers

  • Authorization: The Authorization token returned from /auth/login
  • x-api-key: The API Key to identify the request

Body

  • queryScope

    • thingTypes[]: A list of thing types to include in the search. This is not the same as searching for a thing type in the query as this improves performance by limiting the amount of indices searched.

  • query: The query to run on Elasticsearch specified using Elasticsearch Query DSL. required

    Any query and any aggregations are permitted to use but since Managed IoT Cloud prevents the user from seeing events on domains that the user is not authorized to see the query must satisfy the following requirements:

    • The query must contain a range clause with a timestamp containing at least a gte key.
    • The query cannot contain global aggregations.

Filter context

Consider using a filter context if possible. In filter context, a query clause answers the question “Does this document match this query clause?” The answer is a simple Yes or No — no scores are calculated. Filter context is mostly used for filtering structured data, e.g.

  • Does this timestamp fall into the range 2015 to 2016?
  • Is the status field set to “published”?

Example payload

{
  "queryScope": {
    "thingTypes": ["1"]
  },
  "query": {
    "size": 0,
    "track_scores": false,
    "query": {
      "bool": {
        "filter": [
          {
            "range": {
              "timestamp": {
                "gte": 1460661813164,
                "lte": 1460705013163
              }
            }
          },
          {
            "term": {
              "thingName": "thingName"
            }
          }
        ]
      }
    },
    "aggs": {
      "hist": {
        "date_histogram": {
          "field": "timestamp",
          "interval": "30m",
          "min_doc_count": 1,
          "extended_bounds": {
            "min": 1460661813164,
            "max": 1460705013163
          }
        },
        "aggs": {
          "temperature": {
            "avg": {
              "field": "temperature"
            }
          }
        }
      }
    }
  }
}

Response

The output from the Elasticsearch query.

Errors

Key Params Property Description
INVALID_ARGUMENTS query Returned if the query is invalid or doesn’t meet the requirements specified above.
PROPERTY_REQUIRED query Returned if no query was provided.

CREATE_EXPORT_JOB

Data Export is an additional service priced separately when used.

Required role: ReadWrite
Required privilege: DataExport.CREATE on the home domain of the calling user
Optional privilege: Things.READ on all attached things
Optional privilege: ThingType.READ on the specified thingType

Creates an export job to dump observations into an downloadable compressed file.

API Endpoint: [POST]: /observations/export/jobs

Request

Headers

  • Authorization: The Authorization token returned from /auth/login
  • x-api-key: The API Key to identify the request

Body

  • jobName: A verbose descriptive name of the export job. required
  • thingType: A thingType id. Export all observations for a declared thingType. required if type is things[] are not declared, otherwise forbidden.
  • things[]: A list of thingNames. Export all observations for a list of declared thingNames. required if type is thingType is not declared, otherwise forbidden.
  • type: What type of job it is. Can be either scheduled or instant. required
  • from: The time from when observations are exported. Shall be an ISO8601 timestamp which is in UTC. required if type is instant
  • to: The time to when observations are exported. Shall be an ISO8601 timestamp which is in UTC. required if type is instant
  • schedule: A cron expression specifying when the job should be run. required if type is scheduled
    • Format should follow AWS cron expressions
    • Minute and Hour must be specified with values and cannot be wildcarded.
    • Example for running every day at 02:01 is cron(1 2 * * ? *).
  • timePeriod: Specifies the period for which data should be exported. required if type is scheduled
    • Has the format xd for x days and xh for x hours.
    • Is relative to the configured schedule.
  • resources[]: What resources should be extracted. At least 1 resource must be defined. required
  • notifyByEmail: Boolean stating if an email should be sent when a job execution has finished.
  • recipients: A comma separated list with email addresses that should receive an email when a job execution has finished.

Example payloads

The following payload would require the optional privilege Things.READ on the domain where Things 00000001 and 00000009 exist.

{
  "jobName": "My simple test job",
  "things": ["00000001", "00000009"],
  "type": "instant",
  "from": "2015-02-04 22:45:00",
  "to": "2016-02-04 22:45:00",
  "resources": ["tcxn.connection_status"]
}

The following payload would require the optional privilege ThingTypes.READ on the domain where Thing Type 1 exists.

{
  "jobName": "My simple test job",
  "thingType": "1",
  "type": "instant",
  "from": "2015-02-04 22:45:00",
  "to": "2016-02-04 22:45:00",
  "resources": ["tcxn.connection_status"]
}

The following payload would require the optional privilege ThingTypes.READ on the domain where Thing Type 1 exists.

{
  "jobName": "My simple scheduled test job",
  "thingType": "1",
  "type": "scheduled",
  "schedule": "cron(1 2 * * ? *)",
  "timePeriod": "5d",
  "resources": ["tcxn.connection_status"]
}

Response

  • id: The generated id of the export job (uuid v4).
  • input payload: The payload declared in the request.
  • createdAt: The date and time of when the job was created.
  • updatedAt: The date and time of when the job was last updated.
  • thingTypeLabel: Label of the thing type.

Example Response

{
  "id": "d24ff535-3ea3-4f0b-9923-876bf82e8366",
  "jobName": "My simple test job",
  "resources": ["tcxn.connection_status"],
  "type": "instant",
  "createdAt": "2019-05-01T10:00:00.000Z",
  "updatedAt": "2019-05-13T11:00:00.000Z",
  "from": "2015-02-04 22:45:00",
  "to": "2016-02-04 22:45:00",
  "thingType": "1",
  "thingTypeLabel": "Thing type label"
}

Errors

Key Params Property Description
NOT_AUTHORIZED Returned if the user is not authorized for the operation.
INVALID_ARGUMENTS Returned if a parameter does not fulfil requirements.
THING_TYPE_NOT_FOUND Returned if the thingType does not exist or is not accessible
THING_NOT_FOUND Returned if one thing does not exist or is not accessible

GET_EXPORT_JOB

Required role: Read
Required privilege: DataExport.READ on the home domain of the calling user

Gets an export job.

API Endpoint: [GET]: /observations/export/jobs/{id}

Request

Headers

  • Authorization: The Authorization token returned from /auth/login
  • x-api-key: The API Key to identify the request

Query Parameters

  • id: The id of the job (uuid v4) required

Response

The entire job object.

Example Response

{
  "id": "24f5449c-db92-4905-893f-49ef9b065a4b",
  "jobName": "My simple scheduled test job",
  "resources": ["tcxn.connection_status"],
  "type": "scheduled",
  "enabled": true,
  "createdAt": "2019-05-01T10:00:00.000Z",
  "updatedAt": "2019-05-13T11:00:00.000Z",
  "schedule": "cron(1 2 * * ? *)",
  "timePeriod": "5d",
  "thingType": "1"
}

Errors

Key Params Property Description
NOT_AUTHORIZED Returned if the user is not authorized for the operation.
INVALID_ARGUMENTS Returned if a parameter does not fulfil requirements.
EXPORT_JOB_NOT_FOUND Returned if the job does not exist or is not accessible.

DELETE_EXPORT_JOB

Required role: ReadWrite
Required privilege: DataExport.DELETE on the home domain of the calling user

Deletes an export job and its executions.

API Endpoint: [DELETE]: /observations/export/jobs/{id}

Request

Headers

  • Authorization: The Authorization token returned from /auth/login
  • x-api-key: The API Key to identify the request

Query Parameters

  • id: The id of the job (uuid v4) required

Errors

Key Params Property Description
NOT_AUTHORIZED Returned if the user is not authorized for the operation.
INVALID_ARGUMENTS Returned if a parameter does not fulfil requirements.
EXPORT_JOB_NOT_FOUND Returned if the job does not exist or is not accessible.

UPDATE_EXPORT_JOB

Required role: ReadWrite
Required privilege: DataExport.UPDATE on the home domain of the calling user

Updates a scheduled export job. Only the attributes you want to update needs to be provided.

API Endpoint: [PATCH]: /observations/export/jobs/{id}

Request

Headers

  • Authorization: The Authorization token returned from /auth/login
  • x-api-key: The API Key to identify the request

Query Parameters

  • id: The id of the job (uuid v4) required

Body

  • jobName: A verbose descriptive name of the export job.
  • enabled: A boolean to enable/disable a scheduled job.
  • from: An ISO8601 datetime string defining from when.
  • to: An ISO8601 datetime string defining until when.
  • schedule: A cron expression specifying when the job should be run.
    • Format should follow AWS cron expressions
    • Minute and Hour must be specified with values. Wildcards are not supported.
    • Example for running every day at 02:01 is cron(1 2 * * ? *).
  • timePeriod: Specifies the period for which data should be exported.
    • Has the format xd for x days and xh for x hours.
    • Is relative to the configured schedule.
  • resources[]: What resources should be extracted. At least 1 resource must be defined.
  • notifyByEmail: Boolean stating if an email should be sent when a job execution has finished.
  • recipients: A comma separated list with email addresses that should receive an email when a job execution has finished.

Example payloads

{
  "enabled": false
}

{
  "jobName": "New name of the job"
}

Response

The entire job object.

Example Response

{
  "id": "24f5449c-db92-4905-893f-49ef9b065a4b",
  "jobName": "My simple scheduled test job",
  "resources": ["tcxn.connection_status"],
  "type": "scheduled",
  "enabled": true,
  "createdAt": "2019-05-01T10:00:00.000Z",
  "updatedAt": "2019-05-13T11:00:00.000Z",
  "schedule": "cron(1 2 * * ? *)",
  "timePeriod": "5d",
  "thingType": "1"
}

Errors

Key Params Property Description
NOT_AUTHORIZED Returned if the user is not authorized for the operation.
INVALID_ARGUMENTS Returned if a parameter does not fulfil requirements.
EXPORT_JOB_NOT_FOUND Returned if the job does not exist or is not accessible.

LIST_EXPORT_JOBS

Required role: Read
Required privilege: DataExport.READ on the home domain of the calling user

List all accessible export jobs ordered by creation time. A user can only list its own jobs.

API Endpoint: [GET]: /observations/export/jobs

Request

Headers

  • Authorization: The Authorization token returned from /auth/login
  • x-api-key: The API Key to identify the request

Query Parameters

  • size: The wanted page size. (max 100) optional
  • marker: The marker to get the next page. Received in previous response. optional
  • order: The wanted order of the result. optional

Response

An array of all export jobs.

  • exportJobs[]:
    • id: The id of the job (uuid v4).
    • jobName: The descriptive name of the job.
    • thingType: The declared thingType. (if not things are declared)
    • things[]: The list of declared thingNames. (if not thingType is declared)
    • from: The datetime object from when.
    • to: The datetime object until when.
    • resources[]: The resources to be extracted.
    • type: The type of the job.
  • pageInfo:
    • itemCount: The number of items in the result.
    • size: The page size used.
    • marker: The marker used in the request.
    • hasNext: Boolean stating if there are more pages.
    • nextMarker: Returned if there are more pages. Contains a marker used to get the next page.
  • sortInfo:
    • order: The order the result are sorted by.

Errors

Key Params Property Description
NOT_AUTHORIZED Returned if the user is not authorized for the operation.

LIST_EXPORT_JOB_EXECUTIONS_BY_JOB

Required role: Read
Required privilege: DataExport.READ on the home domain of the calling user

List all executions for an export job ordered by creation time. A user can only list its own jobs and executions.

API Endpoint: [GET]: /observations/export/jobs/{id}/executions

Request

Headers

  • Authorization: The Authorization token returned from /auth/login
  • x-api-key: The API Key to identify the request

Query Parameters

  • id: The id of the job (uuid v4) required
  • size: The wanted page size. (max 100) optional
  • marker: The marker to get the next page. Received in previous response. optional
  • order: The wanted order of the result. optional

Response

An array of all executions.

  • exportJobExecutions[]:
    • id: The id of the execution (uuid v4).
    • jobId: The id of the export job (uuid v4)
    • status: The current state of the execution. One of QUEUED, RUNNING, SUCCEEDED, FAILED, CANCELLED.
    • submissionDateTime: The date and time when the execution was created.
    • completionDateTime: The date and time when the execution was finished.
    • dataScannedInBytes: The amount of data scanned during the execution.
  • pageInfo:
    • itemCount: The number of items in the result.
    • size: The page size used.
    • marker: The marker used in the request.
    • hasNext: Boolean stating if there are more pages.
    • nextMarker: Returned if there are more pages. Contains a marker used to get the next page.
  • sortInfo:
    • order: The order the result are sorted by.

Example Response

{
  "exportJobExecutions": [
    {
      "id": "2f519851-4f76-43dc-b899-26817b8ec51a",
      "jobId": "840da513-3ebc-44b6-bcff-144109894de2",
      "submissionDateTime": "2019-03-13T09:40:32.177Z",
      "completionDateTime": "",
      "dataScannedInBytes": 0,
      "status": "QUEUED"
    }
  ],
  "pageInfo": {
    "hasNext": false,
    "itemCount": 1,
    "size": 10
  },
  "sortInfo": {
    "order": "desc"
  }
}

Errors

Key Params Property Description
NOT_AUTHORIZED Returned if the user is not authorized for the operation.
EXPORT_JOB_NOT_FOUND Returned if the job does not exist or is not accessible.

LIST_EXPORT_JOB_EXECUTIONS

Required role: Read
Required privilege: DataExport.READ on the home domain of the calling user

List all executions ordered by creation time. A user can only list its own job executions.

API Endpoint: [GET]: /observations/export/executions

Request

Headers

  • Authorization: The Authorization token returned from /auth/login
  • x-api-key: The API Key to identify the request

Query Parameters

  • size: The wanted page size. (max 100) optional
  • marker: The marker to get the next page. Received in previous response. optional
  • order: The wanted order of the result. optional

Response

An array of all executions.

  • exportJobExecutions[]:
    • id: The id of the execution (uuid v4).
    • executionFileName: The file name for the execution (jobName-fromDate-toDate.csv).
    • jobId: The id of the export job (uuid v4)
    • status: The current state of the execution. One of QUEUED, RUNNING, SUCCEEDED, FAILED, CANCELLED.
    • submissionDateTime: The date and time when the execution was created.
    • completionDateTime: The date and time when the execution was finished.
    • dataScannedInBytes: The amount of data scanned during the execution.
  • pageInfo:
    • itemCount: The number of items in the result.
    • size: The page size used.
    • marker: The marker used in the request.
    • hasNext: Boolean stating if there are more pages.
    • nextMarker: Returned if there are more pages. Contains a marker used to get the next page.
  • sortInfo:
    • order: The order the result are sorted by.

Example Response

{
  "exportJobExecutions": [
    {
      "id": "2f519851-4f76-43dc-b899-26817b8ec51a",
      "jobId": "3f396c72-dd8f-4fa6-b9ed-4e6c9d364de0",
      "submissionDateTime": "2019-03-13T09:40:32.177Z",
      "completionDateTime": "2019-03-13T09:40:32.969Z",
      "dataScannedInBytes": 0,
      "status": "SUCCEEDED"
    },
    {
      "id": "83c77467-84cf-47f0-b216-7d7b819f025d",
      "jobId": "30e83e10-780e-47cd-b65d-e18cfc6d35bf",
      "submissionDateTime": "2019-03-13T12:35:23.011Z",
      "completionDateTime": "",
      "dataScannedInBytes": 0,
      "status": "QUEUED"
    }
  ],
  "pageInfo": {
    "hasNext": false,
    "itemCount": 2,
    "size": 10
  },
  "sortInfo": {
    "order": "desc"
  }
}

Errors

Key Params Property Description
NOT_AUTHORIZED Returned if the user is not authorized for the operation.

GET_EXPORT_JOB_EXECUTION

Required role: Read
Required privilege: DataExport.READ on the home domain of the calling user

Get a single execution for an export job. A user can only get its own job executions.

API Endpoint: [GET]: /observations/export/executions/{id}

Request

Headers

  • Authorization: The Authorization token returned from /auth/login
  • x-api-key: The API Key to identify the request

Query Parameters

  • id: The id of the execution (uuid v4) required

Response

The execution.

  • dataScannedInBytes: The amount of data scanned during the execution.
  • status: The current state of the execution. One of QUEUED, RUNNING, SUCCEEDED, FAILED, CANCELLED.
  • engineExecutionTimeInMillis: Execution time in millisecond.
  • jobId: The id of the export job (uuid v4)
  • executionFileName: The file name for the execution (jobName-fromDate-toDate.csv).
  • id: The id of the execution (uuid v4).
  • submissionDateTime: The date and time when the execution was created.
  • completionDateTime: The date and time when the execution was finished.

Example Response

{
  "id": "2f519851-4f76-43dc-b899-26817b8ec51a",
  "jobId": "840da513-3ebc-44b6-bcff-144109894de2",
  "submissionDateTime": "2019-03-13T09:40:32.177Z",
  "completionDateTime": "",
  "dataScannedInBytes": 0,
  "status": "QUEUED"
}

Errors

Key Params Property Description
NOT_AUTHORIZED Returned if the user is not authorized for the operation.
EXPORT_EXECUTION_NOT_FOUND Returned if the execution does not exist.

DOWNLOAD_EXPORT_JOB_EXECUTION

Required role: Read
Required privilege: DataExport.READ on the home domain of the calling user

Get a signed URL to the result of the execution.

API Endpoint: [GET]: /observations/export/executions/{id}/download

Request

Headers

  • Authorization: The Authorization token returned from /auth/login
  • x-api-key: The API Key to identify the request

Query Parameters

  • id: The id of the execution (uuid v4) required

Response

The execution.

  • id: The id of the execution (uuid v4).
  • S3SignedURL: The url to download the export.

Example Response

{
  "id": "2f519851-4f76-43dc-b899-26817b8ec51a",
  "S3SignedURL": "https://....."
}

Errors

Key Params Property Description
NOT_AUTHORIZED Returned if the user is not authorized for the operation.
EXPORT_EXECUTION_NOT_FOUND Returned if the execution does not exist.

CANCEL_EXPORT_JOB_EXECUTION

Required role: ReadWrite
Required privilege: DataExport.UPDATE on the home domain of the calling user

Cancel and interrupt an execution for an export job. A user can only cancel its own job executions.

API Endpoint: [POST]: /observations/export/executions/{id}

Request

Headers

  • Authorization: The Authorization token returned from /auth/login
  • x-api-key: The API Key to identify the request

Query Parameters

  • id: The id of the execution (uuid v4) required

Response

The execution.

  • id: The id of the execution (uuid v4).
  • jobId: The id of the export job (uuid v4)
  • status: The current state of the execution. One of QUEUED, RUNNING, SUCCEEDED, FAILED, CANCELLED.
  • submissionDateTime: The date and time when the execution was created.
  • completionDateTime: The date and time when the execution was finished.
  • dataScannedInBytes: The amount of data scanned during the execution.

Example Response

{
    "id": "2f519851-4f76-43dc-b899-26817b8ec51a",
    "jobId": "840da513-3ebc-44b6-bcff-144109894de2",
    "submissionDateTime": "2019-03-13T09:40:32.177Z",
    "completionDateTime": "2019-03-13T09:54:09.244Z",
    "dataScannedInBytes": 0,
    "status": "CANCELLED"
}

Errors

Key Params Property Description
NOT_AUTHORIZED Returned if the user is not authorized for the operation.
INVALID_ACTION Returned if job execution has already been cancelled.
EXPORT_EXECUTION_NOT_FOUND Returned if the execution does not exist.