Google BigQuery Connector Upgrade Document (v1.0.0 to v2.0.0)
The Google BigQuery connector support following operations of Google BigQuery service:
- Create Dataset
- Get Dataset
- Update Dataset
- List Dataset
- Delete Dataset
- Create Table
- Get Table
- Update Table
- List Table
- Delete Table
- Insert All
- List Table Data
- Create Job (Copy Job, Extract Job, Load Job, Query Job)
- Get Job
- List Job
- Get Query Result
- Query
- Cancel Job
Google BigQuery connector v2.0.0 has changed and simplified the output of the above operations as compared to v1.0.0. This document provides the detail of the operations with the change in its output/payload.
Upgrade (Version 1.0.0 to Version 2.0.0)
The following table provides information about the affected content of the output for different operations.
Note- Information provided in the tables is applicable when output type is application/java as below -
%dw 2.0
output application/java
---
Affected output in ‘Create Dataset, Get Dataset, Update Dataset and List Dataset’
| In v1.0.0 | In v2.0.0 |
|---|---|
| payload.acl[0].role.constant | payload.acl[0].role |
Affected output in ‘Create Table, Get Table, Update Table and List Table’
| In v1.0.0 | In v2.0.0 |
|---|---|
| payload.tableDefinition."type".constant | payload.tableDefinition."type" |
| payload.tableDefinition.schema.fields[0]."type".constant | payload.tableDefinition.schema.fields[0]."type" |
Affected output in ‘Create Job, Get Job and List Job’
| In v1.0.0 | In v2.0.0 | Remark |
|---|---|---|
| payload.status.state.constant | payload.status.state | |
| payload.configuration.tableDefinitions.{EXTERNAL_TABLE_NAME}."type".constant | payload.configuration.tableDefinitions.{EXTERNAL_TABLE_NAME}."type" | Applicable only for Query Job. |
| payload.configuration.tableDefinitions.{EXTERNAL_TABLE_NAME}.schema.fields[0]."type".constant | payload.configuration.tableDefinitions.{EXTERNAL_TABLE_NAME}.schema.fields[0]."type" | Applicable only for Query Job. |
| payload.statistics.schema.fields[0]."type".constant | payload.statistics.schema.fields[0]."type" | Applicable only for Query Job. |
| payload.configuration.clustering.fields | payload.configuration.clustering | Applicable for Query Job and Load Job. |
| payload.configuration.schema.fields[0]."type".constant | payload.configuration.schema.fields[0]."type" | Applicable only for Load Job. |
Affected output in ‘List Table Data, Query Job and Get Query Result’
| In v1.0.0 | In v2.0.0 |
|---|---|
| ayload.schema.fields[0]."type".constant | payload.schema.fields[0]."type" |