Edit me

Google BigQuery Connector Upgrade Document (v1.0.0 to v2.0.0)

The Google BigQuery connector support following operations of Google BigQuery service:

-   Create Dataset

-   Get Dataset

-   Update Dataset

-   List Dataset

-   Delete Dataset

-   Create Table

-   Get Table

-   Update Table

-   List Table

-   Delete Table

-   Insert All

-   List Table Data

-   Create Job (Copy Job, Extract Job, Load Job, Query Job)

-   Get Job

-   List Job

-   Get Query Result

-   Query

-   Cancel Job

Google BigQuery connector v2.0.0 has changed and simplified the output of the above operations as compared to v1.0.0. This document provides the detail of the operations with the change in its output/payload.

Upgrade (Version 1.0.0 to Version 2.0.0)

The following table provides information about the affected content of the output for different operations.

Note- Information provided in the tables is applicable when output type is application/java as below -

%dw 2.0
output application/java

Affected output in ‘Create Dataset, Get Dataset, Update Dataset and List Dataset’

In v1.0.0In v2.0.0

Affected output in ‘Create Table, Get Table, Update Table and List Table’

In v1.0.0In v2.0.0

Affected output in ‘Create Job, Get Job and List Job’

In v1.0.0In v2.0.0Remark
payload.configuration.tableDefinitions.{EXTERNAL_TABLE_NAME}."type".constantpayload.configuration.tableDefinitions.{EXTERNAL_TABLE_NAME}."type"Applicable only for Query Job.
payload.configuration.tableDefinitions.{EXTERNAL_TABLE_NAME}.schema.fields[0]."type".constantpayload.configuration.tableDefinitions.{EXTERNAL_TABLE_NAME}.schema.fields[0]."type"Applicable only for Query Job.
payload.statistics.schema.fields[0]."type".constantpayload.statistics.schema.fields[0]."type"Applicable only for Query Job.
payload.configuration.clustering.fieldspayload.configuration.clusteringApplicable for Query Job and Load Job.
payload.configuration.schema.fields[0]."type".constantpayload.configuration.schema.fields[0]."type"Applicable only for Load Job.

Affected output in ‘List Table Data, Query Job and Get Query Result’

In v1.0.0In v2.0.0