Edit me

Configurations


Config

Default configuration

Parameters

Name Type Description Default Value Required

Name

String

The name for this configuration. Connectors reference the configuration with this name.

x 

Connection

The connection types that can be provided to this configuration.

x 

Expiration Policy

Configures the minimum amount of time that a dynamic configuration instance can remain idle before the runtime considers it eligible for expiration. This does not mean that the platform will expire the instance at the exact moment that it becomes eligible. The runtime will actually purge the instances when it sees it fit.

 

Connection Types

Connection

This class (as it's name implies) provides connection instances and the funcionality to disconnect and validate those connections.

All connection related parameters (values required in order to create a connection) must be declared in the connection providers.

This particular example is a PoolingConnectionProvider which declares that connections resolved by this provider will be pooled and reused. There are other implementations like CachedConnectionProvider which lazily creates and caches connections or simply ConnectionProvider if you want a new connection each time something requires one.

</div>
Parameters
Name Type Description Default Value Required

Project ID

String

Project Identity

x 

Service Account Key (JSON) File

String

The service authentication credentials

 

Reconnection

When the application is deployed, a connectivity test is performed on all connectors. If set to true, deployment will fail if the test doesn't pass after exhausting the associated reconnection strategy

 

Pooling Profile

Characteristics of the connection pool

 

</div> </div> </div> </div> </div>

Operations

Cancel Job

<bigquery:cancel-job>

CancelJob operation sends a job cancel request. If the location of the job is not "US" or "EU", the jobId must specify the job location. It logs 'Job Not Found', if specified Job does not exist. It won't log/return anything on successful execution.

Parameters

Name Type Description Default Value Required

Configuration

String

The name of the configuration to use.

x 

Job Name

String

Google BigQuery job name

x 

Job Location

String

Google BigQuery job location

 

Reconnection Strategy

A retry strategy in case of connectivity errors

 

For Configurations.

Throws

  • BIGQUERY:EXECUTION  

  • BIGQUERY:CONNECTIVITY  

  • BIGQUERY:JOB_FAILURE_EXCEPTION  

  • BIGQUERY:JOB_NOT_FOUND  

  • MULE:CONNECTIVITY  

  • BIGQUERY:RETRY_EXHAUSTED  

Create Dataset

<bigquery:create-dataset>

CreateDataset operation creates a new empty dataset. A dataset is a grouping mechanism that holds zero or more tables.

Parameters

Name Type Description Default Value Required

Configuration

String

The name of the configuration to use.

x 

Dataset Name

String

Dataset identity. Dataset ID must be alphanumeric (plus underscores, dashes, and colons) and must be at most 1024 characters long.

x 

Table Life Time

Number

Default lifetime of all tables in the dataset, in milliseconds. Table expiration should not be less than 3600000ms.

 

Description

String

User-friendly description for the dataset

 

Friendly Name

String

User-friendly name for the dataset

 

Labels

Object

List of labels associated with labels

 

Location

String

Geographic location where the dataset should reside

 

ACL

Array of One of:

Dataset's access control configuration

 

Target Variable

String

The name of a variable on which the operation's output will be placed

 

Target Value

String

An expression that will be evaluated against the operation's output and the outcome of that expression will be stored in the target variable

#[payload]

 

Reconnection Strategy

A retry strategy in case of connectivity errors

 

Output

Type

For Configurations.

Throws

  • BIGQUERY:EXECUTION  

  • BIGQUERY:DATASET_NOT_EMPTY  

  • BIGQUERY:CONNECTIVITY  

  • BIGQUERY:DATASET_NOT_FOUND  

  • MULE:CONNECTIVITY  

  • BIGQUERY:RETRY_EXHAUSTED  

Create Job

<bigquery:create-job>

CreateJob operation create and start an asynchronous job.

Parameters

Name Type Description Default Value Required

Configuration

String

The name of the configuration to use.

x 

Job Config

Google BigQuery job configuration

x 

Job Info

Google BigQuery create job options configuration

x 

Target Variable

String

The name of a variable on which the operation's output will be placed

 

Target Value

String

An expression that will be evaluated against the operation's output and the outcome of that expression will be stored in the target variable

#[payload]

 

Reconnection Strategy

A retry strategy in case of connectivity errors

 

Output

Type

Job

For Configurations.

Throws

  • BIGQUERY:EXECUTION  

  • BIGQUERY:CONNECTIVITY  

  • BIGQUERY:JOB_FAILURE_EXCEPTION  

  • BIGQUERY:JOB_NOT_FOUND  

  • MULE:CONNECTIVITY  

  • BIGQUERY:RETRY_EXHAUSTED  

Create Table

<bigquery:create-table>

CreateTable operation creates a new table in existing dataset.

Parameters

Name Type Description Default Value Required

Configuration

String

The name of the configuration to use.

x 

Table Info

Table information configuration to create a table

x 

Table Fields

Table Schema details , Field Type , Field Name , Field Description , Field Mode , Sub-fields.

 

Table Option

Create table information options

 

Target Variable

String

The name of a variable on which the operation's output will be placed

 

Target Value

String

An expression that will be evaluated against the operation's output and the outcome of that expression will be stored in the target variable

#[payload]

 

Reconnection Strategy

A retry strategy in case of connectivity errors

 

Output

Type

For Configurations.

Throws

  • BIGQUERY:TABLE_NOT_FOUND  

  • BIGQUERY:EXECUTION  

  • BIGQUERY:CONNECTIVITY  

  • MULE:CONNECTIVITY  

  • BIGQUERY:RETRY_EXHAUSTED  

Delete Dataset

<bigquery:delete-dataset>

DeleteDataset operation deletes the dataset specified by datasetId. It logs 'Dataset Not Empty' if deleting non-empty dataset is disabled and Dataset is still in use. It logs 'Dataset Not Found', if specified dataset does not exist. It won't log/return anything on successful execution.

Parameters

Name Type Description Default Value Required

Configuration

String

The name of the configuration to use.

x 

Dataset Id

String

Google BigQuery dataset Id

x 

Delete Non Empty Dataset

Boolean

Delete a dataset even if non-empty

true

 

Reconnection Strategy

A retry strategy in case of connectivity errors

 

For Configurations.

Throws

  • BIGQUERY:EXECUTION  

  • BIGQUERY:DATASET_NOT_EMPTY  

  • BIGQUERY:CONNECTIVITY  

  • BIGQUERY:DATASET_NOT_FOUND  

  • MULE:CONNECTIVITY  

  • BIGQUERY:RETRY_EXHAUSTED  

Delete Table

<bigquery:delete-table>

DeleteTable operation deletes the specified table from the dataset. This operation logs 'Table Not found', if specified table does not exist. And it won't log/return anything on successful execution.

Parameters

Name Type Description Default Value Required

Configuration

String

The name of the configuration to use.

x 

Table

Table Configuration with tableId and datasetId

x 

Reconnection Strategy

A retry strategy in case of connectivity errors

 

For Configurations.

Throws

  • BIGQUERY:TABLE_NOT_FOUND  

  • BIGQUERY:EXECUTION  

  • BIGQUERY:CONNECTIVITY  

  • MULE:CONNECTIVITY  

  • BIGQUERY:RETRY_EXHAUSTED  

Get Dataset

<bigquery:get-dataset>

GetDataset operation returns the dataset specified by datasetId.

Parameters

Name Type Description Default Value Required

Configuration

String

The name of the configuration to use.

x 

Dataset Id

String

Google BigQuery dataset identity

x 

Field

Get dataset options configuration

 

Target Variable

String

The name of a variable on which the operation's output will be placed

 

Target Value

String

An expression that will be evaluated against the operation's output and the outcome of that expression will be stored in the target variable

#[payload]

 

Reconnection Strategy

A retry strategy in case of connectivity errors

 

Output

Type

For Configurations.

Throws

  • BIGQUERY:EXECUTION  

  • BIGQUERY:DATASET_NOT_EMPTY  

  • BIGQUERY:CONNECTIVITY  

  • BIGQUERY:DATASET_NOT_FOUND  

  • MULE:CONNECTIVITY  

  • BIGQUERY:RETRY_EXHAUSTED  

Get Job

<bigquery:get-job>

GetJob operation returns information about a specific job.

Parameters

Name Type Description Default Value Required

Configuration

String

The name of the configuration to use.

x 

Job Name

String

Google BigQuery job name

x 

Job Location

String

Google BigQuery job location

 

Job Fields

Job information options configurations

 

Target Variable

String

The name of a variable on which the operation's output will be placed

 

Target Value

String

An expression that will be evaluated against the operation's output and the outcome of that expression will be stored in the target variable

#[payload]

 

Reconnection Strategy

A retry strategy in case of connectivity errors

 

Output

Type

Job

For Configurations.

Throws

  • BIGQUERY:EXECUTION  

  • BIGQUERY:CONNECTIVITY  

  • BIGQUERY:JOB_FAILURE_EXCEPTION  

  • BIGQUERY:JOB_NOT_FOUND  

  • MULE:CONNECTIVITY  

  • BIGQUERY:RETRY_EXHAUSTED  

Get Query Result

<bigquery:get-query-result>

GetQueryResult operation returns the results of a query job.

Parameters

Name Type Description Default Value Required

Configuration

String

The name of the configuration to use.

x 

Job Name

String

Google BigQuery job name

x 

Job Location

String

Google BigQuery job location

 

Job Result Options

Job query result options configurations

 

Target Variable

String

The name of a variable on which the operation's output will be placed

 

Target Value

String

An expression that will be evaluated against the operation's output and the outcome of that expression will be stored in the target variable

#[payload]

 

Reconnection Strategy

A retry strategy in case of connectivity errors

 

Output

Type

For Configurations.

Throws

  • BIGQUERY:EXECUTION  

  • BIGQUERY:CONNECTIVITY  

  • BIGQUERY:JOB_FAILURE_EXCEPTION  

  • BIGQUERY:JOB_NOT_FOUND  

  • MULE:CONNECTIVITY  

  • BIGQUERY:RETRY_EXHAUSTED  

Get Table

<bigquery:get-table>

GetTable operation returns the specified table from dataset.

Parameters

Name Type Description Default Value Required

Configuration

String

The name of the configuration to use.

x 

Dataset Id

String

Google BigQuery dataset identity.

x 

Table Id

String

Google BigQuery table identity.

x 

Table Option

Get table information options

 

Target Variable

String

The name of a variable on which the operation's output will be placed

 

Target Value

String

An expression that will be evaluated against the operation's output and the outcome of that expression will be stored in the target variable

#[payload]

 

Reconnection Strategy

A retry strategy in case of connectivity errors

 

Output

Type

For Configurations.

Throws

  • BIGQUERY:TABLE_NOT_FOUND  

  • BIGQUERY:EXECUTION  

  • BIGQUERY:CONNECTIVITY  

  • MULE:CONNECTIVITY  

  • BIGQUERY:RETRY_EXHAUSTED  

Insert All

<bigquery:insert-all>

InsertAll operation streams data into BigQuery one record at a time without running the load job.

Parameters

Name Type Description Default Value Required

Configuration

String

The name of the configuration to use.

x 

Rows Data

Array of Object

Row data to be inserted

x 

Ignore Unknown Values

Boolean

Whether to accept rows that contain values that do not match the schema.

true

 

Skip Invalid Rows

Boolean

Whether to insert all valid rows of a request, even if invalid rows exist.

false

 

Template Suffix

String

The destination table is treated as a base template.

 

Table Id

String

Table identity.

x 

Dataset Id

String

Dataset identity.

x 

Target Variable

String

The name of a variable on which the operation's output will be placed

 

Target Value

String

An expression that will be evaluated against the operation's output and the outcome of that expression will be stored in the target variable

#[payload]

 

Reconnection Strategy

A retry strategy in case of connectivity errors

 

Output

Type

For Configurations.

Throws

  • BIGQUERY:EXECUTION  

  • BIGQUERY:CONNECTIVITY  

  • MULE:CONNECTIVITY  

  • BIGQUERY:RETRY_EXHAUSTED  

List Dataset

<bigquery:list-dataset>

ListDataset operation returns partial information on each dataset: DatasetId, FriendlyName and GeneratedId. To get complete information use "Get Dataset" operation.

Parameters

Name Type Description Default Value Required

Configuration

String

The name of the configuration to use.

x 

List Dataset Option

List dataset options configuration

 

Target Variable

String

The name of a variable on which the operation's output will be placed

 

Target Value

String

An expression that will be evaluated against the operation's output and the outcome of that expression will be stored in the target variable

#[payload]

 

Reconnection Strategy

A retry strategy in case of connectivity errors

 

Output

Type

Array of Dataset

Attributes Type

For Configurations.

Throws

  • BIGQUERY:EXECUTION  

  • BIGQUERY:DATASET_NOT_EMPTY  

  • BIGQUERY:CONNECTIVITY  

  • BIGQUERY:DATASET_NOT_FOUND  

  • MULE:CONNECTIVITY  

  • BIGQUERY:RETRY_EXHAUSTED  

List Job

<bigquery:list-job>

ListJob operation lists all jobs that you started in the specified project.

Parameters

Name Type Description Default Value Required

Configuration

String

The name of the configuration to use.

x 

Job Info

Google BigQuery job options configuration

 

Page Token

String

Specifies the token from which to start listing jobs

 

Page Size

Number

Specifies the maximum number of jobs returned per page

 

State

Enumeration, one of:

  • DONE

  • PENDING

  • RUNNING

State of the job (DONE, PENDING, RUNNING)

 

Target Variable

String

The name of a variable on which the operation's output will be placed

 

Target Value

String

An expression that will be evaluated against the operation's output and the outcome of that expression will be stored in the target variable

#[payload]

 

Reconnection Strategy

A retry strategy in case of connectivity errors

 

Output

Type

Array of Job

Attributes Type

For Configurations.

Throws

  • BIGQUERY:EXECUTION  

  • BIGQUERY:CONNECTIVITY  

  • BIGQUERY:JOB_FAILURE_EXCEPTION  

  • BIGQUERY:JOB_NOT_FOUND  

  • MULE:CONNECTIVITY  

  • BIGQUERY:RETRY_EXHAUSTED  

List Table

<bigquery:list-table>

ListTable operation list the tables in the specified dataset. This operation returns partial information on each table: TableId, FriendlyName, Generated and type, which is part of Definition.

Parameters

Name Type Description Default Value Required

Configuration

String

The name of the configuration to use.

x 

Dataset Id

String

Google BigQuery dataset identity.

x 

List Table Options

List table options configurations

 

Target Variable

String

The name of a variable on which the operation's output will be placed

 

Target Value

String

An expression that will be evaluated against the operation's output and the outcome of that expression will be stored in the target variable

#[payload]

 

Reconnection Strategy

A retry strategy in case of connectivity errors

 

Output

Type

Attributes Type

For Configurations.

Throws

  • BIGQUERY:TABLE_NOT_FOUND  

  • BIGQUERY:EXECUTION  

  • BIGQUERY:CONNECTIVITY  

  • MULE:CONNECTIVITY  

  • BIGQUERY:RETRY_EXHAUSTED  

List Table Data

<bigquery:list-table-data>

ListTableData operation list the content of a table in rows.

Parameters

Name Type Description Default Value Required

Configuration

String

The name of the configuration to use.

x 

Dataset Id

String

Google BigQuery dataset identity

x 

Table Id

String

Google BigQuery table identity

x 

Page Size

Number

Specifies the maximum number of rows returned per page

 

Page Token

String

Page token of the request.

 

Start Index

Number

Start row index of the table.

 

Target Variable

String

The name of a variable on which the operation's output will be placed

 

Target Value

String

An expression that will be evaluated against the operation's output and the outcome of that expression will be stored in the target variable

#[payload]

 

Reconnection Strategy

A retry strategy in case of connectivity errors

 

Output

Type

For Configurations.

Throws

  • BIGQUERY:EXECUTION  

  • BIGQUERY:CONNECTIVITY  

  • MULE:CONNECTIVITY  

  • BIGQUERY:RETRY_EXHAUSTED  

Query

<bigquery:query>

Runs the query associated with the request, using the given JobId or using an internally-generated random JobId.

Parameters

Name Type Description Default Value Required

Configuration

String

The name of the configuration to use.

x 

Query Configuration

Google BigQuery query configuration

x 

Job Information

Google BigQuery job options configuration

x 

Job Name

String

Google BigQuery job name

 

Job Location

String

Google BigQuery job location

 

Target Variable

String

The name of a variable on which the operation's output will be placed

 

Target Value

String

An expression that will be evaluated against the operation's output and the outcome of that expression will be stored in the target variable

#[payload]

 

Reconnection Strategy

A retry strategy in case of connectivity errors

 

Output

Type

For Configurations.

Throws

  • BIGQUERY:EXECUTION  

  • BIGQUERY:CONNECTIVITY  

  • BIGQUERY:JOB_FAILURE_EXCEPTION  

  • BIGQUERY:JOB_NOT_FOUND  

  • MULE:CONNECTIVITY  

  • BIGQUERY:RETRY_EXHAUSTED  

Update Dataset

<bigquery:update-dataset>

UpdateDataset operation updates information in an existing dataset.

Parameters

Name Type Description Default Value Required

Configuration

String

The name of the configuration to use.

x 

Field

Update dataset options configuration

 

Dataset Name

String

Dataset identity. Dataset ID must be alphanumeric (plus underscores, dashes, and colons) and must be at most 1024 characters long.

x 

Table Life Time

Number

Default lifetime of all tables in the dataset, in milliseconds. Table expiration should not be less than 3600000ms.

 

Description

String

User-friendly description for the dataset

 

Friendly Name

String

User-friendly name for the dataset

 

Labels

Object

List of labels associated with labels

 

Location

String

Geographic location where the dataset should reside

 

ACL

Array of One of:

Dataset's access control configuration

 

Target Variable

String

The name of a variable on which the operation's output will be placed

 

Target Value

String

An expression that will be evaluated against the operation's output and the outcome of that expression will be stored in the target variable

#[payload]

 

Reconnection Strategy

A retry strategy in case of connectivity errors

 

Output

Type

For Configurations.

Throws

  • BIGQUERY:EXECUTION  

  • BIGQUERY:DATASET_NOT_EMPTY  

  • BIGQUERY:CONNECTIVITY  

  • BIGQUERY:DATASET_NOT_FOUND  

  • MULE:CONNECTIVITY  

  • BIGQUERY:RETRY_EXHAUSTED  

Update Table

<bigquery:update-table>

UpdateTable operation updates information in an existing table.

Parameters

Name Type Description Default Value Required

Configuration

String

The name of the configuration to use.

x 

Table Info

Table information configuration to update table

x 

Table Fields

Table Schema details , Field Type , Field Name , Field Description , Field Mode , Sub-fields.

x 

Table Option

Table options configurations

x 

Target Variable

String

The name of a variable on which the operation's output will be placed

 

Target Value

String

An expression that will be evaluated against the operation's output and the outcome of that expression will be stored in the target variable

#[payload]

 

Reconnection Strategy

A retry strategy in case of connectivity errors

 

Output

Type

For Configurations.

Throws

  • BIGQUERY:TABLE_NOT_FOUND  

  • BIGQUERY:EXECUTION  

  • BIGQUERY:CONNECTIVITY  

  • MULE:CONNECTIVITY  

  • BIGQUERY:RETRY_EXHAUSTED  

Types

Reconnection

Field Type Description Default Value Required

Fails Deployment

Boolean

When the application is deployed, a connectivity test is performed on all connectors. If set to true, deployment will fail if the test doesn’t pass after exhausting the associated reconnection strategy

Reconnection Strategy

The reconnection strategy to use

Reconnect

Field Type Description Default Value Required

Frequency

Number

How often (in ms) to reconnect

Count

Number

How many reconnection attempts to make

Reconnect Forever

Field Type Description Default Value Required

Frequency

Number

How often (in ms) to reconnect

Pooling Profile

Field Type Description Default Value Required

Max Active

Number

Controls the maximum number of Mule components that can be borrowed from a session at one time. When set to a negative value, there is no limit to the number of components that may be active at one time. When maxActive is exceeded, the pool is said to be exhausted.

Max Idle

Number

Controls the maximum number of Mule components that can sit idle in the pool at any time. When set to a negative value, there is no limit to the number of Mule components that may be idle at one time.

Max Wait

Number

Specifies the number of milliseconds to wait for a pooled component to become available when the pool is exhausted and the exhaustedAction is set to WHEN_EXHAUSTED_WAIT.

Min Eviction Millis

Number

Determines the minimum amount of time an object may sit idle in the pool before it is eligible for eviction. When non-positive, no objects will be evicted from the pool due to idle time alone.

Eviction Check Interval Millis

Number

Specifies the number of milliseconds between runs of the object evictor. When non-positive, no object evictor is executed.

Exhausted Action

Enumeration, one of:

  • WHEN_EXHAUSTED_GROW

  • WHEN_EXHAUSTED_WAIT

  • WHEN_EXHAUSTED_FAIL

Specifies the behavior of the Mule component pool when the pool is exhausted. Possible values are: "WHEN_EXHAUSTED_FAIL", which will throw a NoSuchElementException, "WHEN_EXHAUSTED_WAIT", which will block by invoking Object.wait(long) until a new or idle object is available, or WHEN_EXHAUSTED_GROW, which will create a new Mule instance and return it, essentially making maxActive meaningless. If a positive maxWait value is supplied, it will block for at most that many milliseconds, after which a NoSuchElementException will be thrown. If maxThreadWait is a negative value, it will block indefinitely.

Initialisation Policy

Enumeration, one of:

  • INITIALISE_NONE

  • INITIALISE_ONE

  • INITIALISE_ALL

Determines how components in a pool should be initialized. The possible values are: INITIALISE_NONE (will not load any components into the pool on startup), INITIALISE_ONE (will load one initial component into the pool on startup), or INITIALISE_ALL (will load all components in the pool on startup)

Disabled

Boolean

Whether pooling should be disabled

Expiration Policy

Field Type Description Default Value Required

Max Idle Time

Number

A scalar time value for the maximum amount of time a dynamic configuration instance should be allowed to be idle before it’s considered eligible for expiration

Time Unit

Enumeration, one of:

  • NANOSECONDS

  • MICROSECONDS

  • MILLISECONDS

  • SECONDS

  • MINUTES

  • HOURS

  • DAYS

A time unit that qualifies the maxIdleTime attribute

Dataset

Field Type Description Default Value Required

Acl

Array of Acl

Dataset's access control configuration.

Creation Time

Number

The time when this dataset was created, in milliseconds since the epoch.

Dataset Id

The dataset identity.

Default Table Lifetime

Number

The default lifetime of all tables in the dataset, in milliseconds.

Description

String

A user-friendly description for the dataset.

Etag

String

The hash of the dataset resource.

Friendly Name

String

A user-friendly name for the dataset.

Generated Id

String

The service-generated id for the dataset.

Labels

Object

A map for labels applied to the dataset.

Last Modified

Number

The time when this dataset or any of its tables was last modified, in milliseconds since the epoch.

Location

String

The geographic location where the dataset should reside.

Self Link

String

URL that can be used to access the resource again.

Acl

Field Type Description Default Value Required

Entity

The entity for this ACL.

Role

String

The role specified by this ACL.

Entity

Field Type Description Default Value Required

Type

String

Types of BigQuery entities.

Dataset Id

Field Type Description Default Value Required

Dataset

String

Dataset Identity

Project

String

Project Name

Job

Field Type Description Default Value Required

Configuration

The job's configuration

Etag

String

The hash of the job resource.

Generated Id

String

The service-generated id for the job.

Job Id

The job identity.

Self Link

String

URL that can be used to access the resource again.

Statistics

Information about the job, including starting time and ending time of the job.

Status

The status of this job.

User Email

String

The email address of the user who ran the job.

Job Configuration

Field Type Description Default Value Required

Type

String

Type of a BigQuery Job.

Job Id

Field Type Description Default Value Required

Job

String

The job's id.

Location

String

The job's location.

Project

String

Job's project id.

Job Statistics

Field Type Description Default Value Required

Creation Time

Number

The creation time of the job in milliseconds since epoch.

End Time

Number

The end time of the job in milliseconds since epoch.

Start Time

Number

The start time of the job in milliseconds since epoch.

Job Status

Field Type Description Default Value Required

Error

The final error result of the job.

Execution Errors

Array of Big Query Error

All errors encountered during the running of the job.

State

String

The state of the job.

BigQuery Error

Field Type Description Default Value Required

Location

String

Location where the error occurred, if present.

Message

String

A human-readable description of the error.

Reason

String

Short error code that summarizes the error.

Job Info Options

Field Type Description Default Value Required

Configuration

Boolean

Job Configuration

true

E Tag

Boolean

Job eTag

false

Id

Boolean

Job Identity

true

Job Reference

Boolean

A job reference is a fully qualified identifier for refering to a job.

true

Self Link

Boolean

A URL that can be used to access the resource again. You can use this URL in Get or Update requests to the resource.

false

Statistics

Boolean

Information about the job, including starting time and ending time of the job.

true

Status

Boolean

The status of this job.

true

User Mail

Boolean

The email address of the user who ran the job.

true

Table Information

Field Type Description Default Value Required

Creation Time

Number

The time when this table was created, in milliseconds since the epoch.

Description

String

Table Description

E Tag

String

A hash of the page of results.

Encryption Configuration

Custom encryption configuration (e.g., Cloud KMS keys).

Expiration Time

Number

The time when this table expires, in milliseconds since the epoch. If not present, the table will persist indefinitely. Expired tables will be deleted and their storage reclaimed.

Friendly Name

String

The user-friendly name for the table.

Generated Id

String

The service-generated id for the table.

Labels

Object

A map for labels applied to the table.

Last Modified Time

Number

Table last modified time

Num Bytes

Number

The size of the table in bytes

Num Long Term Bytes

Number

The number of bytes considered "long-term storage" for reduced billing purposes.

Num Rows

Number

Number of rows in a table

Self Link

String

An URL that can be used to access the resource again.

Table Id

The table identity.

Table Definition

Table Definition

Encryption Configuration

Field Type Description Default Value Required

Kms Key Name

String

KMS key name for encryption configuration

Table Id

Field Type Description Default Value Required

Dataset

String

Dataset Identity

Project Id

String

Project's user-defined id.

Table

String

Table Identity

Table Definition

Field Type Description Default Value Required

Schema

The table's schema.

Type

String

The table's type.

Schema

Field Type Description Default Value Required

Fields

Array of Field

The fields in the current table schema.

Field

Field Type Description Default Value Required

Description

String

The field description.

Mode

String

The field mode.

Name

String

The field name.

Sub Fields

Array of Field

The list of sub-fields if field type is a RECORD.

Type

String

The field type.

Table Info Configuration

Field Type Description Default Value Required

Table

Details of table(Table Name, Dataset Name)

x

Description

String

Table Description

Kms Key Name

String

Creates a EncryptionConfiguration object for KMS key name

Expiration Time

Number

The time when this table expires, in milliseconds since the epoch. If not present, the table will persist indefinitely. Expired tables will be deleted and their storage reclaimed.

Friendly Name

String

The user-friendly name for the table.

Labels

Object

The labels associated with the table. Can be used to organize and group your tables.

Table Configuration

Field Type Description Default Value Required

Table

String

Table identity.

x

Dataset

String

Dataset identity.

x

Table Field Configuration

Field Type Description Default Value Required

Field Name

String

The table field name

x

Field Type

String

The type of the table field Supported types are : BOOLEAN, BYTES, DATE, DATETIME, FLOAT, INTEGER, RECORD, STRING, TIME, TIMESTAMP

x

Field Description

String

The Table field description

Field Mode

String

Mode for a Table field. NULLABLE fields can be set to null, REQUIRED fields must be provided. REPEATED fields can contain more than one value.

Field Sub Fields

Sub field of the field name. Type of the parent field must be RECORD to add subfield.

Table Info Options

Field Type Description Default Value Required

Creation Time

Boolean

The time when this table was created, in milliseconds since the epoch.

true

Description

Boolean

Table Description

true

E Tag

Boolean

A hash of the page of results.

true

Expiration Time

Boolean

The time when this table expires, in milliseconds since the epoch. If not present, the table will persist indefinitely. Expired tables will be deleted and their storage reclaimed.

true

Friendly Name

Boolean

The user-friendly name for the table.

true

Id

Boolean

An opaque ID of the table.

true

Last Modified Time

Boolean

Table last modified time

true

Location

Boolean

The geographic location where the table should reside. Possible values include EU and US. The default value is US.

true

Num Bytes

Boolean

The size of the table in bytes

true

Num Rows

Boolean

Number of rows in a table

true

Schema

Boolean

Table Schema

true

Self Link

Boolean

A URL that can be used to access the resource again. You can use this URL in Get or Update requests to the resource.

true

Streaming Buffer

Boolean

true

Time Partitioning

Boolean

Configures time-based partitioning for the table.

true

Type

Boolean

The type of table.

true

View

Boolean

The view definition of table

true

Dataset Info Option

Field Type Description Default Value Required

Access

Boolean

It defines dataset access for one or more entities

true

Creation Time

Boolean

Creation time of Dataset

true

Dataset Reference

Boolean

A reference that identifies the dataset.

true

Description

Boolean

Dataset description

true

E Tag

Boolean

Dataset eTag

true

Friendly Name

Boolean

Dataset friendly name

true

Labels

Boolean

Dataset Labels

true

Last Modified Time

Boolean

Dataset last modified time

true

Location

Boolean

The geographic location where the dataset should reside. Possible values include EU and US. The default value is US.

true

Self Link

Boolean

A URL that can be used to access the resource again. You can use this URL in Get or Update requests to the resource.

true

Id

Boolean

Dataset Identity

true

Table Result

Field Type Description Default Value Required

Next Page

The next page of result.

Next Page Token

String

The token of next page of result.

Schema

It represents the schema for a Google BigQuery Table or data source.

Total Rows

Number

Total number of rows in the complete result set, which can be more than the number of rows in the first page of results returned by getValues().

Values

Array of Array of Field Value

Represent rows in returned query result (table row).

Field Value

Field Type Description Default Value Required

Attribute

Enumeration, one of:

  • PRIMITIVE

  • REPEATED

  • RECORD

The attribute of this Field Value.

Boolean Value

Boolean

This field's value as a Boolean.

Bytes Value

Binary

This field's value as a byte array.

Double Value

Number

This field's value as a Double.

Long Value

Number

This field's value as a long.

Null

Boolean

true if this field's value is null, false otherwise.

Numeric Value

Number

This field's value as a BigDecimal.

Record Value

Array of Field Value

This field's value as a list of FieldValue.

Repeated Value

Array of Field Value

This field's value as a list of FieldValue.

String Value

String

This field's value as a String.

Timestamp Value

Number

This field's value as a long, representing a timestamp in microseconds since epoch (UNIX time).

Value

Any

This field's value as an Object.

Job Query Results Option

Field Type Description Default Value Required

Max Wait Time

Number

How long to wait for the query to complete, in milliseconds, before the request times out and returns. Note that this is only a timeout for the request, not the query.

Page Size

Number

Number of records to be displayed on a page.

Page Token

String

Specify the page token from which to start listing datasets

Start Index

Number

Zero-based index of the starting row.

Insert All Response

Field Type Description Default Value Required

Insert Errors

Object

All insertion errors as a map whose keys are indexes of rows that failed to insert.

Page Attribute

Field Type Description Default Value Required

Has Next Page

Boolean

Used for paging results. When this field is true, it indicates additional results are available.

Page Token

String

Specify the page token from which to start listing datasets

List Dataset Option

Field Type Description Default Value Required

All

Boolean

List all datasets, even hidden ones.

false

Page Size

Number

Specify the maximum number of datasets returned per page

x

Page Token

String

Specify the page token from which to start listing datasets

List Table Options

Field Type Description Default Value Required

Page Size

Number

Specify the maximum number of datasets returned per page

Page Token

String

The page token from which to start listing datasets

Query Job

Field Type Description Default Value Required

Query String

String

A query string, following the BigQuery query syntax, of the query to execute.

x

Allow Large Results

Boolean

Sets whether the job is enabled to create arbitrarily large results. If true the query is allowed to create large results at a slight cost in performance. If true Destination Table must be provided.

false

Default Dataset

String

Specifies the default datasetId and projectId to assume for any unqualified table names in the query. If not set, all table names in the query string must be qualified in the format 'datasetId.tableId'.

Dry Run

Boolean

If set to true, BigQuery doesn’t run the job. Instead, if the query is valid, BigQuery returns statistics about the job such as how many bytes would be processed. If the query is invalid, an error returns. The default value is false.

false

Flatten Results

Boolean

Sets whether nested and repeated fields should be flattened. If set to false AllowLargeResults must be true. By default results are flattened.

true

Maximum Billing Tier

Number

Limits the billing tier for this job. Queries that have resource usage beyond this tier will fail (without incurring a charge). If unspecified, this will be set to your project default.

Named Parameters

the query parameters to a set of named query parameters to use in the query. Named parameters are denoted using an @ prefix, e.g. @myParam for a parameter named "myParam". SELECT * FROM <table> WHERE <column_name>=@myParam in query

Positional Query Parameter Values

Sets the query parameters to a list of positional query parameters to use in the query. Positional parameters are denoted in the query with a question mark (?). SELECT * FROM <table> WHERE <column_name>=?

Priority

Enumeration, one of:

  • BATCH

  • INTERACTIVE

a priority for the query. If not specified the priority is assumed to be INTERACTIVE. Query is executed as soon as possible and count towards the concurrent rate limit and the daily rate limit.

External Table Definitions

BigQuery’s external tables are tables whose data reside outside of BigQuery But can be queried as normal BigQuery tables.

Use Legacy Sql

Boolean

UseLegacySql must be set to false to use query parameters

false

Use Query Cache

Boolean

Whether to look for the result in the query cache. The query cache is a best-effort cache that will be flushed whenever tables in the query are modified. Moreover, the query cache is only available when Destination Table is not set.

true

User Defined Functions

For legacy SQL, specify user defined function here. For Standard SQL, specify user defined function in Query string.

Create Disposition

Enumeration, one of:

  • CREATE_IF_NEEDED

  • CREATE_NEVER

Whether the job is allowed to create new tables CREATE_IF_NEEDED :- Configures the job to create the table if it does not exist. CREATE_NEVER :- Configures the job to fail with a not-found error if the table does not exist. Create Disposition

Destination Table

Sets the table where to put query results. If not provided a new table is created. This value is required if Allow Large Results is set to true.

Schema Update Option

Enumeration, one of:

  • ALLOW_FIELD_ADDITION

  • ALLOW_FIELD_RELAXATION

Specifies options relating to allowing the schema of the destination table To be updated as a side effect of the load or query job. Schema Update Option

Write Disposition

Enumeration, one of:

  • WRITE_APPEND

  • WRITE_EMPTY

  • WRITE_TRUNCATE

Action that should occur if the destination table already exists. WRITE_APPEND :- Configures the job to append data to the table if it already exists. WRITE_EMPTY :- Configures the job to fail with a duplicate error if the table already exists. WRITE_TRUNCATE :- Configures the job to overwrite the table data if table already exists. Write Disposition

Kms Key Name

String

Creates a EncryptionConfiguration object for KMS key name

Named Parameter Configuration

Field Type Description Default Value Required

Key

String

A parameter given to a query.

x

Named Parameter

Contains the type of the parameter and value of the parameter.

x

Query Parameter Value Configuration

Field Type Description Default Value Required

Array Type

Boolean

The type of the array's elements, if this is an array.

false

Type

Enumeration, one of:

  • BOOL

  • BYTES

  • DATE

  • DATETIME

  • FLOAT64

  • INT64

  • NUMERIC

  • STRING

  • TIME

  • TIMESTAMP

The type of the parameter.

x

Values

Array of String

The values of the parameter.

x

External Table Definition Configuration

Field Type Description Default Value Required

Table Name

String

Name of the table

x

Source Uris

Array of String

The fully-qualified URIs that point to your data in Google Cloud Storage. Each URI can contain one '*' wildcard character that must come after the bucket's name. Size limits related to load jobs apply to external data sources, plus an additional limit of 10 GB maximum size across all URIs.

x

Table Schema

The Schema for the external data

x

Format

Enumeration, one of:

  • AVRO

  • CSV

  • DATASTORE_BACKUP

  • GOOGLE_SHEETS

  • JSON

  • ORC

  • PARQUET

The source format of the external data

x

Table Schema

Field Type Description Default Value Required

Table Fields

Table field contains Datatype , Field Mode , Field Name , Field Type , Field Description to configure Table Schema

User Defined Function Configuration

Field Type Description Default Value Required

Type

Enumeration, one of:

  • FROM_URI

  • INLINE

Type of user-defined function. User defined functions can be provided inline as code blobs ({@link #INLINE}) or as a Google Cloud Storage URI ({@link #FROM_URI}).

x

Content

String

If type is INLINE, it returns a code blob. If type is FROM_URI, it returns a Google Cloud Storage URI (e.g. gs://bucket/path).

x

User Acl

Field Type Description Default Value Required

Email

String

User’s email address.

x

User Role

Enumeration, one of:

  • OWNER

  • READER

  • WRITER

Role specified to user

x

Domain Acl

Field Type Description Default Value Required

Domain

String

The domain name.

x

Role

Enumeration, one of:

  • OWNER

  • READER

  • WRITER

Role specified to domain

x

View Acl

Field Type Description Default Value Required

Project

String

Project IDs must contain 6-63 lowercase letters, digits, or dashes. IDs must start with a letter and may not end with a dash.

x

Dataset

String

Dataset identity. Dataset ID must be alphanumeric (plus underscores, dashes, and colons) and must be at most 1024 characters long.

x

Table

String

Table IDs must be alphanumeric (plus underscores) and must be at most 1024 characters long.

x

Role

Enumeration, one of:

  • OWNER

  • READER

  • WRITER

Role specified to view

x

Copy Job

Field Type Description Default Value Required

Source Tables

Contains Table and Dataset name. It will copy the data from one or more source tables to a destination table.

x

Destination Table

Data will get copied to a destination table.

x

Create Disposition

Enumeration, one of:

  • CREATE_IF_NEEDED

  • CREATE_NEVER

Whether the job is allowed to create new tables CREATE_IF_NEEDED :- Configures the job to create the table if it does not exist. CREATE_NEVER :- Configures the job to fail with a not-found error if the table does not exist. Create Disposition

Write Disposition

Enumeration, one of:

  • WRITE_APPEND

  • WRITE_EMPTY

  • WRITE_TRUNCATE

Action that should occur if the destination table already exists. WRITE_APPEND :- Configures the job to append data to the table if it already exists. WRITE_EMPTY :- Configures the job to fail with a duplicate error if the table already exists. WRITE_TRUNCATE :- Configures the job to overwrite the table data if table already exists. Write Disposition

Kms Key Name

String

KMS key name for encryption configuration

Extract Job

Field Type Description Default Value Required

Destination Uris

Array of String

The list of fully-qualified Google Cloud Storage URIs where the extracted table should be written.

x

Source Table

The table to export.

x

Compression

String

the compression value of exported files.

Field Delimiter

String

The delimiter used between fields in the exported data.

Format

String

The exported files format.

Print Header

Boolean

Whether an header row is printed with the result.

false

Load Job

Field Type Description Default Value Required

Source Uris

Array of String

Fully-qualified URIs that point to source data in Google Cloud Storage

x

Destination Table

Destination table to load the data into

x

Autodetect

Boolean

Automatic inference of the options and schema for CSV and JSON sources

true

Format Option

Enumeration, one of:

  • AVRO

  • CSV

  • DATASTORE_BACKUP

  • GOOGLE_SHEETS

  • JSON

  • ORC

  • PARQUET

Source format and parsing options of the external data

Ignore Unknown Values

Boolean

BigQuery should allow extra values that are not represented in the table schema

false

Max Bad Records

Number

Maximum number of bad records that BigQuery can ignore when running the job

Null Marker

String

String that represents a null value in a CSV file

Destination Table Schema

Schema for the destination table

Create Disposition

Enumeration, one of:

  • CREATE_IF_NEEDED

  • CREATE_NEVER

Whether the job is allowed to create new tables Create Disposition

Kms Key Name

String

Creates a EncryptionConfiguration object for KMS key name

Write Disposition

Enumeration, one of:

  • WRITE_APPEND

  • WRITE_EMPTY

  • WRITE_TRUNCATE

Action that should occur if the destination table already exists Write Disposition

Allow Fixed Relaxtion

Boolean

Allow relaxing a required field in the original schema to nullable when schema of the destination table to be updated as a side effect of the load job

true

Allow Field Addition

Boolean

Allow adding a nullable field to the schema when schema of the destination table to be updated as a side effect of the load job

true

</div>