Model Context Protocol (MCP) finally gives AI models a way to access the business data needed to make them really useful at work. CData MCP Servers have the depth and performance to make sure AI has access to all of the answers.
Try them now for free →
Import BigQuery Data Using Azure Data Factory
Use CData Connect Cloud to connect to BigQuery Data from Azure Data Factory and import live BigQuery data.
Microsoft Azure Data Factory (ADF)) is a completely managed, serverless data integration service. When combined with CData Connect Cloud, ADF enables immediate cloud-to-cloud access to BigQuery data within data flows. This article outlines the process of connecting to BigQuery through Connect Cloud and accessing BigQuery data within ADF.
CData Connect Cloud offers a cloud-to-cloud interface tailored for BigQuery, granting you the ability to access live data from BigQuery data within Azure Data Factory without the need for data replication to a natively supported database. Equipped with optimized data processing capabilities by default, CData Connect Cloud seamlessly channels all supported SQL operations, including filters and JOINs, directly to BigQuery. This harnesses server-side processing to expedite the retrieval of the desired BigQuery data.
About BigQuery Data Integration
CData simplifies access and integration of live Google BigQuery data. Our customers leverage CData connectivity to:
- Simplify access to BigQuery with broad out-of-the-box support for authentication schemes, including OAuth, OAuth JWT, and GCP Instance.
- Enhance data workflows with Bi-directional data access between BigQuery and other applications.
- Perform key BigQuery actions like starting, retrieving, and canceling jobs; deleting tables; or insert job loads through SQL stored procedures.
Most CData customers are using Google BigQuery as their data warehouse and so use CData solutions to migrate business data from separate sources into BigQuery for comprehensive analytics. Other customers use our connectivity to analyze and report on their Google BigQuery data, with many customers using both solutions.
For more details on how CData enhances your Google BigQuery experience, check out our blog post: https://www.cdata.com/blog/what-is-bigquery
Getting Started
Configure BigQuery Connectivity for ADF
Connectivity to BigQuery from Azure Data Factory is made possible through CData Connect Cloud. To work with BigQuery data from Azure Data Factory, we start by creating and configuring a BigQuery connection.
CData Connect Cloud uses a straightforward, point-and-click interface to connect to data sources.
- Log into Connect Cloud, click Connections and click Add Connection
- Select "BigQuery" from the Add Connection panel
-
BigQuery uses OAuth to authenticate. Click "Sign in" to authenticate with BigQuery.
![]( "Authenticating with OAuth (Salesforce is shown).")
-
Navigate to the Permissions tab in the Add BigQuery Connection page and update the User-based permissions.
![]( "Updating permissions")
Add a Personal Access Token
If you are connecting from a service, application, platform, or framework that does not support OAuth authentication, you can create a Personal Access Token (PAT) to use for authentication. Best practices would dictate that you create a separate PAT for each service, to maintain granularity of access.
- Click on your username at the top right of the Connect Cloud app and click User Profile.
- On the User Profile page, scroll down to the Personal Access Tokens section and click Create PAT.
- Give your PAT a name and click Create.
- The personal access token is only visible at creation, so be sure to copy it and store it securely for future use.
With the connection configured, you are ready to connect to BigQuery data from Azure Data Factory.
Access Live BigQuery Data in Azure Data Factory
To establish a connection from Azure Data Factory to the CData Connect Cloud Virtual SQL Server API, follow these steps.
- Login to Azure Data Factory.
- If you have not yet created a Data Factory, Click New -> Dataset.
- In the search bar, enter SQL Server and select it when it appears. On the following screen, enter a name for the server. In the Linked service field, select New.
-
Enter the connection settings.
- Name - enter a name of your choice.
- Server name - enter the Virtual SQL Server endpoint and port separated by a comma: tds.cdata.com,14333
- Database name - enter the Connection Name of the CData Connect Cloud data source you want to connect to (for example, GoogleBigQuery1).
- User Name - enter your CData Connect Cloud username. This is displayed in the top-right corner of the CData Connect Cloud interface. For example, test@cdata.com.
- Password - select Password (not Azure Key Vault) and enter the PAT you generated on the Settings page.
- Click Create.
- In Set properties, set the Name, choose the Linked service we just created, select a Table name from those available, and Import schema from connection/store. Click OK.
- After creating the linked service, the following screen should appear:
- Click preview data to see the imported BigQuery table.
You can now use this dataset when creating data flows in Azure Data Factory.
Get CData Connect Cloud
To get live data access to 100+ SaaS, Big Data, and NoSQL sources directly from your cloud applications, try CData Connect Cloud today!
