Skip to content

Tag: Azure Data Factory

Connecting to HubSpot from Azure Synapse Analytics or Azure Data Factory

HubSpot logo.

I spent a couple of hours today figuring out how to connect to HubSpot from Azure Synapse Analytics. Since it wasn’t straightforward for me, I decided to blog about how I did it. Hopefully, this post can help one or two others (or future me!) save some time 😊

The HubSpot Linked Service

The Azure Synapse Analytics (or Azure Data Factory) linked service connects to HubSpot using a Client ID, Client Secret, Access Token, and Refresh Token:

Screenshot of the HubSpot linked service.

How do we get these values from HubSpot?

The overall process is to:

  1. Create a private app in HubSpot to get the Client ID and Client Secret
  2. Authorize the private app and get authorization code
  3. Exchange authorization code for Access Token and Refresh Token
  4. Create linked service in Azure Synapse Analytics or Azure Data Factory

Let’s dig into the details! 🤓

Training Day at SQLBits 2022: Data Integration using Azure Data Factory or Azure Synapse Analytics

Are you new to Azure Data Factory or Azure Synapse Analytics? Maybe you’re an expert SSIS developer wondering how to take that first step into Azure? Do you feel a little overwhelmed by all the possible ways you can move and transform data these days? Join me at the ExCeL London on Tuesday, March 8th, 2022! I will be delivering a training day at SQLBits 2022 called Beginner’s Guide to Data Integration using Azure Data Factory or Azure Synapse Analytics 🥳

Person announcing the Beginner’s Guide to Data Integration using Azure Data Factory or Azure Synapse Analytics training day at SQLBits 2022.

My goal is to help kickstart your journey as an Azure Data Engineer, and for all of us to have a great and safe day of learning and networking. SQLBits are taking covid safety seriously by requiring proof of vaccination / tests, wearing masks, and ensuring proper spacing between seats. I will be there in-person and hope you will be able to join us too!

(Oh, and if you want to get the most out of your week at SQLBits? Check out Paul Andrew’s training day on Wednesday called Azure Data Integration Pipelines - Advanced Design and Delivery 🤓)

Ask the Experts: Data Integration in Azure Synapse Analytics (at Data Toboggan 2021)

On Saturday, June 12th, 2021, I will be moderating an Ask the Experts session at Data Toboggan! This is a free event focusing on Azure Synapse Analytics. There are over twenty sessions and lightning talks scheduled, covering topics such as architecture, performance, tools, data integration, machine learning and much more.

If you have any questions about Data Integration in Azure Synapse Analytics (or Azure Data Factory), join us! You don’t want to miss this session 🤓

Data Toboggan logo showing a toboggan (sled) going down a hill.

Overview of Azure Data Factory User Interface

Woman standing next to a projector showing the Azure Data Factory logo.

In the previous post, we started by creating an Azure Data Factory, then we navigated to it. In this post, we will navigate inside the Azure Data Factory. Let’s look at the Azure Data Factory user interface and the four Azure Data Factory pages.

Azure Data Factory Pages

On the left side of the screen, you will see the main navigation menu. Click on the arrows to expand and collapse the menu:

Animation of expanding and collapsing the pages menu in the Azure Data Factory user interface.

Renaming the default branch in Azure Data Factory Git repositories from "master" to "main"

Branch icon.

In Azure Data Factory, you can connect to a Git repository using either GitHub or Azure DevOps. When connecting, you have to specify which collaboration branch to use. In most cases, the default branch is used. Historically, the default branch name in git repositories has been “master”. This is problematic because it is not inclusive and is very offensive to many people.

The Git project, GitHub, and Azure DevOps are making changes to allow users to specify a different default branch name. GitHub and Azure DevOps will be changing their default branch names to “main” in 2020. I fully support this change and will be doing the same in my projects.

In this post, we will go through how to rename the default branch from “master” to “main” in Azure Data Factory Git repositories hosted in GitHub and Azure DevOps. Then we will reconnect Azure Data Factory and configure it to use the new “main” branch as the collaboration branch.

For these examples, I’m using my personal demo projects. I’m not taking into consideration any branch policies, other users, third-party tools, or external dependencies. As always, keep in mind that this is most likely a larger change, both technically and organizationally, in production and enterprise projects. 😊

The Short Version

  1. Create a new “main” branch in your Git repository
  2. Set the new “main” branch as the default branch in your Git repository
  3. Delete the old “master” branch in your Git repository
  4. Disconnect from your Git repository in Azure Data Factory
  5. Reconnect to your Git repository in Azure Data Factory using the new “main” branch as the collaboration branch