Hi! I’m Cathrine 👋🏻 I really like Azure Data Factory. It’s one of my favorite topics, I can talk about it for hours. But talking about it can only help so many people - the ones who happen to attend an event where I’m presenting a session. So I’ve decided to try something new… I’m going to write an introduction to Azure Data Factory! And not just one blog post. A whole bunch of them.
I’m going to take all the things I like to talk about and turn them into bite-sized blog posts that you can read through at your own pace and reference later. I’ve named this series Beginner’s Guide to Azure Data Factory. You may not be new to ETL, data integration, Azure, or SQL, but we’re going to start completely from scratch when it comes to Azure Data Factory.
Please note: Since I wrote this post, Wrangling Data Flows have been renamed to Power Queries, and there have been many updates in Azure Data Factory. I'm keeping this post as-is, please make sure you also read the official documentation.
In 2019, the Azure Data Factory team announced two exciting features. The first was Mapping Data Flows (currently in Public Preview), and the second was Wrangling Data Flows (currently in Limited Private Preview). Since then, I have heard many questions. One of the more common questions is “which should I use?” In this blog post, we will be comparing Mapping and Wrangling Data Flows to hopefully make it a little easier for you to answer that question.
Should you use Mapping or Wrangling Data Flows?
Now, we all know that the consultant answer to “which should I use?” is It Depends ™ 😄 But what does it depend on?
To me, it boils down to a few key questions you need to ask:
What is the task or problem you are trying to solve?
Where and how will you use the output?
Which tool are you most comfortable using?
Before we dig further into these questions, let’s start with comparing Mapping and Wrangling Data Flows.
On April 4th, 2019, I presented my Pipelines and Packages: Introduction to Azure Data Factory session at 24 Hours of PASS. I was excited to show some cool features and use cases, including how to handle schema drift in the new Mapping Data Flows feature.
In January 2019, I was honored to be asked to contribute to the PASS Insights BI Edition Newsletter. I said yes, of course! 😊 I chose to create an Azure Data Factory Data Flows introduction video. This is a sneak preview of the upcoming Data Flows feature, with a quick walkthrough of how easy it can be to create scalable data transformations in the cloud - without writing any code!
Last year at Microsoft Ignite, I was fortunate enough to interview Mike Flasko and Sanjay Krishnamurthi. This year, I got to have a follow-up chat with Mike Flasko and Sharon Lo! We talked about the recent and upcoming Azure Data Factory updates 🤓
In this interview, Mike and Sharon share the highlights from their session at Microsoft Ignite 2018. What are visual Data Flows? How are Azure Data Factory Data Flows different from the recently announced Power BI Dataflows? What’s on the Azure Data Factory roadmap? And finally, how can you provide feedback and get involved in private previews?
Azure Data Factory Updates with Mike Flasko and Sharon Lo
(I apologize for the unsteady video 😔 Unfortunately, I didn’t see how shaky it was until post-production. If it gets too distracting to watch, please just listen. Mike and Sharon share a lot of interesting things!)
Thank you so much to Mike and Sharon for chatting with me on a busy day 😃
