Skip to content

Comparing Mapping and Wrangling Data Flows in Azure Data Factory

In 2019, the Azure Data Factory team announced two exciting features. The first was Mapping Data Flows (currently in Public Preview), and the second was Wrangling Data Flows (currently in Limited Private Preview). Since then, I have heard many questions. One of the more common questions is “which should I use?” In this blog post, we will be comparing Mapping and Wrangling Data Flows to hopefully make it a little easier for you to answer that question.

Illustration of Person Comparing Mapping and Wrangling Data Flows in Azure Data Factory

Should you use Mapping or Wrangling Data Flows?

Now, we all know that the consultant answer to “which should I use?” is It Depends ™ :) But what does it depend on?

To me, it boils down to a few key questions you need to ask:

  • What is the task or problem you are trying to solve?
  • Where and how will you use the output?
  • Which tool are you most comfortable using?

Before we dig further into these questions, let’s start with comparing Mapping and Wrangling Data Flows.

Continue reading →

Generating SELECT statements in Biml using GetColumnList

Biml logo representing the blog post "Generating SELECT statements in Biml using GetColumnList"

In a previous blog post, we looked at how to generate SQL using Biml. (If you haven’t read that post, you may want to start there and then come back here.) In this post, we will go through how to generate SELECT statements using the Biml column method GetColumnList.

Using Biml column methods

Biml column methods return code fragments. These code fragments can be used as building blocks to generate custom T-SQL statements. For example, the GetColumnList method returns a list of columns, separated by commas, that you can use in a SELECT statement. You can filter the columns and customize the output by passing parameters.

Examples of GetColumnList code fragments

If you have a table with three columns, the default output will look something like this:

[PersonID], [FirstName], [LastName]

But what if you don’t want to select all three columns? Or what if you want to use an alias for your table? No problem! The customized output can look something like this instead:

p.FirstName, p.LastName

We will go through the different ways of customizing the output a little later in this post.

Continue reading →

Published: Last Updated: Categories: BimlTags: ,

Speaking at 24 Hours of PASS: Past Learnings and Future Visions

24 Hours of PASS: Past Learnings and Future Visions

PASS turns 20 this April! To celebrate 20 years of educating data professionals, they are organizing a 24 Hours of PASS with the theme Past Learnings and Future Visions. From April 3-4, you can watch 24 free webinars back-to-back. If you can’t attend all 24 webinars live (who can!?), they will all be recorded. Woohoo!

I’m presenting a brand new session called Pipelines and Packages: Introduction to Azure Data Factory. If Azure Data Factory is not really your thing, don’t worry! You can choose from 23 other topics, including SQL Server, Power BI, AI, Containers, and even Professional Development.

Continue reading →

Bullying, triggers, reactions, and reflections

After spending a week at MVP Summit with old and new friends, I feel energized. My brain has been refilled with lots of happy memories :)

And one not-so-happy memory:

One. Just one negative experience during an otherwise fantastic week. One negative experience that objectively really wasn’t that big of a deal, followed by an absolutely overwhelming number of replies full of love and support. (Thank you 💙) And yet, this one negative experience has dominated my thoughts and feelings for days.

It’s easy to say that I refuse to let one stranger and one negative experience take over and push my happy memories away. Actually doing it is a whole different situation. No matter how determined I am or how much support I get, dealing with an experience like this is never easy for me.

I have trained my brain for many years to respond in a certain way, to gnaw at the negative even when it’s surrounded by positive. That habit does not change overnight just because I want it to. It’s something I need to actively work on to improve, and that requires determination and effort over time.

Writing is my way of dealing with things like this. Structuring my thoughts, describing my feelings, and reflecting on why I reacted the way I did is all part of my process.

But why are you sharing this publicly?

I’m very aware that my social media accounts are mostly highlight reels. I choose to focus on the positive things in my life, and so what most people see is the happy, bubbly, super excited version of me. I wish life was like that all the time :) But it’s not. Most days are not. Some days, you get flat out told that you suck, and instead of being strong and brushing it off, you end up crying and withdrawing from everything and everyone.

And that’s ok. That’s life. We’re flawed, and if we weren’t flawed, we wouldn’t be human. But being human also means we have choices. We can choose to take responsibility for our own feelings, get back up, try to learn from our experiences, and keep going.

Writing helps me. If I can also help someone else by sharing this publicly, whether it is helping one person understand why someone can react strongly to something seemingly small, or showing another person that they’re not alone in experiencing similar feelings, I will count that as a double win :)

So what happened?

Continue reading →