Woohoo! I’m very happy to announce that Jason Horner (@jasonhorner) and I will be presenting a precon at SQLSaturday Nashville 2018! The precon will be held on Friday, January 12th, and is focused on SSIS and Biml. We will also be at the SQLSaturday where Jason will be presenting about Dimensional Modeling and I will be presenting about Biml. This will be my first time visiting Nashville, so I’m very much looking forward to it :)
SQLSaturday Nashville 2018 Precon (Jan 12)
Jason had been working on a precon idea for a while that would bring together all his experience using SSIS to deliver ETL projects. I wanted to develop a new precon focusing on Biml and SSIS. Instead of working on two separate precons, we decided to team up and combine our knowledge. We will be delivering this precon for the first time in Nashville, and we’re looking forward to helping attendees by providing guidance on how to solve challenges they might face in their projects.
You can read more about why we wanted to present this precon in our interview on the Nashville BI User Group website.
Check out the full abstract and register for our precon on bit.ly/ETL-Precon-Nash. A regular full-day is only $129, and for only $165 you can also attend one of the half-day precons on Thursday. What a bargain!
SQLSaturday Nashville 2018 Sessions (Jan 13)
Jason and I are also presenting a regular session each. Jason will be presenting his Dimensional Modeling Design Patterns: Beyond the Basics session, and I will be presenting my Biml for Beginners: Speed Up Your SSIS Development session.
Check out the full SQLSaturday Nashville schedule and register today to get all the news and updates. You can also follow @SQLSatNash on Twitter and use the hashtag #SQLSatNash to join the conversation. Please help spread the word to all your friends and coworkers, and make sure you sign up before the event is full.
Hope to see you in Nashville in January! :)
One of the sessions I was most looking forward to at Microsoft Ignite 2017 was New capabilities for data integration in the cloud with Mike Flasko. In that session, he talks about Azure Data Factory (ADF) v2 and its new first-class SSIS support.
After the session, I convinced Mike Flasko and Sanjay Krishnamurthi to have a chat with me :) We talked about what’s new in Azure Data Factory v2, including the updated pipeline application model with a new visual design canvas, new Software Development Kits (SDKs) for working with Azure Data Factory, the new Integration Runtime, and the ability to run SSIS packages inside Azure Data Factory v2.
Azure Data Factory v2 with Mike Flasko
Follow Mike Flasko on Twitter @mflasko, and keep an eye out for more news about ADF and SSIS! I may or may not have convinced him to do another interview with me in a couple of months :)
Thank you Mike and Sanjay for sitting down with me on a busy day! If you were unable to attend Microsoft Ignite 2017, you can now watch Mike’s session New capabilities for data integration in the cloud on-demand:
Other interviews from Microsoft Ignite 2017
Data Science with Buck Woody
SQL Server 2017 with Bob Ward
High Performance Analytics with Sunil Agarwal
BIDS Helper and BimlExpress does not support creating SSIS project parameters from Biml out of the box. There are workarounds (and I have previously blogged about my solution for creating project parameters from Biml), but why is this not a standard feature in BIDS Helper or BimlExpress? Many people have asked about this, so I sat down with Biml creator Scott Currie (@ScottCurrie) to get the full story.
Why doesn’t BIDS Helper or BimlExpress emit SSIS project parameters from Biml?
Well, technically it could, but it shouldn’t. The user experience would have serious issues, leading to confusion, frequent errors, and the potential for data loss. How can that be?
Continue reading →
If you are using BIDS Helper or BimlExpress to generate SSIS packages in the Project Deployment model, you have probably noticed that it is not possible to create project parameters from Biml. You can write Biml for the project and project parameters, but BIDS Helper / BimlExpress will only generate the SSIS packages for you and not the SSIS project parameters. The recommended solution is that you create the project parameters manually before you generate your SSIS packages from Biml.
(Want to know why? Read Scott Currie’s explanation in my blog post Why can’t I create SSIS Project Parameters from Biml?)
However, if you are a lazy developer like me, you probably don’t want to create and update project parameters manually. Perhaps you want to automatically create or update project parameters based on some metadata? You can do that!
Let’s take a look at a (semi-hardcoded, semi-hack) solution for creating SSIS project parameters from Biml in BIDS Helper / BimlExpress :)
Continue reading →
T-SQL Tuesday #68 is hosted by Andy Yun (@SQLBek). Many SQL Server defaults are not ideal, and most of us have a list of defaults we always change. Andy wants us to Just Say No to Defaults and blog about what, why or how we change defaults.
If you are an SSIS developer like me, there is a big chance that the ProtectionLevel in SSIS Packages is on top of your list of defaults to change. The default ProtectionLevel is EncryptSensitiveWithUserKey (ugh), but most of the time it is not the best option. Raise your hand if you have ever asked your favorite search engine for advice on issues like “SSIS package fails in SQL Server Agent job” or if you have ever heard someone exclaim “but it works on my machine!?” :)
There are many great blog posts about the different ProtectionLevels, why you probably want to change to DontSaveSensitive as your default, and how to use configurations and parameters instead of encrypted SSIS packages. I will not go into details about any of that in this post, but I will use ProtectionLevel as an example default property you want to change in many SSIS packages at the same time.
How do you batch update properties in existing SSIS packages? You probably don’t want to open up every single package and change them manually?
Continue reading →