Table Partitioning in SQL Server – Partition Switching

This post is part 2 of 2 in the series Table Partitioning in SQL Server

Partition SwitchingInserts, updates and deletes on large tables can be very slow and expensive, cause locking and blocking, and even fill up the transaction log. One of the main benefits of table partitioning is that you can speed up loading and archiving of data by using partition switching.

Partition switching moves entire partitions between tables almost instantly. It is extremely fast because it is a metadata-only operation that updates the location of the data, no data is physically moved. New data can be loaded to separate tables and then switched in, old data can be switched out to separate tables and then archived or purged. All data preparation and manipulation can be done in separate tables without affecting the partitioned table.

Partition Switching Requirements
There are always two tables involved in partition switching. Data is switched from a source table to a target table. The target table (or target partition) must always be empty.

(The first time I heard about partition switching, I thought it meant “partition swapping“. I thought it was possible to swap two partitions that both contained data. This is currently not possible, but I hope it will change in a future SQL Server version.)

Partition switching is easy – as long as the source and target tables meet all the requirements :) There are many requirements, but the most important to remember are:

  • The source and target tables (or partitions) must have identical columns, indexes and use the same partition column
  • The source and target tables (or partitions) must exist on the same filegroup
  • The target table (or partition) must be empty

If all the requirements are not met, SQL Server is happy to tell you exactly what went wrong and provides detailed and informative error messages. Some of the most common examples are listed near the end of this blog post.

Partition Switching Examples
Partitions are switched by using the ALTER TABLE SWITCH statement. You ALTER the source table (or partition) and SWITCH to the target table (or partition). There are four ways to use the ALTER TABLE SWITCH statement:

  1. Switch from a non-partitioned table to another non-partitioned table
  2. Load data by switching in: Switch from a non-partitioned table to a partition in a partitioned table
  3. Archive data by switching out: Switch from a partition in a partitioned table to a non-partitioned table
  4. Switch from a partition in a partitioned table to a partition in another partitioned table

The following examples use code from the previous Table Partitioning Basics blog post. It is important to notice that these examples are meant to demonstrate the different ways of switching partitions, they do not create any indexes and they map all partitions to the [PRIMARY] filegroup. These examples are not meant to be used in real-world projects.

1. Switch from Non-Partitioned to Non-Partitioned

The first way to use the ALTER TABLE SWITCH statement is to switch all the data from a non-partitioned table to an empty non-partitioned table:

Before switch:
Partition Switch: Non-Partitioned to Non-Partitioned (Before)

After switch:
Partition Switch: Non-Partitioned to Non-Partitioned (After)

This is probably not used a lot, but it is a great way to start learning the ALTER TABLE SWITCH statement without having to create partition functions and partition schemes:

2. Load data by switching in: Switch from Non-Partitioned to Partition

The second way to use the ALTER TABLE SWITCH statement is to switch all the data from a non-partitioned table to an empty specified partition in a partitioned table:

Before switch:
Partition Switch: Non-Partitioned to Partition (Before)

After switch:
Partition Switch: Non-Partitioned to Partition (After)

This is usually referred to as switching in to load data into partitioned tables. The non-partitioned table must specify WITH CHECK constraints to ensure that the data can be switched into the specified partition:

3. Archive data by switching out: Switch from Partition to Non-Partitioned

The third way to use the ALTER TABLE SWITCH statement is to switch all the data from a specified partition in a partitioned table to an empty non-partitioned table:

Before switch:
Partition Switch: Partition to Non-Partitioned (Before)

After switch:
Partition Switch: Partition to Non-Partitioned (After)

This is usually referred to as switching out to archive data from partitioned tables:

4. Switch from Partition to Partition

The fourth way to use the ALTER TABLE SWITCH statement is to switch all the data from a specified partition in a partitioned table to an empty specified partition in another partitioned table:

Before switch:
Partition Switch: Partition to Partition (Before)

After switch:
Partition Switch: Partition to Partition (After)

This can be used when data needs to be archived in another partitioned table:

Error messages
SQL Server provides detailed and informative error messages if not all requirements are met before switching partitions. You can see all messages related to ALTER TABLE SWITCH by executing the following query, it is also quite a handy requirements checklist:

Summary
Partition SwitchingPartition switching moves entire partitions between tables almost instantly. New data can be loaded to separate tables and then switched in, old data can be switched out to separate tables and then archived or purged. There are many requirements for switching partitions. It is important to understand and test how partition switching works with filegroups, indexes and constraints.

This post is the second in a series of Table Partitioning in SQL Server blog posts. It covers the basics of partition switching. Future blog posts in this series will build upon this information and these examples to explain other and more advanced concepts.

Who is Cathrine Wilhelmsen?

Cathrine is a Microsoft Data Platform MVP, BimlHero, author, speaker, blogger and chronic volunteer who loves teaching and sharing knowledge. She works as a consultant, technical architect and developer, focusing on Data Warehouse and Business Intelligence projects. She loves sci-fi, chocolate, craft beers, ciders, cat gifs and smilies :)

40 thoughts on “Table Partitioning in SQL Server – Partition Switching”

Pingback: SQL New Blogger Digest – Week 3 | The Rest is Just Code

Question about this part: 1. Switch from Non-Partitioned to Non-Partitioned:
have you used, tried or any experience in renaming tables instead of switching? It works also quite fast.

HI catherine, such a clear, concise and well written post covering just the basics of table partition and sliding window technique. thank you for the post. are you planning on writing part 3 covering all the advanced concepts?

Yes, I am currently working on more posts in the table partitioning series, covering other and more advanced topics.

Hi , i need to archive partitioned table from one table to another on regular basis. Example : I have database which will create partition every day. As i required only 90 days data in the table which means 90 partitions. I need to schedule job that can execute daily or weekly basis to move old partition data to new table. Could you please share query for this based on time stamp. Thank you.

you just need to use the $partition function to get your partition number and then switch the partition to the other table. If you are using the same partition function and scheme for both table, then you only need use the same partition ID. if not, then you need to get the partition number of the other table in order to switch the partition to the correct other partition.

Hi Catherine
This is an excellent post on Table Partitioning in SQL Server – Partition Switching
Hoping to learn more from you.

Very well Explained thanks :)

Any excellent overview. Thanks for sharing!

Nice and simple .thanks

geat !

Hello,
I use that techniques to calculate the information coming from my archive database in order to generate a fact table.
As the production records are partitioned and archived based on an expiry date and the archived data are partitioned on their working date, I always have some partitions that are filled in during an archival process.
We work by hours so for an expiry date of ‘20160101 02:00:00’ we will have records for 3 partitions ‘20160101 00:00:00’, ‘20160101 01:00:00’ and ‘20160101 02:00:00’
In order to work on my fact table, I switch out the 3 dates that are archived and I calculate all records for my fact table into a “switch In” table that is partioned on the date.
In order to see if I have correct data, I want to count the number of rows of the switch In table and Switch out table and switch the partition from the table that has the more records into my fact table. So I always have all the records in case of trouble.

I am working with sql server 2014 sp1 version 12.00.4422 (CU2) as for now we have 7500 partitions (working on huge data)

When I try to count the record of my switched table, this process is taking too long (I generally stop after a few minutes).
If I had manually and index on the date column, the count is nearly instant.
If I add this index aligned to the partition, I get stuck too.
if I do it in code, it doesn’t work -> stuck
I tried to update the statistics as incremental and update them only on the switched partition -> stuck
I tried to count by the $PARTITION function -> Stuck
I tried to count by aggregation on the date -> Stuck
if I count on the table without aggregation on date -> good

Do you have any idea on that problem ?

For the moment, I only see a way to use a temp table to calculate the # of records. But this is a loss or ressource and will impact the life cycle of the ssd as I will do it every hours for some millions of data and this will increase in size in the future…
I will try to switch the table into a flat table for calculation and switch it back to the swicth table…

thanks

nb: My archive tables contains now more than 1,5 billions rows. The only way to get good performance with partition is to use where clause on the partition column and be aware of the parameter sniffing of SQL that kills the partition elimination (use cast as do avoid the sniffing), and to use incremental statistics that needs to be updated each time a partition is coming into the table.

I used a second simple table to switch the record in order to count them and switch it back to the table. This is working now.
Although I would be happy to have a better working solution or understanding on why the count of the switched out table is incredibly long…

Great article and presented very well with details!! Thanks Cathrine!

I am implementing partition switching using 3 tables. I am using the following steps:

1. Populate the delta data from the source to a partitioned table in switch schema.
2. Switch partitions that correspond to delta data from table in dbo schema (this is our main table) to a partitioned table in swap schema
3. Switch partitions from partitioned table in switch schema to table in dbo schema
4. Truncate the table in swap schema

Currently the table in dbo schema contains a columnstore index. So as per partitioning requirements in relation to indexes, all the tables involved in partition switching should contain the same indexes. However I can see that the table in swap schema doesn’t necessarily need to have one. Can someone explain why should this be the case?

Well Explained, Very Nice Article ..

For me the code is erroring out saying “Invalid object name ‘GetNums’.” Any idea why?

Yes, you will have to create the GetNums helper function from the first blog post in the series: Table Partitioning in SQL Server – The Basics

Thanks Cathrine :)

Pingback: Tuan Vo (dot) info | Table Partitioning in SQL Server – Partition Switching

Thanks, this and the webinar are both really useful and very well explained for the reader/viewer

Pingback: How To Drop Partition Function In Sql Server | Information

Great . Can’t be better than this :)

I Know that the table must exist in same File group. But I heard that in SQL 2016 version, its possible to switch between databases. Is that true ?

Hello,
Partitioning is Something I use everyday, there are many challenges that I could overcome, but I still have one. When the system is working on the partition, somehow the data is not available on higher level.
I thought about using views to always have a view on all data at any time as we have consumers worldwide and so the data needs to be available at any time. The problem is that SQL server then doesn’t use the partition elimination and so the queries are very slow as the number of row is growing…
Do you have any idea ?

Hi Oliver,

your question is fairly general, so it’s not clear what the specific nature of your problem is. Partitioning is generally used for large tables; the benefits are mostly around inserting new rows (or aging old data.) When it comes to range queries, you can get some performance benefits when a query only looks for results from within a single partition; but for that to work well, I’ve found you need to have a clustered index with the partitioning column being the first element in the index.

Clustered indices (or indexes) are a whole ‘nuther topic on their own, but there’s plenty of good information available about them.

I hope this helps.

Hello Russel, the thing is that I have big table that are filled in each hours by a lot of data. In order to be efficient, I do all the job of inserting/merging data in another table using the partition switching. So every process only works on “few” records at a time.
At that moment, the whole partition is out of the table and so if someone queries the table, then the data are not available.
I am just looking for a way to keep those data available when working.
The size of all makes the process long enough to have that need. I talk about billions of rows in the main table, 11k partitions, several million rows in each partitions…
I already needed to add incremental statistics, change all queries that acces the tables in order to cast the partition column , use option (optimize for unknown),…

What will be the fragmentation status if we switch the huge tables which contains 200 million records will it be taking the same amount of time. what if the page or index is corrupted on the source data will the corrupted page or index gets transferred on the target table.

When you switch a table partition, you just change the “owner” of that partition, you don’t transfer data, you change nothing else. That is why it is so fast, and of course, you can then understand that it doesn’t change the rest.
The advantage of that is that if you have bad data in one partition, then you can prepare correct data in a working table, switch bad data out of the table, swicth the correct data in and then clean the bad data.

Pingback: Indexing a HUGE Table – nate_the_dba

Thanks a lot :) Can you please write more in this series. For example, partitioning current unpartitioned tables, modifying partitions, dropping partitions, etc. ?

Excellent post,Thanks. Really helped me to figure out a problem quickly

Hi Catherine, I’ve got one question. How to set constraints for the edge partitions – first and last – where based on your check script Partition Range is (Period > Infinity and Period 202012 and Period <= Infinity) for the last one ? I tried to use constraints like (Period IS NOT NULL AND Period = '201001') or (Period IS NOT NULL AND Period <= '201001') but both doesn't work. I get an error message : ALTER TABLE SWITCH statement failed. Check constraints or partition function of source table 'BI.dbo.FactPartitioning' allows values that are not allowed by check constraints or partition function on target table 'BI.dbo.FactPartitioning_Staging'. Partitioning function first element is Period '201001'.

PS: First edge partition: (Period > Infinity and Period 202012 and Period <= Infinity)

Something is calculating the text in here so again. First partition is Period > Infinity and Period <= 201001

Last partition is Period > 202012 and Period <= Infinity

Hello, what do you want to achive with this ?

For example Switched out the data from the PARTITION 1 to the staging table. Right now everything works well for every partition except the first and the last partition. So the question is how to specify the boundaries of the PARTITION 1 for the CHECK CONSTRAINT of the staging table to be able to SWITCHED OUT the PARTITION 1 there ?

The switch partition command needs to specify the partition number…
So if you switch from one table to another table that are mapped to the same partition, then it’s easy, you just need to do your command
alter table x switch partition 1 to table Y partition 1
Now if you are using different partition function, then you get the partition number by using the $Partition function….
Normally your table should contain one “date” in partition field of the partition 1 unless you missed something.
Then you get your first date and you use select $partition.partfunction(yourdate) to get the partition number of your second table.
Then you just switch the partition 1 to the partition x of the target table.
You can do the same for the max partition.
If you look the last filled date in your table, be aware that every date that is higher that your last partition will be in that partition and so you cannot just ask the system to give you the last date, but you can look the partition before the last one and add 1 to get the partition number…
Anyway, you should consider to let the first and last partition “empty” to avoid trouble. As when you let them filled with data, merging partition takes time.

I found out the solution. The problem was with the incorrect constraints for the staging table PARTITION 1 (Period 201001).

I’ve implemented following constraint for the staging table and now SWITCHING OUT and SWITCHING IN works well.

ALTER TABLE [dbo].[Staging]
WITH CHECK ADD CONSTRAINT ckPeriodRangeMin
CHECK (Period IS NOT NULL AND Period > ‘200912’ AND Period <= '201001');

It's strange a bit because Period (201001) is not a date but INT column and partitions in partitioning functions are like 201001, 201002 etc.
There is no information at all about '200912'. I'm wondering how SQL Server is able to assess that left boundary for the first partition is '200912' while it's not a date but INT data type ?

Share Your Thoughts?