As organisations consider migrating content to SharePoint or SharePoint online, it is important to understand that there are many technical considerations that can affect the quality and the duration of the migration. This article will identify and address a few key concepts that, if properly understood, can improve any migration to SharePoint or SharePoint Online.

With any migration, getting off to a good start is very important. Before any migration planning or preparation can begin, the data that is to be migrated must be interrogated so that the organisation can extract crucial information about the content that is to be migrated.

For the purposes of this document, the collection of all documents and folders located in any given storage platform is known as the “Corpus”. The constitution of the corpus can have a significant impact on transfer throughput. A detailed interrogation of the documents to be migrated will help to define the corpus and its impact on the migration project. The following questions should be answered during the interrogation process.

How many documents and/or folders are included in the migration? 

SharePoint performance is optimised when documents are stored in optimal folder sizes. SharePoint will perform better and the migration will operate faster when there are as few folders as possible while folder item recommended limits (currently 5000 – related to list view threshold) are also respected. In addition to respecting Microsoft list view guidelines, following this pattern will also result in fewer folder objects and ultimately fewer API calls that need to be made to execute the migration.

What is the total storage volume and average file size of the files being migrated? 

Of course, the actual amount of data that needs to be transferred will affect the duration of the migration. But what many organizations to not realize is that the average file size can have a dramatic impact on migration efficiency as well due to the number of API calls necessary for a given amount of data.

Given 100GB of data, if that 100GB consists of (10,240) 10MB files, transfer throughput will be considerably higher than if that data consisted of (209,715) 500KB files. This is because SkySync will have to make approximately 200,000 more API calls to transfer the 100GB of 500KB files vs 100GB of 10MB files.

If the corpus is weighted more towards many small files versus relatively fewer large files, it should be expected that the transfer throughput will generally be lower due to the latency expense of significantly more API calls.

What are the departmental or taxonomical classifications of the documents? 

Answering this question will help to determine how granular the site and library structure should be. When an extended number of departmental or business process driven site and library structures are required to manage a wide variety of content types, migration configuration and execution will be more cumbersome, resulting in a longer migration project duration. When fewer sites and libraries are needed, migration configuration and execution will be more efficient, resulting in a shorter migration project duration. 

Understanding the corpus is a necessary requirement before proper planning and preparation for a migration can begin.

Understand and Prepare for Records Management and Content Disposition

There are obvious benefits to minimizing the content that is migrated. Less content can mean less planning, less structure that must be deployed, less taxonomy configuration and of course, lower storage and infrastructure costs. But eliminating unnecessary content can also have another tremendous fiscal impact by also lowering RISK!

Many organisations must adhere to strict compliance requirements that dictate how documents must be classified and retained for a defined period. Once that period has expired, any content that does not have specific business value should be eliminated. If it is not eliminated there is a possibility that it can be used against the organisation in the event of litigation. However, if that content was properly disposed after the compliance mandated retention period has expired, it can no longer harm the organisation.

Even if the organisation is not beholden to strict compliance requirements, it is still recommended that structured content disposition be practiced. Common sense destruction of content that no longer holds business value will reduce risk as well as facilitate the management of long term storage costs.

Process Improvement

The “old” way is often not the “best” way. SharePoint is an extremely flexible platform that is capable of being twisting into a knot to mimic any given legacy document management solution.  While that is an admirable quality of the platform, it is also an enabler for the propagation of inefficient design. 

In the SharePoint paradigm, there are often many ways to accomplish the same thing. To determine the most efficient solution that follows Microsoft prescriptive guidance, the following steps should be considered.

• Engage a business analyst and power users to deconstruct business processes. 
• Leverage the knowledge of the power users to determine if there would be a better or more efficient way to accomplish the process.
• Determine if the improved solution was impeded by the legacy technology and bring in a SharePoint Architect to determine if SharePoint features can facilitate the improved solution.
• Allow the SharePoint Architect to design an optimized solution that implements the thought leadership of the power users by while leveraging the feature richness of SharePoint.
• Solutions should be streamlined to minimize any custom development to facilitate maintainability and future upgrade and/or migration.
• By engaging the power users throughout solution re-envisioning, the result will be better processes that have the benefit of immediate user adoption. This is a far better approach than just attempting to “mimic” the legacy solution using new SharePoint technology. A migration project is usually the best possible time to implement process improvement.

Phased, Slow Walk Migration

One of the most effective ways to execute a migration is to use a phased approach as opposed to a “big bang” cutover. There is far too much risk in the big bang approach unless it is a small and manageable migration.

A better approach is to start with a small departmental file move first. Since IT is usually responsible for managing or even executing the migration, their own department is a suitable candidate for a pilot migration. Once a migration target has been identified for the pilot phase, the next order of business is to choose a cutover plan. One of the best ways to execute a migration is a sort of “slow walk” migration. 

The slow walk migration consists of an initial structural deployment followed by the configuration of a continuous copy/synchronisation of the source content with the new destination structure. Ideally, this should be an automated synchronisation that continuously propagates file create, update and delete changes from the source system to the destination system (SharePoint or SharePoint Online). 

When the official cutover takes place, the source system is simply placed into a read only state and a small amount of final create, update and delete changes propagate to the destination. This method results in a smooth and relatively short cutover process that is easy to back out of in the event of an issue.

Migration Duration

One of the most important concepts of a migration to SharePoint online is that the duration of the migration can’t be accurately predicted. There are many factors that affect migration duration. Some can be managed and others are out of the control of the migration team. Several important migration duration factors are identified below.

• Empirical evidence of achievable throughput must be gathered through testing.  Even then, migration duration predictions will fluctuate based on the average file sizes of already migrated content vs average file size of remaining content. In other words, if actual migration metrics are gathered using a small number of large files, inflated metrics may result in an artificially short migration duration. If the actual migration metrics are gathered using a large number of small files, deflated metrics may result in an artificially long migration duration.
• SharePoint Online is multi-tenant environment that implements throttling / rate limiting.  In order to protect all tenants, Microsoft must not allow any one tenant to monopolise the available hardware resources for a shared tenant environment. At any given time, there are a finite amount of resources available to all tenants. The SharePoint Online throttling engine is dynamic. There may be times when more resources can be consumed for migration processing and there may be times when far less resources are available and significant throttling must occur to keep the environment responsive for all tenants.
• The quality of source system and migration hardware will affect duration either positively or negatively.  It is quite common for an aging file share or legacy document management system to affect overall migration throughput due to inferior performance.  It is very possible that the migration could go faster but only to the detriment of business users who are impacted by the pressure that the migration places on the legacy hardware.

In addition, the migration hardware itself must be robust. SQL Servers and, specifically, the quality of the SQL Server storage subsystem (IOPS, etc.) will have a significant impact on the ability of the migration solution to facilitate high transfer throughput.

Summary

The purpose of this article was to address important high-level technical factors that should be considered when migrating to SharePoint or SharePoint Online. A deeper dive into these concepts could consume an entire book. The intention of this document was to help organisations understand some of the important challenges that will be encountered as well as provide some prescriptive guidance on how to work through those challenges.

Kevin Ells is VP of Marketing at SkySync, a platform developed to integrate ECM, network file systems and cloud file sharing services. For information and additional white papers on SharePoint Migration visit http://skysync.com/sharepoint_migration-whitepaper/