Kirk Liemohn is a Principal Software Engineer at ThreeWill. He has over 20 years of software development experience with most of that time spent in software consulting.
To get started, we’ll break the Microsoft 365 Migration out into discrete phases. We will provide guidance on what happens during each phase. A key proven step that should not be skipped is to include a pilot for the project. This gives you a chance to send “water through the pipes” for the entire migration process and address any issues/risks. Issues will come up. The next step is to plan how you are going to triage issues. The next thing we’ll discuss is defining the support process for the migration. We’ve learned through the years that having an archive strategy can be a key component to keeping a cap on the budget and the overall migration time.
Editor’s Note – We originally developed this guide just for targeting complex SharePoint online migrations. As a result, many of the technical aspects of this guide are related to SharePoint. We are taking this content in a more holistic view based on what we are doing with customers. This includes migrating content into not just SharePoint online, but also Microsoft Teams, Yammer, and products that are a part of the Microsoft 365 suite.
Microsoft 365 Migration Process in Stages
Any large or complex migration needs to have a process in place for managing the migration. ThreeWill breaks up our migration process into four stages:
In the Assessment stage, we work with the client to understand the high-level Migration needs. We like to do this in a workshop. In this workshop, we will devote some focused time over a few days to gain a better understanding of what is needed. Focused time is devoted over a few days by us to gain a better understanding of what is needed. This may take the form of a couple of full-day sessions or several half-day sessions.
During the Assessment stage, we review the overall vision and strategy for the migration. We also discuss the scope of the migration. Then we develop a high-level plan for the migration including tasks, resources, and tools required to complete the migration. We break the plan into a high-level Product Backlog. From this, we can identify an estimated budget and schedule for the migration.
During the workshop, we also discuss the current state and desired future state including key future state capabilities as well as project risks. The output of the workshop is a baseline of the Migration Project Charter which includes:
- SharePoint Vision, Strategy, Objectives, and Critical Success Factors
- Migration Approach and Plan (Roadmap)
- Resource Plan
- Migration Schedule
- Initial Risk Assessment
- Initial Communication Plan
- Estimated Budget for Migration
Failure to plan is planning to fail.
It sounds trite, but with large and complex migrations it is very true. You need to plan with a detailed approach to be successful. It’s no coincidence that the Plan portion of the process is the largest subject. We break this stage into the following components:
A big part of the plan is understanding what is being migrated, so we need to create an inventory. At a high level, the inventory includes:
- Site collections
- Customizations (e.g., custom branding or web parts)
- 3rd party products (e.g., Nintex Workflow)
- Integrations with Other Systems (e.g., BCS connections)
- Users and user permissions
There are a lot of details that can go into the inventory for each site collection such as the site collection administrators (useful for communications), quota used (size), time zone, language, running/configured workflows, views that exceed the list view threshold, and many more. Information considered critical to the migration is used to create a complexity score for each site collection. This information can be used to help schedule the migration as well as understand the effort involved. If your inventory is accessible to site owners, it can also be used to show the status of the site’s migration.
Tools are available to aid with creating an inventory:
- Microsoft’s inventory, planning, and reporting tool: Microsoft Assessment and Planning Toolkit.
After the inventory in the migration is cataloged, the mapping can be done. The most obvious mapping exercise is mapping the source site collections to the target site collections. Sometimes this is a “lift and shift” exercise where the mapping is little more than a URL change. Other times the mapping can be more involved (e.g., mapping a single site collection into multiple site collections). The Archive Strategy can play a big part in this as well, where some of the source content is archived and not migrated.
There are several other types of mapping to consider as well, including:
- Mapping source and target site templates – when moving to a different version of SharePoint (e.g., SharePoint 2013 to SharePoint Online) some site templates in use in the source environment may not exist in the new environment.
- Mapping users – if you are not moving from one user directory to another, this is not necessary. However, when moving from on-prem to online, you will likely need to map windows accounts to Microsoft 365 accounts. This may require feeding a tool with mapping data or a mapping transformation script (for an example from one tool, you can read more on Metalogix Content Matrix transformations here).
- Mapping customizations/integrations – another aspect of “mapping” is determining how to handle customizations, 3rd party integrations, and custom requirements. Some may or may not be supported in the new environment. Once cataloged, each case should be examined. If required, proof of concepts (POC) should be put in the plan to address any unknowns.
Potential FTC Mapping Issue
If you are moving from SharePoint on-prem to SharePoint Online, mapping customizations and integrations is a critical exercise. Many of these will not move over easily. Full Trust Code (FTC) is not supported in SharePoint Online and many home-grown and 3rd party integrations rely on FTC. Migrating each application that uses FTC is a project by itself. FTC applications must be re-architected to use the Cloud Add-in (or “App”) Model (CAM) if you want to migrate them to SharePoint Online. Site owners must be given lead time to convert their applications if needed, so early communication is important.
Even if FTC is not used, there can be complications moving custom applications. There are even issues migrating custom applications built on SharePoint when moving from one on-prem version to another on-prem version. For example, the REST API changed significantly between SharePoint 2010 and SharePoint 2013.
After the mapping exercise is well underway, patterns can be found on how to do the migration. Efficiencies may be gained by automating based on those patterns. For example, if you are migrating many site collections in a “lift and shift” manner, you can automate the migration of those site collections using custom code and a migration tool. Even for those cases where custom automation is not warranted, you may need to plan for testing various methods of migration to determine which will work best for you.
POCs were mentioned for mapping customizations and integrations. They can also be beneficial for testing critical use cases with the migration tool. If not already decided during the Assessment stage, the migration tool (or tools/utilities) is usually determined during the Plan stage. All migration tools are not created equal and source/target environments play a critical role in what is and is not supported by the tool. For example, do you need running workflows migrated? What about external users? There are a lot of scenarios to consider especially when changing versions of SharePoint or hosting environments.
Finally, during the Plan stage, significant effort must be devoted to communications. What is going to be communicated, to and by whom will it be communicated, and when will it be communicated? How and when are pilot site owners contacted? Make sure you allow for ample time to create the appropriate communication documents during this stage.
The output of the Plan stage includes the plan itself. We do this in the form of a more detailed Product Backlog broken out into Sprints with a timeline. See ThreeWill Approach for more details. Other updates include:
- Custom requirements
- An update to the communication plan
- An update to the Migration Project Charter
The Verify stage starts with several activities including:
- Performing POCs outlined in the Plan stage.
- Writing and testing any custom migration code/scripts.
- Testing critical use cases with the migration tool.
- Refining the communication plan and updating documentation as needed.
- Ensuring support and triage processes are in place
After those activities are complete and pilot owners are ready, the pilot process can begin.
Results from the pilot(s) and from other tests are fed back into the plan. The scope may need to be adjusted based on findings. The plan and Migration Project Charter are adjusted accordingly. This could involve a change to the timeline or scope.
The Execute stage is when the real migration is done. Content is moved by manually configuring the migration tool or through automation. Issues are triaged and handled with the support process. Sites are cut over to the new environment. Other sites are archived. Delta migrations are performed as needed. Communication is provided to all parties as planned.
Note: Surprises during the Execute stage are going to happen. If enough time was not given to the Verify stage, then the chances of surprises having an impact on the project are more likely. These may require adjustment to scope and/or budget during the Execute stage.
Finally, at the end, the migration project itself is transitioned to a sustainment team.
Always Include a Pilot
A pilot involves getting a set of users and sites to agree to be part of the early adopters to move to the new environment. Or at least to test moving to the new environment. Ideally, this group of individuals is a good cross-section of the organization and can be flexible while you work through the kinks in the migration process.
The pilot process for migrations is intended to test the “water through the pipes” of the entire process. This clearly needs to involve migrating various types of sites in your organization, but it also needs to test communications, support processes, and issue remediation processes.
With end-to-end tests of these processes, issues found during the pilot can be addressed before large-scale production migrations occur. Otherwise, issues are found when the spigots are fully open and it can be hard to update your processes if helpdesk and issue remediation are consuming a lot of bandwidth.
Communications During Pilot
The pilot process can tell you how effective you are with your communications. Are your policies and run book sufficient for your Site Owners? Are the users able to use your self-help documentation? Is your support organization able to use the knowledgebase documentation? Are banner and email communications effective?
Tip: Don’t confuse initial test migrations with the pilot. Before any pilots, you will want to perform test or proof of concept (POC) migrations of content to validate that the migration tool can successfully move data from the source environment to the target environment. During these tests, you may uncover some gaps in what the tool can do for some specific sites or test cases in your organization. These test migrations can also be used to validate any automation or customizations you are putting in place around the migration tool. These tests are done in the Plan and/or Verify stages of the project.
Once the initial testing is done, the pilot process can further vet your migration tool and any automation. Ideally, the number of pilot sites is large enough that you expand the variability beyond the initial testing and learn about more improvements for your content migration process.
Include Actual Users
In addition to testing different types of sites and content, the pilot should include actual users to review the results of the migration. Their feedback is crucial as they represent all of the individuals that use the system after it is migrated. You need to make sure the pilot participants are willing and able to provide feedback in a timely manner. It is very easy for those users to get caught up in their daily activities and not take the time to pitch in. Getting feedback too late from pilot participants can impact the schedule.
After content is migrated, the pilot process helps uncover holes in your support and issue remediation process. Is your ticketing system ready for migration? Does your helpdesk or first line of support have proper instructions on how to handle incoming issues? Can they properly handoff issues to Tier 2 or 3 support? Your triage process will likely be improved during the pilot. New types of issues may be discovered and techniques for handling them either proactively (maybe through automation or manual tweaks in the process) or after-the-fact.
Each One is Unique
With large and complex migrations, each one is unique. The pilot process can help answer and resolve many issues that show up in migrations in a more controlled way. For more complex migrations, running at least 2 pilots is suggested. Early planning is key to allow for the time to prepare for the pilots and make course corrections with information gathered during them. Without including the entire process during the pilots, the value will be limited to only those aspects of the process that are in the pilots, adding risk to those parts of the processes that are not being exercised until full-on production migration.
Have a Plan for Triaging Issues
Triage typically refers to sorting or prioritizing patients in need of medical care. For migrations, we are referring to the process of sorting and prioritizing issues that occur during the migration process.
There are plenty of issues that can occur with large and complex migrations, so you want to have a way of processing these issues that don’t overwhelm the team supporting the migration. Early testing can give you an understanding of many, but not all issues. Running Pilots will help even more. Some issues may be reported by the migration tool or automated checks, but others may only be found through visual inspection.
Ideally, many of the issues found during testing and pilots are later prevented by modifying the migration process. Some examples of this could be:
- Using the migration tool to map missing users to a service account and communicating via the Policy Document that migrating user accounts for terminated or inactive individual accounts is not supported.
- Migrating very large sites using a more measured approach – maybe migrating one subsite or library at a time instead of the whole site collection at once.
- Writing scripts to overcome deficiencies in the tool for your given scenario. Maybe the tool does not properly configure information management policies or some obscure feature in SharePoint that is a critical use case for your organization.
Automatic Triage System
Even if you can overcome many issues proactively, you may find that you need to have a process for handling them after-the-fact as well. If you are migrating enough content, an automated triage system may be warranted. This system can be used to check for migration issues and compare them to known issues – we call them Issue Definitions. The Issue Definitions can prioritize the issue and assist in assigning the issue (i.e., to a site owner, tier 1 support, tier 2 support, etc.). It can also describe how to fix the issue by referencing Knowledgebase articles. Alternatively, it may indicate that solving this issue is not supported as discussed in the Policy Document.
The POCs may give insight to some initial Issue Definitions, but during the pilot processes, the Issue Definitions really start to take shape. It is during those pilots that the automated triage system is refined and improved as new issues come in that don’t match up to existing Issue Definitions. This feedback loop is important and can even be continued during production migration.
The objective of triage in migration scenarios is to identify and classify typical and unique challenges, issues, or anomalies that exist in a specific environment. In small migrations, the objective may simply be to enable prioritization of help desk calls. In massive migrations, the triage can assist in informing the other components of the migration that may impact the business; e.g. issues that impact revenue or customer-facing resources.
Define the Support Process
A large migration requires a well-defined support process that has been communicated and tested thoroughly, beginning with the Pilot testing. The support process must in all cases complete in-incident resolution and address the following issues:
- Inputs from multiple sources: As discussed in Step Five, some issues can be discovered by the migration tool or by the internal migration team. Other issues will be reported by site owners and users through a service desk or other communication methods.
- Closed documentation feedback loop: As incidents are resolved, support documentation (knowledgebase, self-help, policies, run books) should be reviewed and updated as needed. This ensures future incidents can be resolved more efficiently and opportunities to improve the migration process are more easily identifiable.
- Escalation path: In efforts to resolve incidents, support resources will need a process to raise issues with migration engineers, developers and other team members. This requires a well-defined escalation path with a return to support process.
- Clear path to resolution: All support processes for incident management should have a clear path to incident resolution, minimizing opportunities for issues to go unresolved.
Here is a simplified flow diagram on how the support process may look. Larger migrations will need to have a ticketing system. Additionally, they may want to use the corporate help-desk system if one is already in place.
Use an Archive Strategy
If there is a lot of content to migrate, an archive strategy can be a key component to keeping a cap on the budget and the overall migration time. The general idea is to archive content that is no longer needed or actively used. And additionally, only migrate content that is needed going forward. This can reduce clutter in the newer environment and can vastly reduce the migration effort and schedule if enough content is archived.
The first step is to determine the guidelines or rules for what will be archived or deleted instead of being migrated. Some factors that may influence this decision include:
- Recent activity (or lack thereof)
- Amount of content – e.g., a team site may be created and no significant content was ever added or a user created a blog site, created 1 or 2 posts, and then stopped updating the content
- Known business circumstances – e.g., divisions that have spun off or teams that no longer exist
- Compliance reasons – governance, eDiscovery, audit, etc.
You will also want to decide on the granularity of what is archived or deleted (e.g., site collection, subsite, library). Once the factors are agreed upon, this can be fed into the Inventory and mapping exercises.
The next step is to determine how to archive the content. Third-party migration and archive tools are one option for this. If SharePoint is on-prem, you may also want to consider site collection backups or content database backups. Keep in mind that depending on the approach the backup may not contain everything that is used by a SharePoint site such as farm solutions or files stored on the SharePoint server file system (aka, the “SharePoint Hive”).
For all content that is archived, you will want to have a strategy in place to restore that content if it is needed at a future date. Keep in mind that some content (such as the aforementioned farm solutions and server files) may require extra attention. Those maintaining the environment(s) must understand the steps required to handle the restoration process.
An alternative to an archive strategy that still reduces the number of migrated sites is to consider a hybrid environment. To do this, some content remains available on the old farm until its end-of-life. This may make sense for farm solutions that have a limited lifespan when moving from SharePoint on-prem to SharePoint Online, or for business solutions or applications that have a limited lifetime or ROI.
When choosing a hybrid solution, consider integrations that may be useful to assist when going across multiple farms for navigation, searching, and profiles. There are plenty of articles on a hybrid strategy with SharePoint. There are many in regards to a hybrid between SharePoint on-prem and SharePoint Online. One place to start is here.
You’ve now got a better understanding of the steps required for a successful Microsoft 365 migration. Notice how important the approach is to the migration and how many steps are about reducing risk. For example, doing a pilot first to test out all your processes. Let’s take a final look of each of the steps.
- Break the Process into Stages – We’ve identified 8 distinct phases – Assess, Plan, Inventory, Map, Streamline, Communicate, Verify and Execute.
- Always Include a Pilot – This is a non-negotiable if you want to be successful.
- Have a Plan for Triaging Issues – Issues will come up. Design an appropriate Triage Plan to handle them when they do.
- Well-Defined Support Process – You’ll need to handle input from multiple sources, document resolutions to incidents, and have a clear path to resolution of incidents.
- Use an Archive Strategy – An archive strategy can be a key component to keeping a cap on the budget and the overall migration time.