Some recent data migration lessons
Recently, I’ve been consulting with a mid-sized organisation which is embarking on an initiative to implement a new asset inventory system.
Just a few days into my stint there have been some serious lessons learned by the organisation on how to approach data migration in the right way.
Chances are you’ll face these lessons in the future too so hopefully this article will help you when the time is right.
Most organisations need to learn some basic lessons of survival if they are to succeed.
Some recent data migration lessons
For a company of this size the new target system will have a dramatic effect not only on how it manages its operations but will also provide new services that can hopefully be rolled out in the face of some serious competition in the sector.
Prior to the new system the company had a mix of different databases, spreadsheets and even local paper records out in the engineering centres.
All that will change in the next few months when the new system comes online. However, the company has the small task of the data migration to contend with first.
What has interested me most about engaging with this company is the perception it had about the data migration prior to our consultation.
Data migration was seen simply as an IT technique, bolted on at the end, tool driven, tech-focused, limited business impact and highly tactical. I think a lot of this perception was driven by data volumes, which by industry standards, are indeed quite small.
However, there is a tremendous amount of complexity in the way this organisation does business. What’s more, in the new target system there will be many more demands placed upon the data.
As we assessed the data it was also clear there were major data quality issues that had not been addressed for many years.
So, we’ve put together a plan and we’re moving forward but I think there are a few lessons that are probably worth sharing with other organisations who may find themselves in a similar position:
Data migration is a strategic enabler – treat it like one
Data migration may look tactical in nature but it has the ability to completely destroy any credibility in the strategic system you are about to implement if you get it wrong.
Consider data migration a strategic weapon and move it up the priority and awareness stack.
Joining the party late is very unfashionable
It is extremely difficult to make improvements on a data migration when the main programme is in full swing. Data migration should also be viewed as a separate project that should ideally commence with the start of the main programme.
Leave it late and your system go-live is almost guaranteed to slip.
Data quality is mandatory
The company in question did not invest in a data quality framework. Notice, I use the word framework, not tools because tools don’t create solutions. In their words – “We didn’t want to do a big data cleanse” highlighted their lack of experience in how data quality tools and skills should really be adopted on a migration project.
This is a big topic that we’ve covered in numerous places on the site but there needs to be a complete DQ framework in place from day 1 of the migration otherwise every stage of the project is open to risk, delay or failure.
Get help at the start
It’s often tempting to ask for data migration expert assistance part way through the main programme as your target application won’t really take shape until that point. However, specialists arriving mid-way through the project are akin to a lone policeman showing up at a teenage end-of-term party that has got way out of hand. No-one wants to stop, no-one wants to take responsibility and those who planned it are wondering how much trouble they will face from their seniors.
If you have doubts about your capabilities to deliver the project then seek expert advice right from the outset, before the migration has even been planned.
Complexity is more important than volume
Although the business in question have low volumes of data, their business model is complex. Most businesses look to new target systems as a means of changing or improving their current business model. This change often places pressures on the legacy data to perform amazing new feats of business agility that sadly it isn’t fit for.
Assess the complexity of your underlying data models and relationships within the data, both in the target and source systems. This has to be fully understood, data quality management is critical here for measuring the gaps and defects. Many companies mistakenly view a small data load as a simple migration and fail to factor in the complexity dimension.
Conclusion
All of these lessons point to the fact that data migration should be granted strategic importance.
Ensure that your organization understands the critical role it plays in bringing high-quality applications into existence and work harder to increase the awareness of this much misunderstood discipline.