The Dangers of “Data Shoveling”: A 5-Year-Old Demonstrates

Like any good parent, I made a promise to take my daughter out for dinner on the condition that she cleaned her room as I couldn’t even walk across the floor. I left her alone to get started, and 10 minutes later she came running out stating her room was clean. Upon entering, sure enough, her floor was clear!

The same scenario plays out frequently in digital transformation, where a data team is pressed to transfer data over to a new system. They may be under financial or timeline pressures to get data moved, and moved fast, and “voila”, they are able to create magic.

Whether you believe in magic, or not, it doesn’t work with data transformation or in cleaning rooms. Upon opening my daughter’s closet, I found waist-high piles of toys, clothes, papers, and even some food scraps. She had literally “shoveled” everything into her closet in order to “clean” her room. We find the same thing in the data world, where data is simply shoveled into a new system.

In speaking with companies who have attempted data shoveling, the common explanation is “We will clean it up once the new system is up, and we have some more time”. Sounds good in theory, but let’s ask my daughter her thoughts. The following morning, she asks me where her red dress is.

“Did you put it in your closet?”, I asked.

“I don’t know?”, she replied.

“Well, better go look”, I smirked.

20 minutes later she came out with her wrinkled red dress, smiling that she had found it. When I walked past her room, I noticed that all the junk from her closet was back out on the floor!

The point should be clear by now. By “shoveling” data into your new system you are not only delaying but potentially expounding and creating more issues down the road. Ideally, data needs to be mapped, cleansed, and standardized before moving. Otherwise, you have the same mess of data, but in a new system – and one that you paid a lot of money for! In my daughter’s example, she previously had some piles and some recognition of where things were in her messy room. But once shoved into the closet, those piles and concepts of where things might be disappeared.

When you push unclean or ill-prepared data into a new system, the standards from your old system do not apply to the new system (especially in the case of customization), and the new system is simply not able to read or structure through the piles of dirty data. You have just created far more work and put the business at risk. Symptoms of data shoveling may include:

• Lack of data visibility
• Dirty data breaks the configuration of the new system
• Reporting becomes muddled and people don’t trust the new reports
• Fixing issues after the fact takes more time and money
• Go-live is delayed
• Integrators and clients start pointing fingers as to who’s to blame
• Everyone (especially executives) becomes angry, and… (fill in the blanks)!

Not a good situation. Keep in mind as you head into any data transfer that not knowing or utilizing data migration best practices will cost you heavily down the road. There are methodologies and toolsets that can help expedite this tedious process. Such a small cost compared to the risk of data failure at go-live. The Data Readiness effort should ideally start with an initial data assessment completed as part of your system and SI selection (or soon thereafter) with details mapped out during implementation planning. Make data a critical workstream of your digital transformation.

And for those of you without kids, go look in your garage or shed for a similar example😊

If you have questions or would like to discuss more, please reach out to info@validdatum.com. We also welcome you to subscribe to receive weekly posts on data management, security, and technology transformation best practices.