The market for data quality and data management is undergoing a paradigm transition, with the focus shifting to the business user. Business users have always been at the mercy of overburdened IT departments with little resources, but IT is not to blame. Even for the most straightforward inquiry, the standard response used to be “It will be six weeks before we can provide you with that.” However, in today’s world of self-service data, the business requires it immediately. While tools are emerging that enable customers to have a different connection with their company’s data, Data Quality becomes an even more challenging challenge in this new terrain. (Data Science in Malaysia)
“Thirty years ago, systems were in silos,” Kevin McCarthy explained in a DATAVERSITY® interview.
Customer Data Platforms (CDPs)
“Nine-track tapes containing customer savings and checking account information would be place onto a mainframe and then processed through a series of tools to standardise the data and identify the components of the customer’s names and addresses.”
Individual and household linkages would then be form between those records, he explained. Without complete or identical records, it is now possible to standardise personal data, construct associations, and do matching: “The ability to distinguish fuzzy matches, transpositions, and double characters, among other things.”
While approaches have evolved greatly over that time period, what McCarthy refers to as “enterprise players” — huge Fortune 2000 corporations such as IBM and SAP — continue to rely extensively on information technology. These larger firms place a premium on the “one-stop shop” concept in order to offer sophisticated and highly technical capabilities. On the other hand, he noted, developing customer data platforms (CDPs) are more focused on a single market or service requirement.
Data Science in malaysia
Additionally, there are MDM players that differ slightly from CDP players, as well as other organisations that offer a variety of campaign management and marketing solutions. “There are still a lot of tools available, and everyone is vying for a piece of the action.”
McCarthy utilised the metaphor of a safe deposit box to describe the changing nature of data ownership and governance. “You place items in a safe deposit box, and the bank is responsible for keeping them secure. They also provide the box, but the bank isn’t really concerned with what’s inside.” Historically, IT has served as the bank: they provide the technology and software necessary to store the data, but business users care about the content and want to ensure they have access to it when they need it. “And that, my friends, is the switch. Now, the enterprise is seeking that level of control.”
Time to Value is Reduced with Self-Service Data
By providing a sandbox environment in which business users have access to the data, they can experiment, run queries, and investigate it without waiting for IT. Business users can configure those rules, filters, and processes themselves via a drag-and-drop interface, he explained, “which means you don’t have to be a SQL programmer to run SQL-like procedures.”
Possibility of Data Enrichment
Experian has a plethora of data assets and also provides tools for data quality management. McCarthy discussed the potential usefulness of augmenting high-quality consumer data. Names, addresses, emails, and telephone numbers, he believes, are “one of the most difficult data sets to work with.” For instance, from a statistical standpoint, a consumer called Peg Smith on Avenue of the Americas in New York and a Margaret Smith on 6th Avenue in New York are unrelated. However, “Peg” is a nickname for “Margaret,” and Avenue of the Americas is actually 6th Avenue once postal certification is complete – therefore it is the same street, and Peg and Margaret are very certainly the same person.
Contextualizing the Single Customer View
McCarthy discussed the evolution of the term “single customer view” over time to refer to a more contextual picture of each customer’s information.
“From a marketing standpoint, my definition of ‘client’ may be a little broader since I want to include everyone; therefore, I may group them together to avoid having to send out many catalogues.”
Whereas the aim of accounting is to verify that a bill is given to a single specific individual at a single address, “I have to be more precise in how I match those records to locate that consumer. That’single customer vision’ is subjective,” and it differs not only between industries, but also inside a single organisation.
“And we still have an IT legacy.” When they’ve already taken that route and developed a single customer perspective, they expect it to apply throughout the entire organisation.” In actuality, McCarthy is observing that departments have varying requirements and may choose to contextualise data differently. “It’s about offering tools for them to sandbox and experiment with different relationship strategies and matching technologies in order to define ‘customer’ based on their needs — and that is precisely what Experian is doing.”
The Customer Information Challenge
While organisations have prioritised consumer information for more than three decades, the complexity of handling that information has expanded enormously. Along with data input by skilled data entry personnel, call data from customer service reps is now included. Add to it the intricacy of data generated by customers entering information via web forms, which is enter in a variety of ways.
It’s less of an issue in the IoT world, where, for example, a refrigerator reports its temperature. “My refrigerator is not having a bad day when it sends the temperature in Celsius rather than Fahrenheit.” However, someone submitting their name on a web form may spell it “McCarthy,” “MaCarthy,” or “MacCarthy,” depending on their level of concentration at the time, and if an existing record differs in any manner from that spelling, a duplicate record can be produce.
No matter how much data standardisation is implement, the underlying problem of recognising a unique client persists. “Charlie’s record appears six times since he was entered three different ways, but he is truly the same person.” Data quality issues are inherent in the process, and he stated that they will continue to exist as long as people continue to enter data. “Regardless of how hard you try to prevent people from entering incorrect data into the system, you cannot protect everyone, and when you have millions of people submitting data, something is bound to go wrong.”
A Data Management Company
Experian is a market leader in data and analytics on a global scale. While the company is well-known for their credit products, they also offer data quality management software. Their attention has shifted significantly in recent years to the business user and making Data Quality Management more accessible to them. The new advancements place a premium on the business user’s simplicity of use and time to value, as well as on Experian’s substantial data assets.
Source: data science course malaysia , data science in malaysia
