Data Quality: An On-going Challenge For The Financial Industry And Regulators

By Vipul Parekh

 

Every financial, insurance, and asset management services firm has significantly changed its operations due to wide ranging global regulations such as Basel III, Dodd Frank, MiFiD, EU Solvency II, and the Volker rule in the US and Europe. More regulatory reforms are in the works for Asia. At the same time, all sectors in the financial markets have seen exponential increases in data volume through a variety of sources. The majority of CEOs want to leverage data in the quest for leaner organizations, better operational efficiency, and increased revenues. Sound data quality has always been critical and continues to play the central role in the future success of multi-billion dollar investments in this area. The quality of data not only impacts the success of these initiatives, but also is imperative to the firm’s business agility, productivity and survival likelihood. For regulated firms in financial, insurance and health care sectors, poor data quality can also results in breaches, potential fines, and reputation loss.

With the current regulatory focus, most of the firms have implemented data governance and IT infrastructure to process large amounts of data and produce regulatory reports in a timely manner. However, the next big challenge for regulators and firms is to start making sense of this data and assess if generated data can be used for meaningful market analysis at the macro (behavior of a group of banks) and micro level (behavior of a specific bank). The ultimate goal of these initiatives is to produce better quality data so that market trends and risks can be identified early enough to avoid crisis. Both firms and regulators have acknowledged the importance of data quality as one of the key bottlenecks in obtaining full transparency of the market. For example, Dodd-Frank regulation has established harmonization rules to closely monitor and improve data quality of key attributes for a number of derivative asset classes. Similarly, the Basel Committee has recognized the critical importance of data quality and issued a consultative paper (BCBS 239) with principles of a sound data governance framework (to be implemented by 2016). The firms looking to commercialize their data obviously need to produce near pristine data quality for their enterprise data analytics initiatives.

Essential Characteristics of Good Quality Data

With so much reliance on good quality data, let’s look at some essential characteristics of good data, what leads to bad data, and a few key points to ensure better data quality. The following are essential characteristics to assess if quality of data is good:

Completeness  - Data attributes should contain values other than blank or default values.
Validity – Data attributes need to contain valid values as specified by the business to ensure consistency and integrity.
Accuracy – Data attributes should contain accurate values by enforcing conditional or field dependency rules.
Ease of use – The data values need to be easily interpreted without the need of complex parsing.
Availability – Data should be available in a timely manner for analytical use.

Firms should spend time performing current state assessment of data quality relative to the above characteristics and establish performance metrics for quality measurements.

Factors Leading to Bad Data Quality

Historically, most data quality issues result from poor data governance processes, tactical solutions, and weak rules governing firm’s systems and data stores. Some key root cause factors contributing to poor data quality are:

Lack of standardized approach to data definitions, metadata taxonomy, and validation rules
Weak change management policy for data dictionary updates
Duplicate copies of reference data with no clear identification of golden data source
Data ownership not clearly defined between the business and IT
Same data is interpreted differently by businesses and systems
Data may not be fitting the purpose for analytical or reporting use
Poor change management around data governance processes

How to Ensure Good Data Quality

At a high level, strong data governance processes with sound data validations at the time of ingestion, a well-designed data model, and metadata taxonomy are the key ingredients to produce good quality data. However, maintaining good data quality is an on-going challenge and may not receive enough attention by business and IT teams. So, establishing a formal data quality team to lead the initiative with focus on building key performance metrics, tools, and controls to monitor data quality will truly pay off in the long run. The team can collaborate with data stewards and data governance teams to re-enforce data quality standards and provide feedback into changes proactively. Additionally, the following investments will further add value to firms:

Build solid understanding of where to find data, what it means, and how to use it within business and technology teams.
Establish sound change management policies for firm’s data dictionaries, quality rules and data models.
Identify the golden source of data and avoid keeping duplicate copies of data.
Establish data standards (i.e. standard formats for dates, allowed values, alphabets, codes etc.) and leverage reference metadata to reduce variations in values.
Keep data validation rules and processes close to data ingestion processes as much as possible.

As firms and regulators rely more and more on data for analysis and decisions, the focus on data quality will continue to increase. Establishing a culture of data sensitivity and awareness is imperative for firms to maintain good data quality standards. The initiatives, such as formal data quality programs, training of data SMEs and IT teams to find value from data, and right tools to monitor data quality are key necessities for the success of data analytics and regulatory reforms and will be essential for firms to thrive in the long run.

 

 

Vipul Parekh is a leader of Financial Services practices at Optimity Advisors.