Data Management Services
In most organisations the management of data is treated as secondary to the core of business operations; there is no focused area for the management of data or content, and many have no executive responsible for driving effective use of data for competitive advantage. Most enterprise data initiatives are focused on individual projects rather than trying to develop enterprise-wide capabilities such as data quality, data governance and data operations. The desire to treat data as an asset or to use data to deliver efficiencies or advantage can only be supported by robust data management services. Standard practices developed over the last decade and embodied by groups such as DAMA-DMBOK set out the core data management services that are critical to successful data exploitation and control.
Data Management Solutions
The architecture & technology options for any data management solution, must consider your technical roadmap. Data management services should primarily be consistent to help drive maturity, however the technical choices made can have a huge effect: for example, an event driven data architecture or microservice architecture pattern has considerably different data management requirements compared with monolithic architectures- for example when assessing data lineage (aiding data governance efforts) across cloud polyglot data stores and multiple data warehouses.
Data Management solutions vary according to the data architecture under management. Technology choices should consider each area of the data management lifecycle and how technology enables that capability; driving decision making this way will enable the right technology choices.
Typical solutions for a centralised data management model include SQL Server MDM, Oracle EDM, IBM DataStage, Talend and Databricks. For microservice or event driven architecture data management models’ typical solutions include Ab Initio, Informatica, Spark, Kafka, Python, AWS Glue, AWS Kinesis, AWS Firehose, Azure Data Catalog, Azure Data Factory, Azure Event Hubs and Google Dataflow.
Although a wide range of technology solutions fulfil data management best practices, in our experience the key factor is not the solution but that the best approaches to data management are being adhered to. At Peru Consulting we focus on the business problem not the technology.
Why Does your Company need Data Management Services?
We define data management services as those services that when combined provide a sustainable foundation on which to scale and gain advantage. Although they might not seem exciting, they are critical to building the appropriate data management engine necessary to improve data quality, data processing, business rules management and business intelligence and insight. What are these foundational services and why would you need them?
Data Operating Model: Provides structure and clarity of purpose
The operating model covers all competencies needed to execute a successful data management programme: this covers principles, the scope of data sources, a portfolio of use cases to achieve stakeholder goals and the governance framework supporting that.
Metadata Management: Provides clear path to data usage and applications – accelerates decision making
Metadata provides data about data – it describes its location, its use and how it is related to other pieces of data. The management of metadata is a key enabler for modern data led enterprises. Metadata management can help support regulatory compliance (GDPR), improved business productivity by identifying where data resides and how to use it and can help in impact analysis when introducing new programmes of work or new applications.
Data Quality Management: Fosters trust in data and drives ownership and accountability
Most organisations lack essential data quality roles or don’t know where to place them for optimal impact. The assignment of data stewards are a key part of successful data quality initiatives and sustaining quality management practices. Data quality has many dimensions, and it is important to assess which dimensions are appropriate for each data entity – not all will apply.
Master Data & Registers Management: breaks reliance on legacy technology data stores and enables higher levels of data automation
Master data is the consistent and standardise method of identifying and describing the core data entities of an organisation. The three parts to MDM are focused on people, processes and technology – in that order. Master and Reference data management also includes the ability to manage attribute hierarchies, ontology (vocabulary that relates to business domain or specifies semantics) and synonyms (alias’ or alternate names)
Document & Content Management: provides access and search capabilities across unstructured data stores
Data sitting outside of databases – one files shares, in OneDrive – created internally or downloaded from external resources form a large part of the content set that exists in most organisations. The traditional ways of managing these documents is not fit for purpose if this content is to be used for insight. The ability to trawl, derive subjects, tags, context and relevance is readily available but organisations use of these tools is limited. Cloud productivity suites such as MS365 can provide some of this functionality.
Data processing capabilities covers numerous capabilities. Some examples of these are complex mapping, real time processing, event processing, streaming, batch, bulk data and ad hoc data processing. Typically, though data processing is looked at in two distinct phases: data preparation and data transformation.
Data preparation and cleansing capabilities include data quality techniques, depersonalisation, transformation (classification and manipulation of data for processing), data streaming. Event processing, batch and bulk data processing.
Data transformation capabilities cover validation, cleansing, enrichment, de-compression, encoding, classification, anonymisation, aggregation, formatting and error/exception handling.
As well as the previous data governance considerations for legal directives, citizen consent, data stewardship, GDPR etc, appropriate security controls must be maintained. We use the four A’s (Access, Audit, Authentication & Authorisation) with the necessary Entitlement to ensure data is managed securely. This could be as granular as applying security protocols in the DMZ and user access controls in the API management layer, or system wide data protection such as obfuscating, pseudonymisation and encryption of data in transit or at rest. Ultimately monitoring through appropriate mediums and technology such as Splunk to ensure proactive data protection and data security. At Peru Consulting we have specialist data protection and cyber security personnel to support data management activities with extensive experience of internal policies, processes & governance.
Data, Machine Learning, APIs and Analytics are at the heart of all digitisation efforts – how these come together and how people advocate and use these elements will determine success or failure. There needs to be an element of “adaptive governance” – that is focusing on accountability and trust rather than rules and control. Gartner has highlighted seven foundations for governance of data:
Value & Outcomes: governance efforts need to be directly attributable to business outcomes. Governance should aid decision making and provide clarity and direction to corporate wide initiatives. The value comes when governance is done by exception rather than as a rule.
Trust: determining what data is trustful and which is not is an important part in helping determine if information is valuable knowledge. Trust is provided by understanding data lineage and context.
Transparency & Ethics: how information and knowledge is managed should be based on clear and ethical principles. Governance decisions based on these principles need to be clear, defensible and documented. This is particularly true of adherence to GDPR and other regulatory requirements.
Risk & Security: information management approaches should balance business opportunity with risks and security. As data is there to support business outcomes and decisions any governance decisions on how to use data need to assess the trade-off between opportunity and risk.
Education & Training: individuals involved in the governance of data and information need to the right blend of skills, competencies and attitude towards the changes being assesses and decided on – otherwise data initiatives will fail. People involved in governance groups need to be aware of why they are attending, their role and the scope of decision making allowed.
Collaboration & Culture: probably the most important aspect of data governance is enabling a culture that is seen as collaborative and enabling as opposed to centre led and bureaucratic.
Accountability and Decision Rights: is ownership of data assigned and is it clear what authority ownership gives and the extent of decision making authority there is? In many organisations’ accountability is assigned but there is a lack of understanding what this means causing inability or unwillingness to make decisions causing delays to initiatives.
What is Data Management Consulting?
Clients come to Peru Consulting for data management consulting to:
- Transform their knowledge assets and generating IP
- Understanding customer behaviour and using trend analysis to enhance the customer experience
- Enable Business Intelligence on which to inform decision making and proactively monitor performance
Typical challenges we see include:
- Multiple sources of information, with significant manual reporting overhead, limiting confidence in a single version of the truth
- A lack of understanding about their customer’s behaviour and needs across all their interactions
- Inefficient internal operations, creating significant cost overhead and delays
Our consultative approach initially focuses on:
- Data Discovery to understand the data you have, where it resides and how it is accessed
- Data Health Check to determine the accuracy of your data and increase your confidence in utilising it
Data Maturity Assessment to understand how you compare to your competitors and industry best practice