Data Intelligence for Enterprise Digital Transformations

Data is the most valuable asset in business intelligence. Data steers and leads to business and technology transformation. One significant fact is that data, especially Big Data, is ubiquitous in every enterprise. Every enterprise generates massive amounts of data. As digital leaders, data is our bread and butter; hence, we need to understand every aspect of it in its lifecycle.

Big data differs from traditional data. The main differences come from characteristics such as volume, velocity, variety, veracity, value and overall complexity of data sets in a data ecosystem. Let me provide a quick overview of these critical terms.

Volume refers to the size or amount of data sets. We can measure them in terabytes, petabytes or exabytes. There are no specific definitions to determine the threshold for Big Data volumes. Ironically, even though it is called the Big Data, and it is a signifier, the volume is not the main characteristics of the Big Data as far as architecture, design and deployments are concerned.

Velocity refers to the speed of producing data. Big Data sources generate high-speed data streams coming from real-time devices such as mobile phones, social media, IoT sensors, IoT edge gateways, and the Cloud data stores. Velocity is an essential factor in all phases of the Big Data architecture and management considerations.

Variety refers to multiple sources of data. The data sources include structured transactional data, semi-structured such as web sites or system logs, and unstructured data such as video, audio, animation, and pictures. Variety is also a significant factor for Big Data architecture and management considerations.

Veracity means the quality of the data. Since volume and velocity are enormous in Big Data, veracity is very challenging. It is essential to have quality output to make sense of data for business insights. Veracity is also related to value.

Value is the primary purpose of Big Data to create new insights and gain business value from Big Data. We can create value with innovative and creative approaches taken by all the stakeholders of a Big Data solution.

Overall complexity for Big Data refers to more data attributes and difficulty to extract desired value due to large volume, wide variety, enormous velocity and required veracity for the desired value.

Even though architecturally similar to traditional data, Big Data requires newer methods and tools to deal with data. The traditional methods and tools are not adequate to process Big Data. The process refers to capturing a substantial amount of data from multiple sources, storing analysing, searching, transferring, sharing, updating, visualising and governing huge volumes data in the magnitude of petabytes or even exabytes. I covered these topics in my recent publications to share with my readers.