Mehmet Yildiz's Blog: Updates from Dr Mehmet Yildiz, page 6

Tolerance to Uncertainty for Technical Excellence, a Mandatory Attribute for Digital Transformations

Tolerance to uncertainty and ambiguity is one of the well understood and accepted technical leadership attributes. Technical and technology leaders deal with future outcomes. They make the future. However, the future is unknown to us as humans, as outcomes are affected by a myriad of causes beyond the control of people. Therefore, uncertainty is a reality to deal with future events.

Uncertainty is a closely related term to risk management. Taking risks is one of the necessities and most fundamental characteristics of leaders for success. Risk and opportunity are like inseparable yin and yang. We can even simplify at the most fundamental level that no risk, no opportunity. Leaders know that opportunities are created by taking risks. They take calculated risks using their logic and intuition and mitigate them to be able to deal with uncertainty in a systematic way. One of the techniques they use is to learn from past failures and use these learnings in their risk-taking engagements.

Dealing with uncertainties and ambiguities for digital transformations creates new options and choices leading to innovation. Having more options to choose and linking those options in a creative, an intelligent, and an integrated way can create new transformational solutions. To this end, digital transformation that excellent leaders strive for require tolerance to uncertainty and ambiguity. Master digital transformers passionately embrace uncertainty.

In my recent publication "A Technical Excellence Framework for Innovative Digital Transformation Leadership", I attempted to provide valuable insights for digital transformational leadership using a pragmatic five-pillar framework. This empowering framework aims to help the reader understand the essential characteristics of technical and technology leaders in a structured way.

Enterprise Modernisation: from Chaos to Coherence

Enterprise modernisation is a long journey moving the enterprise from chaos to coherence. The modernisation process includes every aspect of the enterprise. Even though enterprise IT systems look only a tiny bit of an organisation in overarching enterprise, this domain by itself can be gigantic especially for the large organisations. Enterprise IT systems include business IT processes, business data, business applications, IT infrastructure, and IT service delivery. These domains can even be more complicated with the addition of geographical factors such as adding multiple countries to the equation. The good news is that these primary domains can be modernised iteratively in parallel.

Both a top-down and bottom-up approach can be applied. At the top tier business, IT processes and at the bottom tier IT infrastructure. These two domains can independently be modernised using parallel activities. However, an integrated approach is essential as there can always be dependencies from multiple angles.

Once the modernisation strategy is set by the team, Enterprise Architects refine the strategy and convert it to the architectural speak. The strategy document is a critical artifact to bring all parties and stakeholders on the same page. Then the Enterprise Architects identify the critical dependencies among these domains based on the short term, midterm and long term considerations.

By using the strategy and considering the dependencies Enterprise Architects develop a high-level roadmap to inform the sponsoring executives. This roadmap can indicate the key outcomes, timelines and a ballpark cost for the overall modernisation. These indications can be very high level as there may be many factors affecting timelines and cost.

Once the roadmap for the enterprise modernisation is set the Enterprise Architects need to make a comprehensive viability assessment considering the current state of the scoped initiatives, their indicative future state and the strategies to reach the end state. This viability assessment must include key risks, constraints, and dependencies. The viability assessment is the most informative tool an Enterprise Architect can provide to the sponsoring executives to make informed decisions.

After review and approval of the viability assessment, Enterprise Architects delve into collecting the high-level requirements of the solutions based on the domains we mentioned earlier. As dealing with the requirements of those domains can be daunting, Enterprise Architects delegate the requirements collection process with the domain and program architects, including business analysts. In this phase, the role of Enterprise Architect is to coordinate and facilitate the requirements management team which can consist of multiple architects and business analysts.

After requirements are collected and analysed at a reasonable amount, the next important activity is to prioritise the requirements based on business impact. Enterprise Architects need to develop criteria to prioritise the requirements based on factors depicted in the strategy and roadmap documents, as well as the financial and business priorities set by the sponsoring executives.

I provide a practical view on enterprise modernisation in my recent publication A Modern Enterprise Architecture Approach Empowered with Cloud, Mobility, IoT & Big Data.

A Modern Enterprise Architecture Approach Empowered with Mobility, Cloud, IoT & Big Data

Modernise and transform the enterprise with pragmatic architecture, powerful technologies, innovative agility, and fusion

I have been practising enterprise architecture over two decades. Large organisations are substantially challenged with rapid change in technology and increasing demands of consumers. Every large organisation that I worked for had some transformation and modernisation programs to some extent at the enterprise level. I witnessed several failed initiatives caused by multiple factors which could be in their control or beyond their control. One of the major causes of the failure was difficulty in dealing with complexity. Enterprises have multiple dimensions spanning to many domains. These domains are tightly interrelated; hence, a minor issue with one domain can be reflected in many others.

For example, in a typical large organisation, just strategy and planning phase took over a year while hundreds of highly paid employees were churning and debating the ideas extensively. Once the program finally reached a consensus on the scope and approached the requirements management phase, the entire budget for the program was consumed. The organisation had to make all those talented people redundant. This typical and unfortunate example was a valuable lesson learned on how important to approach the modernisation iteratively rather than trying to perfect everything upfront. From hindsight, they could have set the strategy at a high level for a single domain and only plan one aspect of the strategy in the selected domain, test it with the allocated budget, and produced some desirable results.

The other reasons for failure are too much focus on technologies which were challenging to implement at enterprise-wide due to inhibitive cost, lack of required functionality, and capabilities perspectives. For example, while an organisation could have started testing the Cloud with a cheap public Cloud offering and move their workloads iteratively, they were trying to build a full-fledged private Cloud platform with many emerging technologies and expensive gear. The hidden cost in such a monolithic approach, unfortunately, destroyed all good intentions.

There are many more similar lessons learned from failure; therefore, I want to share my experience how these deadly errors can be prevented with a different mindset, novel approach, an innovative structure, and with use of supportive tools, and empowering technologies.

I authored this book titled A Modern Enterprise Architecture Approach Simplified with Cloud, Mobility, IoT & Big Data to provide essential guidance, compelling ideas, and unique ways to Enterprise Architects so that they can successfully perform complex enterprise modernisation initiatives transforming from chaos to coherence. This is not an ordinary theory book describing Enterprise Architecture in detail. There are myriad of books on the market and in libraries discussing details of enterprise architecture.

As a practising Senior Enterprise Architect myself, I read hundreds of those books and articles to learn different views. They have been valuable to me to establish my foundations in the earlier phase of my profession. However, what is missing now is a concise guidance book showing Enterprise Architects the novel approaches, insights from the real-life experience and experimentations, and pointing out the differentiating technologies for enterprise modernisation. If only there were such a guide when I started engaging in modernisation and transformation programs.

The biggest lesson learned is the business outcome of the enterprise modernisation. What genuinely matters for business is the return on investment of the enterprise architecture and its monetising capabilities. The rest is the theory because nowadays sponsoring executives, due to economic climate, have no interest, attention, or tolerance for non-profitable ventures. I am sorry for disappointing some idealistic Enterprise Architects, but with due respect, it is the reality, and we cannot change it. This book deals with reality rather than theoretical perfection. Anyone against this view on this climate must be coming from another planet.

In this concise, uncluttered and easy-to-read book, I attempt to show the significant pain points and valuable considerations for enterprise modernisation using a structured approach. The architectural rigour is still essential. We cannot compromise the rigour aiming to the quality of products and services as a target outcome. However, there must be a delicate balance among architectural rigour, business value, and speed to market. I applied this pragmatic approach to multiple substantial transformation initiatives and complex modernisations programs. The key point is using an incrementally progressing iterative approach to every aspect of modernisation initiatives, including people, processes, tools, and technologies as a whole.

Starting with a high-level view of enterprise architecture to set the context, I provided a dozen of distinct chapters to point out and elaborate on the factors which can make a real difference in dealing with complexity and producing excellent modernisation initiatives. As eminent leaders, Enterprise Architects are the critical talents who can undertake this massive mission using their people and technology skills, in addition to many critical attributes such as calm and composed approach. They are architects, not firefighters. I have full confidence that this book can provide valuable insights and aha moments for these talented architects to tackle this enormous mission turning chaos to coherence.

Architecting Big Data Solutions Integrated with IoT & Cloud

IoT, Big Data and Cloud Computing are three distinct technology domains with overlapping use cases. Each technology has its own merits; however, the combination of three creates a synergy and the golden opportunity for businesses to reap the exponential benefits. This combination can create technological magic for innovation when adequately architected, designed, implemented and operated.

We can start with a high-level view of these technologies, defining them from architectural perspectives and provide an overview of their relationships in creating the synergies and potential benefits for business.

To identify the relationships amongst these three technologies, we can start with the IoT and Big Data relationship. I propose IoT as the input or source data for the Big Data solutions. Big Data includes many types of data sets; however, the IoT data is essential to create innovations, new insights, and new business opportunities.

You may ask where does the Cloud Computing fit in this magical combination. Cloud Computing presents enabling and empowering capabilities, not only as a hosting platform for the Big Data but also providing advanced processing and analytics in an economical, scalable, reliable and agile manner.

Another view for identifying relationships can be obtained by looking at IoT an enriching factor for Big Data and Cloud as empowering. With the contributions from IoT and Cloud, the Big Data can achieve unprecedented results for creating new businesses and growing existing ones.

Big Data solutions without Cloud can be costly and complicated due to infrastructure requirements for storage, process and analytics requirements. Not only the enormous volume of the Big Data but also other vital characteristics such as a variety of data sources, velocity (speed), the veracity of data and required value from data in motion makes it a very complex system.

One of my recent books titled Architecting Big Data Solutions Integrated with IoT & Cloud delves into details for these three distinct technologies integrated for coherence and creating strategic business insights with agility.

Transformational Benefits of Using the Cloud for Big Data Analytics

There are many business benefits and compelling use cases of Cloud Computing for Big Data solutions and particularly for Big Data processing, storage, and analytics. Speed to market for Big Data solutions is crucial for competitive advantage in business. The cloud service model is ideal to meet this critical business challenge and requirement.

Big Data can be sent to Cloud-based data lakes with breakneck speed. The analytics engines in the Cloud infrastructure can be made in the required power thanks to the elasticity and scalability of the Cloud services model. Increasing resources in traditional infrastructure model, as opposed to Cloud services, can be slower and more costly.

Cost is an inevitable business factor for enterprises. Cloud computing pay as you go and on-demand characteristics can make the solutions more economically viable by reducing total cost of ownership. Use of Cloud services based on consumption is sharing financial risk with an outsourcing organisation.

In terms of cost, also the Cloud service model can help reduce management overhead of the infrastructure resources. Infrastructure support costs can be very high. For example, supporting traditional infrastructure system require many professionals supporting the operations, procurement of the hardware, software and other components for different purposes can increase the cost substantially for an organisation. The cost incurred by those employees and upfront hardware and software costs is reduced by using Cloud services for complex Big Data solutions.

By using Cloud service model, creating a proof of concept or pilots for Big Data solution can be performed in much shorter times without upfront investment for required hardware, software, and other infrastructure components. This is especially more important for smaller and start-up companies with limited IT budget. This is a desirable business proposition for the entrepreneur and start-up companies.

Using Cloud-based Big Data services, the analytics processes can be completed in faster speed with reliable service levels providing speed to market for the business. This can enable new revenue streams for the data in motion which could be much more difficult in traditional settings in relatively shorter timeframes with established SLAs.

For the complex Big Data solutions, the use of Cloud services can provide a cultural shift. This positive cultural shift can yield more productivity for the solution team. The team can collaborate better and be more innovative as they don’t have to worry about a myriad of in-house infrastructure issues.

Access to Cloud services can be tailored based on the user’s profiles and can be taken from their desks or any location they may prefer working from using any device they may need. Through the use of proof of concepts and easy to use sandbox environments in Cloud services, they can be more experiential, innovative and uplifting for improving their solutions.

For those who may be interested in digital transformations using the magic of integrated Cloud, IoT and Big Data solutions, I pointed out compelling benefits, business and architectural considerations for using Cloud services for Big Data Analytics in my recent book titled Architecting Big Data Solutions Integrated with IoT & Cloud

IoT Solutions Security and Privacy

When comparing all aspects of an IoT (Internet of Things) solutions, it is clear that security tops the list. In society, there is a great deal of fear surrounding the perception that IoT systems are easily hackable. To an extent, this fear is justifiable as the consequences of hacked IoT devices and services can often be life-threatening.

In relation to security, the other concern for IoT solutions is privacy. In IoT solutions, security and privacy go hand in hand. This means that whilst we are analysing and validating the security requirements, we also consider the privacy requirements.

Some IoT solutions could be compared to unchartered waters. As IoT solution architects, we need to understand the security pain points in these dangerous zones. The main reason for this prerequisite is that IoT is an emerging field; hence, there are still loopholes that should be systematically identified and addressed.

Therefore, we need to start asking powerful and open-ended questions to understand the security issues, risks, concerns, constraints and dependencies. At a high level, we may start posing the questions as to ‘What are the security pain points in this solution?’, ‘What are the new technologies that may create risks?’ and ‘How can we address the identified risks?’ among many more exploratory questions.

Of course, by asking many more questions, we prompt our minds to find effective resolutions for each concern. As IoT solution architects, we usually cover the breadth rather than depth in developing solutions, like any aspect of the solution, it is essential to have a security subject matter expert on hand to help delve into the details of security risks, issues, dependencies and constraints. These consulting subject matter experts can help validate our solution proposals. Therefore, it is highly recommended that the security subject matter experts review the security architecture of the solution and give their approval.

In addition to the security subject matter expert, the solutions are also reviewed by a security governance body in an organisation. The members of the governance body may review various aspects of the security, such as identity management, authorisation, encryption and so on. Then, it is the IoT Solution Architect’s role to ensure the recommended security actions fit into the overall solution. As you may have guessed, specialists of a specific domain are often unaware of the other domains and the overall solution. Understanding the importance of this point is critical as architects often make the assumption that subject matter experts in security know every aspect of the systems or solutions.

As IoT lead solution architects, we need to analyse and define the key security threats. Then, we need to propose solutions to address those threats in the Security Model of the IoT solution. These points in each solution building block need to be carefully reviewed by the security subject matter experts and peer-reviewed by other solution architects in the program or organisation who understand the security landscape for applications, middleware, data, hosting infrastructure, databases, network, storage and all other aspects of the solution.

IoT Security and privacy requirements need to be analysed using reliable trust and assurance frameworks. These requirements need to consider the privacy laws in the geographies of the solutions that are developed. These requirements may not use traditional security controls. These requirements may have been developed in agility and may differ, state to state, country to country, and continent to continent.

As a critical point, I delve into details for the security aspect of IoT in my recent book titled "A Practical Guide for IoT Solution Architects: Architecting secure, agile, economical, highly available, well-performing IoT Ecosystems". You can find my updates by following my author profile on Amazon.

Business Benefits of Big Data as a Service (BDaaS)

BDaaS (Big Data as a Service) is a new and evolving Cloud-based service for creating Big Data solutions. It is a type of outsourcing model for deployment of Big Data projects. The service has several different facets. The most common type of BDaaS is a supply of data management and analytics tools performing the actual analysis and providing required reports in various formats. Some DBaaS service providers can also offer additional services such as advisory and consulting services to complement their consumption-based services.

BDaaS is an excellent opportunity for small business or start-ups and even large organisations with a limited budget and resources for Big Data solutions. It can increase their competitiveness, innovation, and revenues.

BDaaS is based on SOA (Service Oriented Architecture) combined with virtualised Big Data storage, scalable and event-driven processing and analytics tools provided Cloud service, consumption model. There are different deployment models for BDaaS. Some service providers provide, core, performance-based, feature-based and integrated BDaaS services. The terms and conditions may be different for each service provider. The critical consideration for the Big Data solution architect is to be aware of the requirements and which service model fits into the requirements of the solution in the most cost-effective way.

The primary benefits of BDaaS are rapid deployment, capacity, and scalability on-demand with established QoS (Quality of Services) for network speed and SLAs (Service Level Agreements). The key business outcome is the agility and cost-effectiveness with guaranteed service levels without an investment of funds on massive internal IT costs.

BDaaS is a relatively new service and proliferating. We can find many Cloud service organisations providing Big Data and Analytics based on self-service in their Big Data platforms. Some well known BDaaS providers are Amazon Web Services, Google Cloud Dataproc, Salesforce Wave Analytics, IBM BigInsights on Cloud, Microsoft Azure HDInsight, and Qubole Data Service. The service is well received and profitable for consumers and providers hence the number of service providers globally proliferating and more granular and customised services are being offered.

To inform solution architects and designers, I provided a broader view for Architecting Big Data Solutions Integrated with IoT & Cloud: to create strategic business insights with agility.

Integration of Cloud to IoT can create new revenue streams

Cloud marked a paradigm shift to Information Technology and Computing field. Growing trends for Cloud Computing is undeniable. However, adding the IoT to the Cloud, in other words, integrating IoT with Cloud, makes the real difference for new market opportunities and poses invaluable business value propositions. IoT Cloud is a crucial player in the digital ecosystem. The primary role the Cloud plays in IoT is to facilitate the essential data integration of the solution components. This role results in agility and cost-effectiveness. These critical outcomes - cost and speed to market- are essential to success for any organisation.

IoT solutions are mainly used to provide real-time information to consumers through the lens of Big Data Analytics. The data required to generate real-time information can be massive in scale; hence, it is called the Big Data. The Cloud, along with computing power, storage, analytics, metering and billing components, can make this information available for the consumers securely, reliably and rapidly.

The integration of Cloud to IoT can create new revenue streams. Integrating the Cloud with the IoT can create new business models enriched by real-time analysis and directly-consumed information at the same time. There are many use cases for this integration to create valuable insights leading to new revenue streams. It is evident that without the Cloud, the IoT can hardly add any value due to its real-time data and information-rich nature.

The addition of the Cloud to the IoT can also contribute to improved security, availability and performance of the IoT solutions. Cloud providers have rigorous security, availability and performance metrics established based on a service consumption model. In particular, IoT-enabled Cloud systems seem to pose additional security risks hence require additional security measures.

When integrated with Edge computing, Cloud computing can add better value to the IoT ecosystem. The main reason for this is that Edge computing can do the filtering of useless, dirty, and noisy data for the Cloud to focus on the usable and valuable bits.

It is essential for the IoT solution architects to understand the Cloud Computing architecture and how it can be integrated into the IoT solutions. Being aware of the capabilities of Cloud technologies and offerings can be extremely beneficial in creating large-scale commercial IoT solutions. To this end, my recent publication "A Practical Guide for IoT Solution Architects: Architecting secure, agile, economical, highly available, well-performing IoT Ecosystems" intends to create necessary awareness on this matter.

Introductory pages to my recent Kindle books

In this post, I'd like to provide the links to introductory pages of my Kindle books.

A Technical Excellence Framework for Innovative Digital Transformation Leadership
Architecting Big Data Solutions Integrated with IoT & Cloud: Create strategic business insights with agility  A Practical Guide for IoT Solution Architects: Architecting secure, agile, economical, highly available, well performing IoT Ecosystems

A Practical Guide for IoT Solution Architects: architecting secure, agile, economical, highly available, well-performing IoT ecosystems

In this post, I’d like to introduce my recent studies on IoT (Internet of Things) architecture for guiding the IoT solution architects. I have been studying IoT for a decade now and since then technology has substantially improved however there was a huge gap to guide solution architects to undertake IoT solution architectures addressing key architectural factors and concerns.
With an attempt to fill in this gap, I took the plunge and authored a simple guide to help the IoT architects reflecting my insights stemming from my solution architecture background and IoT knowledge. I set the intended audience for this book comprising information technology architects producing IoT solutions, enterprise architects who want to understand the IoT solution development in large organisations and other IT professionals who wish to become IoT solution architects to produce solutions in IoT ecosystems.
The main purpose of this book is to guide solution architects and designers who want to understand the architectural rigour for IoT solutions in an easy, effective, and clear way. Reading this book can help these professionals understand the key challenges and practical resolutions in IoT solution architectures broadly without going too much into the details. It aims to provide them a broad checklist hence may exude confidence. Each chapter focuses on the key methodical aspects that form the framing scope for this book; namely, security, availability, performance, agility and cost-effectiveness.
While authoring this book in the last two years, I conducted a comprehensive review of the practical industry-based publications on IoT. Through my findings, I concluded that there was a tremendous need for secure, agile, highly-available, well-performing and cost-effective IoT systems. I found out that the contemporary issues in the literature and associated media mainly revolve around the five topics, which comprise the key business concerns; namely, security, availability, performance, agility and cost-effectiveness. In this book, I use these five key points as the use cases of effective and efficient IoT solutions.
I also provide useful definitions, a brief practical background on IoT, and a concise guiding chapter on solution architecture development. The content is mainly practical; hence, it can be applied or be a supplemental input to the architectural projects at hand. It is vendor and technology agnostic, purely focuses on architectural rigour. Supplemented by a succinct list of key points to take away, I provided 50 key action points which can be applied to the IoT solution architecture projects.
The book is available at Amazon both as Kindle Version and the Paperback. I look forward to your feedback here to improve for next edition.