Machine Learning (ML) and Artificial Intelligence (AI): Big data & Data Science (DS) and their importance: Part-Nine by Dr. RGS Asthana Senior Member IEEE

Machine Learning (ML) and Artificial Intelligence     (AI): Big data & Data Science (DS) and their importance: Part-Nine

by

Dr. RGS Asthana

Senior Member IEEE

Figure 1: Machine Learning [20]

Summary

Big Data Analytics is a very well established technology. There are many tools and software in the market to handle Big Data.  AI and machine learning are dependent on large amounts of data. But big data is hard to organize and analyze.  ML is a technology which can work on Big Data to find hidden patterns which is almost impossible for human to find merely due to data being vast. However, big data and ML/AI congregation can produce interesting results for organizations.

We define big data, Data Science (DS) and ML/AI and their role and importance in solving an organization’s problems. A list of general problems which can be solved using ML/AI is also given. The key areas of Big Data, DS and ML/AI and what do they mean? Way forward provides what we expect in the near future?

Prerequisite

Read article [1] to [14]

Keywords

Machine Learning (ML) Tools, Artificial Intelligence (AI), Neural Networks, Internet of Things (IoT), Data Science (DS), Deep Mind, IBM’s Watson

Prelude

AI refers to approaches to not only developing systems that not only do intelligent things but also can learn from experience. ML is subset of AI.  The meeting of big data with ML/AI [21] has given us an opportunity to handle a lot of data which is partially made available for research in anonymized form and ML algorithms have capability to build predictive models easily. This approach has empowered companies to drive business value from their data and analytics capabilities.   Earlier limitations of data availability, limited sample sizes, and an inability to analyze massive amounts of data in real-time is now overcome due to availability of these technologies  

However, first major task is that this data is raw and needs to be pre-processed.  The second major task is to combine various databases such as shopping behavior, daily exercise and diet info, maybe personal genomes and also occasional blood test data. Pathway Genomics [15, 16] – an IBM backed company - is developing a simple blood test to determine if early prediction of certain cancers is possible.  This company produces actionable and accurate genetic information for physicians and as well as their patients to sustain health and wellness. It’s AI and deep learning empowered mobile health applications use personal genetic information to give personalized health and wellness guidance to the users.

Big Data [17] - Big data may comprise of the collection of structured as well as unstructured data. The main sources of data are social sites, YouTube and even online commerce site having a lot of data about their customers.  In fact, the big data may include information of employees, company purchase, sale records, business transactions, the previous record of organizations, social media [22] etc.

Typically, big data is defined by the factors, such as, Volume, Velocity, Variety, Complexity and Variability.

Volume: Data stored is both structured and unstructured and with the cost of storage being negligible, the collected data can also now be stored.  Cloud technologies are pretty developed now and play a major role in this.

Velocity:   Smart sensors, smart metering and RFID tags make it necessary to deal with huge data influx in almost real-time.  

Variety: The data may be structured or unstructured e.g. texts to emails, video, audio and financial transactions. In fact data comes in every conceivable form.  

Complexity:  It is necessary to deduce a connection and hierarchy between multiple data sets. The lack of such organization will cause torrents of incoming data to flood your system and memory without any valid utility.

Variability:  As the data sources vary the data flow rates also vary.  Unstructured data gives very inconsistent data flow.  Big data may have multiple data sources, such as Social Media, streaming unstructured data and/or from public sources: e.g. the CIA World, Facebook and/or European Union Open Data Portal are the most common ones. Therefore, traditional data storage like RDBMS is not suited to handle such huge quantities of data. Therefore, a different storage and computing paradigm to handle big data is necessary.  Hadoop, Spark and NoSQL are examples of such tools.

Data now resides in silos where accountability, focus, and its mission are clear. Server-less, micro-service architectures are making it  easy to access, analyze, and manage data without racking servers, configuring virtual machines, or even paying by the hour. Going server-less allows data owners to focus on their data application and pay just for use by the minute and not by hours as earlier.

DS (DS) – DS, also known as data-driven science [34], is an interdisciplinary field about scientific methods, processes, and systems to extract knowledge or insights from data in various forms, either structured or unstructured. It is a broad discipline similar to Data Mining and overlaps with fields such as statistics, IoT [14], operations research, applied mathematics and also ML/AI & deep learning. Dealing with unstructured and structured data, DS is a field that includes anything related to data cleansing, preparation, and analysis. As in any fields, data scientists may also use appropriate techniques and  algorithms from other fields to handle very large unstructured data sets in automated ways i.e. without Human-in-the-loop (HITL) [25] not only to perform transactions in real-time but also to make predictions.

DS, in fact, is much wider discipline than ML. In DS, data may or may not originate from a machine automatically like survey data is manually collected and clinical trials may be only a specific type of small data. In particular, DS covers aspects such as data integration, distributed architecture, automating ML, data visualization, dashboards & BI and automated data-driven decisions [33].

Analytics will also improve as more and more data is available and better algorithms are developed. In order to feed proper training data into ML systems, it must first be cleansed or pre-processed i.e. all errors in format, duplications, etc. are fixed. Making sense of data or making it useful to enable solve a specific problem is job of the data scientist.

ML/AI – The ML/ AI success depends on data labeling and analysis. In brief, ML [20] is an application of AI (see figure 1) that provides systems the ability to automatically learn and improve from experience without being explicitly programmed.  ML in other words is a data analysis method to program extrapolated model building algorithm which in turn is a set of instructions. ML, in fact, offers a deeper insight into collected data and helps find hidden patterns.  The main goal of ML when used on big data is to identify useful patterns in it and ML algorithms predict future patterns too.  ML/AI applications based mainly on ML depend on data to develop more predictive models. 

ML can be applied to smaller datasets also; however the outcome may not be very accurate as learning chance is low. ML over the years has evolved from simple analytical algorithms to automatic application of customized algorithms and mathematical calculations to big data. The speed and iteration are unique qualities of ML.

We expect by 2020, there will be about 44 zeta bytes of data, according to IDC [30]. Here, ML/AI may play a big role as it puts an intelligence layer to big data.   Thus, complex analytical tasks can be handled much faster than humans can do.  In fact, ML/AI may take only routine jobs but as per an optimistic view there will always be work for good intellectuals, irrespective of how smart machines get.

IoT [14] will also swell with introduction of new toys, car accessories, home and security practice but will also open access for hackers and this could become cause for worry. In Deep Learning you do not tell machines what to do but machines based on the data determine what to do. This approach got Stephen Hawking and Elon Musk worried [1] — as the idea eventually may lead to development of thinking machines that will eventually surpass humans, and due to vast connectivity may take over the world, and threaten our very existence. Facebook recently [26] shut down AI bots based app where bots started ignoring the set programmed instructions and began speaking in a language unintelligible to humans instead of English which was supposed to be set method for communication. 

With advances in data processing and cloud applications, every platform is becoming cloud-available.  There are also a number of ML platforms offered by tech companies, e.g., Baidu, Amazon, Microsoft, IBM, Google, and many others.  Baidu research [32] works on ML/AI technologies in areas such as image recognition, speech recognition, high performance computing,   natural language processing (NLP) and deep learning

Problems ML/AI can solve  

In brief, the certain problems which can be solved through ML/AI (see figure 2) are given below:

·   Automate Repetitive Tasks

·   Fraud detection

·   Recommendation Systems
• Real-time ads on mobile devices

·   Credit scoring and best offer ads

·   WebPages

·   Web search results

·   New pricing models

·   Email spam filtering

·   Prediction of equipment failure

·   Detection of network intrusion(s)

·   Image recognition

·   Pattern recognition

·   Speech processing

·   Sentiment analysis from texts

·   Self-Driving Cars (SDCs) - can analyze data collected by numerous cameras mounted on SDCs thus vehicles are able to read their environment in real-time and react, much like a human driver, making trip not only faster and safer but also more efficient.

Figure 2: AI and intelligence [22]

·   Healthcare: Analyzing vast amount of medical data coming from number of locations and demographics will determine which conditions will improve the effectiveness of certain treatments and which do not.

·         Natural Language Processing (NLP)

·         Machine Translation

·         Fraud Detection

·         Financial Management and investment planning

Congregating Big Data, DS and ML/AI

The convergence of big data and ML/AI has changed how businesses value their data and use it to stay ahead of their competitors.  As the importance of big data further matures in the coming years, particularly, as ML/AI is   implemented at a rapid pace to take over laborious, routine, risky and mission-oriented jobs and to free employees to do not only more human jobs functions but also concentrate on the development of new profitable schemes for the business.

Although big data analytics is an established and independent field but Both Big Data and ML/AI are influential on their own [18]. Think of a scenario, what happens when the big data and ML/AI are merged? The availability of greater volumes and sources of data will enable capabilities in ML/AI which existed due to lack of data availability, limited sample sizes, and an inability to analyze massive amounts of data in almost real time. 

Big data empowers data scientists to access and work with massive sets of data without restriction and hesitation. Organizations can now load all of the data and let the data itself point the direction and tell the story.  Big data enables an environment that encourages data discovery through iteration.

Big Data analytics is used at Intermountain Healthcare [19] to obtain better health outcomes, thus, achieving improvement in its health services, particularly, in the area of cardiovascular, endocrinology, surgery, and other areas. The company has not only lowered its infection rates but also lowered its operating as well as total costs. Think of benefits if this company uses ML/AI on big data.

Role of ML and Big Data, DS in an organization

The role of ML/AI methods is to map to the big data or DS methods deployed in an organization to fulfill its needs. Most common ML/AI methods include supervised learning: which is generally predictive, unsupervised learning– which is generally exploratory and may deploy clustering and associated algorithms as the main tool; semi-supervised:  HITL [25], deep learning: mainly works on gradient descent methods which determine the regularities – the common clusters of visible (activated) or invisible(deactivated) pixels across multiple training images and neural nets with hidden layers and back propagation and finally reinforcement learning: which is based on award of penalties and rewards given based on the actual outcomes for the application of the negative (so called) or positive policies but it is highly computer power intensive. Q-learning - a model-free  technique used as a reinforced learning - can help find an optimal action-selection policy for any finite Markov decision process (MDP) [24].

Every business organization today needs someone like a data analyst or a data scientist to look after data [36]. This person should be capable of really pulling out trends from data. Therefore, strong business acumen and communication ability is required in a data scientist/analyst as he needs to communicate with business and IT leaders in a way that can influence how an organization approaches a business challenge. 

Infosys Nia [27] - Company’s next-generation AI platform combines technologies like big data/analytics, ML/AI, knowledge management, and cognitive automation capabilities of Mana - first-generation AI platform. The offering end-to-end Robotic process Automation (RPA) capabilities of AssistEdge [28]; advanced, high-performance and o build more accurate models in a shorter amount of time, you need high performance algorithms and to utilize all of your data. The Skytree [29] platform is built for algorithmic speed and scales to perform ML on massive amounts of data – structured and/or unstructured; and optical character recognition (OCR), natural language processing (NLP) capabilities and infrastructure management services. Supply chain as well as manufacturing [31] nowadays use machines and robotics that can think and function autonomously, or with minimal supervised programming, to perform specific actions.

Big data, DS and ML/AI

Big data and DS lets ML evolve and adjust to the everyday requirements of data analysis that an organization [22] needs. ML is not limited by human thinking or by speed and thus is able to mine valuable data from big data without any bias. ML is able to find patterns in the chunks of unstructured data, collected from everyday activities and transactions and make big data useful for the organization.

Figure 3: Success of DS Solutions: A Venn Diagram showing relationship between DS, IT and Business Skills [33]

Venn diagram in figure 3 shows the relationship between fields of DS, IT and Business Skills.

Eighty-eight percent of Data Scientists have a Master’s Degree, and 46% have PhDs [35]. Other skills data scientists need include:

·  In-depth knowledge of SAS and/or R: For DS, R is generally preferred.

·  Python coding: Python is the most common coding language that is used in DS along with Java, Perl, C/C++.

·  Hadoop platform: Although not always a requirement, knowing the Hadoop platform is still preferred for the field. Experience in Hive or Pig is a huge plus.

·  SQL database/coding: Though NoSQL and Hadoop are the major focus for data scientists; preferred candidates can write and execute complex queries in SQL.

·  Working with unstructured data: ablity to work with unstructured data say, from social media, video feeds like youtube, audio, or other sources.

·  Typical salary of Data scientist is USD 120000+.

For Big Data roles following skills may be needed:

·  Analytical skills: The ability to make sense of the enormous amounts of data. 

·  Creativity:   Ability to create new methods to gather, interpret, and analyze a data strategy.

·  Mathematics and statistical skills: Good, old fashioned “number crunching” is necessary.

·  Computer science:   Programming experience is preferred as professional may be required to develop algorithms.

·  Business skills: Big Data professionals should have an understanding of the business objectives that are in place and must have ideas about how to increase the enterprise profits.

Most jobs that specifically have "ML" in the title seem to be looking for CS people with some experience in ML.  "Data scientist" jobs seem to fall into one of two categories: (1) "data analyst" jobs that are looking for people with some background in data analysis, often looking for R/SAS/SPSS.   (2) "Computational statistician" - Python and databases experience with good statistics background.

Way forward

As, oxygen is vital to existence of living organisms so is the availability of vast amount of data or Big Data is vital for the success of the ML/AI algorithms. More the data better the results and their accuracy.  ML /AI algorithms derive patterns from big data which a human cannot locate. With vast amount of success, ML is perceived as the new management tool and companies like Coca Cola, MetLife and Netflix are using Big Data Analytics for long time and also benefitting monetarily from it. Scaled up algorithms, such as, recurrent neural networks and deep learning are powering the breakthrough of ML/AI [21].

At present ML/AI algorithm reach a conclusion without a valid explanation, work has to be done that ML/AI algorithm have a valid explanation for each action they suggest, this will improve adaptability of ML/AI in many vital fields like healthcare.

Congregating big data, DS and ML/AI may improve business performance as well as our life we, however, need real people in real-world companies to help in development of right combination of technologies onto useful platforms.

Cloud computing, Big Data and DS are enabling technologies and riding on these ML/AI has been galloping at a tremendous pace in last few years and may reach state of General AI - which can perform any intellectual task that a human can do - sooner than expected.  Use of Big Data and DS techs enables seamless capabilities in ML/ AI [38] that remained dormant for decades due to lack of data availability, limited sample sizes, and an inability to analyze massive amounts of data in almost real-time.  With the global market of AI expected to grow at 36% annually, reaching a valuation of $3 trillion by 2025 from $126 billion in 2015 [37].   Big Data and DS are more or less independent techs and have strength of their own. DS allows cleansing of data and prepares it for any desired application.  The meeting of big data & DS with AI has emerged as the single most important development that is shaping the future of how organizations are getting motivated to derive greater business value from their data and analytics capabilities.   

References

[1] Progress and Perils of Artificial Intelligence (AI) 

http://newblogrgs10.blogspot.in/2017/04/progress-and-perils-of-artificial_5.html

[2] Invited Chapter 6 - Evolutionary Algorithms and Neural Networks, Pages 111-136, R.G.S. Asthana in book, Soft Computing and Intelligent Systems (Theory and Applications), Academic Press Series in Engineering, Edited by:Naresh K. Sinha, Madan M. Gupta and Lotfi A. Zadeh ISBN: 978-0-12-646490-0

http://www.sciencedirect.com/science/book/9780126464900

[3] Future 2030 by Dr. RGS Asthana, Senior Member IEEE

https://www.linkedin.com/pulse/future-2030-dr-rgs-asthana-senior-member-ieee-r-g-s-asthana

[4] Machine Learning (ML) and Artificial Intelligence (AI) – Part 1, by Dr. RGS Asthana, Senior Member IEEE

https://plus.google.com/113904112609362672954/posts/4ti2gyQVPZ7

[5] Machine Learning (ML) and Artificial Intelligence (AI) – Part Two, by Dr. RGS Asthana, Senior Member IEEE

https://plus.google.com/113904112609362672954/posts/X5wB7baJLwt

[6] Machine Learning (ML) and Artificial Intelligence (AI): Cognitive Services and Robotics – Part Three by Dr. RGS Asthana, Senior Member IEEE

https://plus.google.com/113904112609362672954/posts/EA9hZcY6AwL

[7] Machine Learning (ML) and Artificial Intelligence (AI):  Big Data and 3 D Printing – Part four by Dr. RGS Asthana, Senior Member, IEEE.

https://plus.google.com/113904112609362672954/posts/agAa6pY9ekA

[8] Machine Learning (ML) and Artificial Intelligence (AI):  Drones and Self-driving Cars– Part Five by, Dr. RGS Asthana, Senior Member IEEE

http://newblogrgs10.blogspot.in/2017/05/machine-learning-ml-and-artificial_17.html

[9] Machine Learning (ML) and Artificial Intelligence (AI): Healthcare– Part Six by, Dr. RGS Asthana, Senior Member IEEE

http://newblogrgs10.blogspot.com/2017/05/machine-learning-ml-and-artificial_26.html

[10] Machine Learning (ML) and Artificial Intelligence     (AI):  Will AI/ML intelligence surpass humans? Part Seven by Dr. RGS Asthana, Senior Member IEEE

http://newblogrgs10.blogspot.in/2017/06/machine-learning-ml-and-artificial.html

[11] Machine Learning (ML) and Artificial Intelligence     (AI): Impact of AI/ML in Healthcare: Part-Eight by Dr. RGS Asthana, Senior Member IEEE

http://newblogrgs10.blogspot.in/2017/07/machine-learning-ml-and-artificial.html

[12] Deep mind website

https://deepmind.com/

[13] IBM Watson Website

https://www.ibm.com/watson/

[14] Internet of Things (IoT)

http://www.cisco.com/c/en_in/solutions/internet-of-things/overview.html

[15] How ML, Big Data and AI are changing healthcare forever

https://www.forbes.com/sites/bernardmarr/2016/09/23/how-machine-learning-big-data-and-ai-are-changing-healthcare-forever/#741f90ac1a1c

[16] Pathway Genomics pathway.com
https://www.cbinsights.com/company/pathway-genomics

[17] What is difference between Big Data and Machine Learning?

https://www.quora.com/What-is-difference-between-Big-Data-and-Machine-Learning

[18] How Big Data Is Empowering AI and Machine Learning at Scale

http://sloanreview.mit.edu/article/how-big-data-is-empowering-ai-and-machine-learning-at-scale/

[19] Using Data to Improve Health Outcomes

https://intermountainhealthcare.org/news/2015/07/using-data-to-improve-health-outcomes/

[20] What is Machine Learning? A definition

http://www.expertsystem.com/machine-learning-definition/

[21] How Big Data Is Empowering AI and Machine Learning at Scale

http://sloanreview.mit.edu/article/how-big-data-is-empowering-ai-and-machine-learning-at-scale/

[22] Overview of Artificial Intelligence and Role of Natural Language Processing in Big Data

https://www.xenonstack.com/blog/overview-of-artificial-intelligence-and-role-of-natural-language-processing-in-big-data

[23] 5 big data trends that will shape AI in 2017

http://www.techrepublic.com/article/5-big-data-trends-that-will-shape-ai-in-2017/

[24] Q-learning

https://en.wikipedia.org/wiki/Q-learning

[25] Human-in-the-loop

https://en.wikipedia.org/wiki/Human-in-the-loop

[26] Facebook researchers shut down AI bots that started speaking in a language unintelligible to humans

http://www.firstpost.com/india/facebook-researchers-shut-down-ai-bots-that-started-speaking-in-a-language-unintelligible-to-humans-3876197.html

[27] Infosys Launches Infosys Nia™ - The Next Generation Integrated Artificial Intelligence Platform

https://www.infosys.com/newsroom/press-releases/Pages/nia-artificial-intelligence-enterprise.aspx

[28] Edgeverve website

https://www.edgeverve.com/assistedge/robotic-process-automation/[29] [29] Skytree website

http://www.skytree.net/

[30] Why Big Data and AI Need Each Other -- and You need them both

https://www.forbes.com/forbes/welcome/?toURL=https://www.forbes.com/sites/homaycotte/2014/12/16/why-big-data-and-ai-need-each-other-and-you-need-them-both/&refURL=https://www.google.co.in/&referrer=https://www.google.co.in/

[31] How Big Data Drives AI

https://dzone.com/articles/how-big-data-drives-ai

[32] About Baidu Research

http://research.baidu.com/

[33] Difference between ML, DS, AI, Deep Learning, and Statistics

http://www.datasciencecentral.com/profiles/blogs/difference-between-machine-learning-data-science-ai-deep-learning

[34] DS

https://en.wikipedia.org/wiki/Data_science

[35] DS vs. Big Data vs. Data Analytics

https://www.simplilearn.com/data-science-vs-big-data-vs-data-analytics-article

[36] What is the role of DS for Big Data

https://www.quora.com/What-is-the-role-of-data-science-for-Big-data’

[37] Hindustan Times dated Aug. 16, 2017 page 12: Why we should not be afraid of AI

http://paper.hindustantimes.com/epaper/viewer.aspx

[38] How Big Data is empowering AI and ML at scale

http://sloanreview.mit.edu/article/how-big-data-is-empowering-ai-and-machine-learning-at-scale/