Comparison of key frameworks: Machine Learning (ML) and Artificial Intelligence (AI) Part - 15
by
Dr. RGS Asthana
Life Member IEEE
Summary
ML as a service as cloud service is offered by companies viz. Amazon, Microsoft, Google, and IBM. These services are being compared.
Further, this paper discusses ML and AI frameworks like CNTK from Microsoft, TensorFlow from Google and ML.Net from Microsoft. it also gives details of creating DL environment on Mac as well as on windows PC and importance of data analysis and its link to performance with respect to feature engineering. Here we also identify one very simple dataset. Emphasis is on ML.Net framework as it is new.
Prerequisite
Read articles [1] to [19]
Keywords
Machine Learning (ML) Tools, ML.Net , Artificial Intelligence (AI), Neural Networks, Internet of Things (IoT) and Deep Mind.
Prelude
ML as a service (MLaaS) [30] includes tools for data visualisation, facial recognition, natural language processing (NLP), image recognition, predictive analytics, and deep learning. The key players who offer ML/AI on the cloud [55] as a service are Amazon, Microsoft and Google through their Amazon ML, Amazon SageMaker, Microsoft and IBM [54] ML model builder respectively. Both Amazon SageMaker and google ML Engine use TensorFlow framework. We are not discussing BigML here. Although this and many more offer cloud services.
In case of IBM Watson, You will need to create an account with Bluemix to begin with. However, there’s a free 30 day trial available. After expiry of this period, you then need to choose one of the 3 available options viz. Lite, Standard, and Professional. Lite option is free under 5,000 predictions and 5 compute hours, Standard and Professional depend on actual usage of your computing hours. Predictions run $0.40 – $0.50 per 1000 predictions.
For MLaaS comparison see [32, 56] w where MLaaS offered by AmazomML, Microsoft Azure ML Studio, Google PredictionAPI [A deprecated Service now] and IBMWatsonML Model Builder are compared on classification, regression, clustering, anomaly detection, recommendation and ranking methods respectively. Although, Amazon Sagemaker and IBMWatsonML Model Builder are the best bet on model building on the cloud but the later supports methods viz. Classification and regression on the cloud. The MicrosoftAzure ML Studio, however, supports all methods described above whereas AmazonML supports only first three. In figure 1, we compare three key aspects of the NLP services i.e. features, code execution & output, and price [57, 58].
Features Amazon Google Microsoft Azure IBM Watson
Comprehand Natural Language Text Analytics NEW
Entity extraction Y Y Y Y
Key phase Extraction Y Y Y Y
Semantic Analysis Y Y Y Y
Syntax Analysis X Y Y X
Topic Modelling Y Y X X
Multiple Language Support 100 + 110+ 120 Y
Parts of speech X Y Y X
Figure 1: Feature Comparison of cloud NLP Services where 'X' means this feature is not offered and ‘Y’ means yes this feature is offered by the key players [57]
We only discuss in this paper three key frameworks viz. CNTK from Microsoft, TensorFlow from google and ML.Net from Microsoft.
we also put some emphasis on deep learning or DL. It is mainly Neural networks and clustering leading to unsupervised learning.
Convolutional Neural Network or CNN and are good for image processing as they capture and preserve topological features in the image. CNN are comprised of three types of layers viz.:
- Convolutional Layers consisting of filters and feature maps.
- Pooling Layers that down sample the activations from feature maps.
- Fully-Connected Layers at the end of model that are used to make predictions.
Although all weights are set and activation functions are decided based on the model accuracy obtained but the key limitation of neural net is that no explanation is available from the Neural Net that how it arrived at a particular decision.
Java and Python [31] are two key languages used in ML/AI computations and UI development but ML.Net enables C# to become a language of choice for ML/AI development work, particularly, as there’re lot of C# developers in the world today. Java has taken second place when compared to Python due to its massive use in data science and ML/AI.
What is ML and Deep Learning (DL) Framework?
A ML framework is, in fact, a library written in a programming language to assist in constructing ML models, train, test, and evaluate the defined model. Although without framework ML can be implemented, however, frameworks are used as they are optimised to carry out ML tasks. further, using ML frameworks saves time as they are tested and designed to enable developers to use ML easily.
There's a difference between a ML and a DL framework [21]. Whereas, ML framework may cover a variety of learning methods for classification, regression, clustering, anomaly detection, and data preparation, and it may or may not include neural network methods. A DL framework covers a variety of neural network topologies with many hidden layers.
Use of DL [22] has become synonym of accuracy and DL algorithms can outperform even humans, particularly, in classifying images (as machines can beat bare human eye on seeing, in general, and speed of scanning images for a specific pattern) and also playing Go game. We all know that GPUs [25] have played salient role in the success of DL by reducing the training time by up to a factor of 10 to 100 depending on the hardware employed during training. DL needs massive amount of data as accuracy is function of the amount of data.
DL, in fact, is a subset of ML. DL mainly uses neural networks and its called deep as it uses a number of layers. The main drawback of neural net based DL framework is that no explanation is available why certain conclusion is reached by the DL model. In order to improve the efficiency in developing new deep neural networks, many open-source deep learning toolkits have been recently developed, including Caffe from UC Berkeley [28], CNTK from Microsoft [26], TensorFlow (TF) from Google [27] and many other frameworks with similar or less capability. For achieving high-performance, these frameworks do support multi-core CPUs and many core GPUs.
What is ML and Deep Learning (DL) Framework?
A ML framework is, in fact, a library written in a programming language to assist in constructing ML models, train, test, and evaluate the defined model. Although without framework ML can be implemented, however, frameworks are used as they are optimised to carry out ML tasks. further, using ML frameworks saves time as they are tested and designed to enable developers to use ML easily.There's a difference between a ML and a DL framework [21]. Whereas, ML framework may cover a variety of learning methods for classification, regression, clustering, anomaly detection, and data preparation, and it may or may not include neural network methods. A DL framework covers a variety of neural network topologies with many hidden layers.
Use of DL [22] has become synonym of accuracy and DL algorithms can outperform even humans, particularly, in classifying images (as machines can beat bare human eye on seeing, in general, and speed of scanning images for a specific pattern) and also playing Go game. We all know that GPUs [25] have played salient role in the success of DL by reducing the training time by up to a factor of 10 to 100 depending on the hardware employed during training. DL needs massive amount of data as accuracy is function of the amount of data.
DL, in fact, is a subset of ML. DL mainly uses neural networks and its called deep as it uses a number of layers. The main drawback of neural net based DL framework is that no explanation is available why certain conclusion is reached by the DL model. In order to improve the efficiency in developing new deep neural networks, many open-source deep learning toolkits have been recently developed, including Caffe from UC Berkeley [28], CNTK from Microsoft [26], TensorFlow (TF) from Google [27] and many other frameworks with similar or less capability. For achieving high-performance, these frameworks do support multi-core CPUs and many core GPUs.
Cognitive Toolkit (CNTK) from Microsoft
CNTK is a free, easy-to-use and open-source toolkit that trains DL algorithms to learn like the human brain. It is a known Microsoft ML framework [26] allowing not only compatibility but also helps you in optimising computational resources and enables developers to play with ML models. Developers can download predefined models for certain tasks in case they are unfamiliar with the idea of ML. DL is a subset of ML that has led to innovations in the area of speech and image recognition as well as due to performance requirements has enhanced research on architectures with multiple-CPUs and GPUs.
How to install and use Microsoft CNTK is given in blog [20]. To begin with using CNTK, it is advisable to use the CNTK lab. The CNTK lab instructs a beginner to train the predefined CNTK model with MNIST dataset [50] and test the trained model with the given test dataset as well as a user given dataset. for full installation instructions see [20].
TensorFlow
It is an open source ML framework developed by the Google Brain Team its current version is 1.10 at the time of writing this paper. The TensorFlow web-site [27] gives all details required on how to get started with TensorFlow, for different language understanding and perceptual tasks.
In fact , it is an extensive library on deep neural networks. It does support new CPUs and GPUs. it is also used by Google in many of its services such as Gmail, Speech recognition, Google Photos and even Google Search.
Using KERAS APIs [38], the accuracies of TensorFlow and CNTK backends are similar across all benchmark tests of DL i.e. on all neural network models, except speed variation is much more when we compare performance [24] of TensorFlow and CNTK (a Microsoft product) on LSTM - Long short Term Memory [39] network, CNTK is about 2 to 4 times faster than TensorFlow. I
Its difficult for a sophomore to run and use TensorFlow libraries as a lot of code needs to be written. however, this difficulty can be overcome if you use KERAS[40, 41] with it. KERAS tries to make things easy for the user and keeps him in full control when needed i.e. user has flexibility to extend the source code anytime he feels like. Since, the concept of DL is easy to grasp, so KERAS makes their implementation also easy. The popularity of TensorFlow framework for ML is more than established as it has been downloaded more than 13 million times till May 2018, as per google.
macOS - High Sierra and compatible to Xcode installation for DL with Python, TensorFlow as backend, and KERAS
This tutorial [42] gives step by step installation process on configuring your development environment for DL with Python, TensorFlow as backend and KERAS. This development environment will reduce need of coding to a good extent as compared to TensorFlow only with Python.
However, when tried to install on my Mac with two cores i could not compile mainly as all versions have changed right from python onwards … since writing of [42]. The two errors i got in compiling $Make -j2 at step 6 which got aborted after compiling 85%.
I have written to the author and would come back with an addendum once I get reply from the author.
The Best Way to Install TensorFlow with GPU Support on Windows 10 (Without Installing CUDA) [43]
The aim is to get a good GPU accelerated work environment for TensorFlow with KERAS backend and Jupyter notebook[44] up and running for Windows 10 without CUDA. To be very frank I have not compiled this myself.
ML.Net
It is a open source and cross platform ML framework [29] which is a code-driven and UI driven [45, 53]. This approach enables one to introduce ML/AI in any existing application written in .net. As per Microsoft .net web-site, ML.NET V0.1 as an extensible framework, with support for Light gradient boosting machine (GBM) [33], accord.NET (ideal for scientific computing as it has libraries for apps like pattern recognition, artificial neural networks, statistical data processing, linear algebra and image processing etc.) [34], and libraries like CNTK [26], and TensorFlow [27]. Microsoft announced ML.NET 0.3 [35] recently. With this framework one can export models developed in the open neural networks exchange (ONXX) format or can develop new types of models with Factorisation Machines and LightGBM.
The key features of ML.Net 0.3 [35] are explained below:
- Export of ML.NET models to the ONNX-ML format which is an interoperable standard format for representing DL and ML models enabling developers to save trained models (from any framework) to the ONNX format
- LightGBM is added to ML.Net. LightGBM is a framework that basically helps binary classification, Multi-class Classification or predict a value based historic data (regression) {see Figure 2}.
GBM [33] is, in fact, a high-performance gradient boosting framework based on decision tree algorithms.
Figure 2. [35]
- Added multiple learners in model {see Figure 3} and
Figure 3. [35]
- Added LightLDA transform for topic modelling [37] - see example on sentiment analysis {also see Figure 4}.
Figure 4. [35]
in the month of August 2018, Microsoft announced version 0-4 [51]. In this release Microsoft has introduced a few important features such as improved support for natural language processing (NLP) by adding the Word Embedding Transform- which replaces a word by a number or numeric vector to be precise, Keeping its meaning to a limited extent it has improved performance of sentiment analysis by about 5%. The changes in program are as given below:
// Change TextFeaturizer to output tokens (list of words in the text)
pipeline.Add(new TextFeaturizer("FeaturesA", "SentimentText") { OutputTokens = true});
// Add word embeddings
pipeline.Add(new WordEmbeddings(("Features_TransformedText”, “FeaturesB”)));
// Combine the features from word embeddings and text featurizer into one column
pipeline.Add(new ColumnConcatenator("Features", "FeaturesA", "FeaturesB"));
In version 0.4, Microsoft has introduced clustering support and parallel Stochastic Gradient Descent (SGD) algorithm called SymSGD that not only retains the sequential semantics of SGD but also offers a better performance by enabling multithreading. SymSGD is now available for binary classification. SGD is a effective method in ML such as regression and classification. Here’s how you add a SymSGD Binary Classifier learner to the pipeline:
pipeline.Add(new SymSgdBinaryClassifier() { NumberOfThreads = 1});
pipeline.Add(new SymSgdBinaryClassifier() { NumberOfThreads = 1});
Further, ML.NET in its 0.4 Version supports use of property-based row classes in F# [52]. Microsoft has also updated the dot.net ML samples even for F#.
Data Analysis and its link to performance
The performance of a model which is its accuracy is linked to quality of feature engineering one does on the data like removing outlier data i.e. identifying those values in dataset that are out of overall pattern in the data and replacing missing values and many other operations suitable for making data appropriate for processing. For example, If we do not need a column in the dataset we can drop it. In ML.Net, it can be done by adding ColumnDropper in the LearningPipeline like:
pipeline.Add(new ColumnDropper() { Column = ‘NameOfTheFeature’});
In ML.Net, missing values are detected by adding MissingValueIndicator class to the pipeline. This class creates a boolean output column with the same number of slots as the input column, where the output value is true if the value in the input column is missing.
ML.Net framework does not have a nice way to detect outliers like we have in Python, where we can use Box-plot, Histogram or Scatter Plot. Counting number of occurrences of data is one way to clean data.
The science and art of making a set of data more useful is called feature engineering. In the field of pattern recognition, this step is referred to as pre-processing. However, feature engineering can do more operations on the dataset then only pre-processing.
MNIST dataset [50] is widely used as it’s very simple dataset. It comprises of 60,000 training images and 10,000 test images of handwritten digits from 0 to 9. Each image is of size 28*28. All digits in the dataset have been size-normalised and centred. It is a subset of a larger set available from NIST. We convert a number in a form of data (in this case, form of a row vector) that has all elements 0 except the position corresponds to the number, which would be 1. For example, 2 will be converted to 0000000100 and 0 will be converted to 0000000001 as there are only 10 possible digits i.e 0…9.
Every image is converted in a long row vector (1X784). This process is called Flattening or vectorising. This step enables batch training since many row vectors can fie used to form a matrix. Thus, a simple matrix multiplication does the trick.
Way forward
In fact, KERAS runs on top of TensorFlow and CNTK and reduces need of coding to a great extent, particularly, with TensorFlow. Use KERAS if you need a DL library that:
- One can use both CPUs and GPUs in any configuration even a combo of both,
- One can quickly prototype a model and view it as a arbitrary graphs of layers, and
- Use convolutional as well as recurrent networks, or even use both networks in one model.
The ML tutorials [46], one can use ML.Net into existing .Net applications or developing custom ML solutions:
- Sentiment analysis [37]: depicts how to apply a binary classification - when you choose between A or B [53] {see figure 2} - using ML.Net 0.3/0.4 and find the difference in performance in terms of accuracy.
- Taxi fare predictor [47] shows how to apply a regression task - how much or how many or we attempt to quantify [53] - using ML.NET.
- Iris clustering [48] shows how to apply a clustering task - i.e. grouping of similar data points in one cluster - using ML.Net 0.3/0.4 and find the difference.
Today Python is the de facto language used for ML/AI development. Hopefully, one day Microsoft will be more thoughtful so that, one as .Net developer can learn and use ML.Net easily, particularly, use API’s already developed in .Net say in C#. ML.Net is basically used internally by Microsoft for a long time as they can easily leverage all the existing APIs and ML/AI libraries like CNTK but now for the first time Microsoft has provided this framework externally. To make this step successful it is necessary that Microsoft puts all effort and also uses all of the experience of the company to make ML.Net rich for the .Net developer. The other advantage of ML.Net is to improve accuracy of any ML/AI model by providing more data.
Now we explain, How to create a learning pipeline?
step1: declare pipeline
var pipeline = new LearningPipeline();
step 2 load data
pipelineAdd( new TextLoader<TaxiTrip>
( Datapath, UseHeader: true, Separator: ‘,’));
step3: vectorise i.e. everything is converted to numbers
pipeline.Add( new CategoricalOnehotVectorizer;
(‘Vendor_id’,
‘rate_code’,
‘payment_type’));
step4: Concatenate - only keep those column of data which are needed for the algorithm
pipelineAdd( new ColumnConcatenator(“Features”, “vendor_id”, “rate_code”, -);
step 5: choose learning algorithm -e.g. regression, classification and clustering as an example we can take taxi fare prediction i.e. a regression problem
pipelineAdd( new FastTreeRegressor());
step 6: train your model
pipeline.Train<TaxiTrip, TaxiTripFarePrediction>();
Microsoft also wishes to develop a simple UI which will automate the above process, thus, reducing the need for coding as the code will be generated automatically, hopefully for all type of learning models. Next Steps
Data exploration and visualisation seem to be the areas in which ML.NET needs improvement. Python has better data exploration capabilities and a better approach for data visualisation [49] as compared to .net. It will be fascinating to see what Microsoft will come up in future versions of ML.Net.
Microsoft also needs to put some effort for preprocessing or feature engineering in the model building UI.
However, ML.Net is code- and UI- driven and all tools and API’s of .net framework are available making development easy, particularly, for .Net developer using C#.
References
[1] Progress and Perils of Artificial Intelligence (AI)
[2] Invited Chapter 6 - Evolutionary Algorithms and Neural Networks, Pages 111-136, R.G.S. Asthana in book, Soft Computing and Intelligent Systems (Theory and Applications), Academic Press Series in Engineering, Edited by Naresh K. Sinha, Madan M. Gupta and L.A. Zadeh ISBN: 978-0-12-646490-0
[3] Future 2030 by Dr RGS Asthana
[4] Machine Learning (ML) and Artificial Intelligence (AI) – Part 1, by Dr. RGS Asthana, Senior Member IEEE
[5] Machine Learning (ML) and Artificial Intelligence (AI) – Part Two, by Dr. RGS Asthana, Senior Member IEEE
[6] Machine Learning (ML) and Artificial Intelligence (AI): Cognitive Services and Robotics – Part Three by Dr. RGS Asthana, Senior Member IEEE
[7] Machine Learning (ML) and Artificial Intelligence (AI): Big Data and 3 D Printing – Part four by Dr. RGS Asthana, Senior Member, IEEE.
[8] Machine Learning (ML) and Artificial Intelligence (AI): Drones and Self-driving Cars– Part Five by, Dr. RGS Asthana, Senior Member IEEE
[9] Machine Learning (ML) and Artificial Intelligence (AI): Healthcare– Part Six by, Dr. RGS Asthana, Senior Member IEEE
[10] Machine Learning (ML) and Artificial Intelligence (AI): Will AI/ML intelligence surpass humans? Part Seven by Dr. RGS Asthana, Senior Member IEEE
[11] Machine Learning (ML) and Artificial Intelligence (AI): Impact of AI/ML in Healthcare: Part-Eight by Dr. RGS Asthana, Senior Member IEEE
[12] Machine Learning (ML) and Artificial Intelligence (AI): Big data & Data Science (DS) and their importance: Part-Nine by Dr. RGS Asthana, Senior Member IEEE
[13] Machine Learning (ML) and Artificial Intelligence (AI): Super-Intelligence - Are we afraid? Part-ten; by Dr. RGS Asthana, Senior Member IEEE.
[14] Machine Learning (ML) and Artificial Intelligence (AI): ML Algorithms: Part- Eleven
[15] Machine Learning (ML) and Artificial Intelligence (AI): Prominent ML & AI applications including those on Mobile devices: Part - Twelve
[16] Robotics advances with Machine Learning (ML) and Artificial Intelligence (AI) and its impact on healthcare Part - 13
http://newblogrgs10.blogspot.com/2018/04/robotics-advances-with-machine-learning.html
[17] Deep mind website
[18] IBM Watson Website
[19] Internet of Things (IoT)
[20] First impressions on the CNTK and a comparison with Google’s TensorFlow.
[21] Review: The best frameworks for machine learning and deep learning
[22] Introducing Deep Learning with MATLAB
[23] Comparison of deep learning software
[24] How do you compare (Microsoft) CNTK and (Google) Tensorflow? Does one hold a clear advantage over the other?
[25] Benchmarking State-of-the-art DL software tools
[26] The Microsoft cognitive toolkit
[27] Tensorflow website
[28] Caffe website
[29]Microsoft .NET website
[30] Top 5 Machine Learning-as-a-Service providers
[31] Python’s growth comes from the enormous expansion of data science and machine learning
[32] Comparing Machine Learning as a Service: Amazon, Microsoft Azure, Google Cloud AI, IBM Watson
[33] Gradient Boosting
[34] Accord.NET
[35] Announcing ML.NET 0.3
[36] FieldAwareFactorizationMachineTrainer Class
https://docs.microsoft.com/en-us/dotnet/api/microsoft.ml.runtime.factorizationmachine.fieldawarefactorizationmachinetrainer?view=ml-dotnet
[37] Tutorial: Use ML.NET in a sentiment analysis binary classification scenario
[38] Keras functional API
https://www.packtpub.com/mapt/book/big_data_and_business_intelligence/9781787128422/7/ch07lvl1sec49/keras-functional-apiutm_source=google&utm_medium=CPC&utm_campaign=dynamic_ads_august_search_pages?gclid=Cj0KCQjw5NnbBRDaARIsAJP-YR9TxJDEPcgJcLrTCrVQjXtYnYO1ZEDfkq-ceXjAyBunkfLRJo47_jkaAsI1EALw_wcB
[39] Recurrent Layers - Keras documentation
[40] Keras Tutorial: The Ultimate Beginner’s Guide to Deep Learning in Python
[41] Keras
[42] macOS for deep learning with Python, TensorFlow, and Keras
[43] The Best Way to Install TensorFlow with GPU Support on Windows 10
(Without Installing CUDA)
[44] Jupyter Notebook Tutorial: The Definitive Guide
[45] ml.net introduction
[46] ML .Net Tutorials
[47] Tutorial: Use ML.NET to predict New York taxi fares (regression)
[48] Tutorial: Use ML.NET to cluster iris flowers (clustering)
[49] Machine Learning with ML.NET – Comparing Data Exploration in Python with Data Exploration in ML.NET
[50] The MNIST Database of handwritten digits
[51] Announcing ML.NET 0.4
[52] F# website
[53] Introducing ML.NET : Build 2018
[54] A faster, more secure journey to cloud
https://www.ibm.com/cloud/why-ibm/?leadspace=anthem&S_PKG=AW&cm_mmc=Search_Google-_-Corporate+Advertising_Pillars-_-NA_US-_-+ibm++cloud_Broad_AW&cm_mmca1=000027JN&cm_mmca2=10006691&cm_mmca7=9012939&cm_mmca8=kwd-301973236515&cm_mmca9=d5372a0f-e9d6-4b6f-a0f6-b009eada551f&cm_mmca10=291864992307&cm_mmca11=b&mkwid=d5372a0f-e9d6-4b6f-a0f6-b009eada551f|447|135522&cvosrc=ppc.google.%2Bibm%20%2Bcloud&cvo_campaign=000027JN&cvo_crid=291864992307&Matchtype=b&gclid=CjwKCAjwt7PcBRBbEiwAfwfVGA1LCJKJg9bro0WDq7fd8Aa-7NXG5_1IcsLbF_Uj7b7w4PPrHA6oRBoCdI4QAvD_Bw
[55] Top cloud providers 2018: How AWS, Microsoft, Google Cloud Platform, IBM Cloud, Oracle, Alibaba stack up
[56] A quick comparison of Machine Learning platforms of Amazon, Microsoft, Google and IBM
[57] Comparing Machine Learning (ML) Services from Various Cloud ML Service Providers
mmorpg oyunlar
ReplyDeleteinstagram takipçi satın al
TİKTOK JETON HİLESİ
TİKTOK JETON HİLESİ
antalya saç ekimi
referans kimliği nedir
İNSTAGRAM TAKİPÇİ SATIN AL
metin2 pvp serverlar
TAKİPCİ