Machine Learning and AI

In this section we will begin to explore the basic principles of machine learning. Machine Learning is about building programs with tunable parameters(typically an array of floating point values) that are adjusted automatically so as to improve their behavior by adapting to previously seen data.

Machine Learning can be considered a subfield of Artificial Intelligence since those algorithms can be seen as building blocks to make computers learn to behave more intelligently by somehow generalizing rather that just storing and retrieving data items like a database system would do.

A very simple example of a machine learning task can be seen in the following figure: it shows a collection of two-dimensional data, colored according to two different class labels. A classification algorithm is used to draw a dividing boundary between the two clusters of points:

Example Linear Decision Boundary

As with all figures in this tutorial, the above image has a hyper-link to the python source code which is used to generate it.

 Features and feature extraction

Most machine learning algorithms implemented in scikit-learn expect a numpy array as input X. The expected shape of X is (n_samples, n_features).

n_samples:	The number of samples: each sample is an item to process (e.g. classify). A sample can be a document, a picture, a sound, a video, a row in database or CSV file, or whatever you can describe with a fixed set of quantitative traits.
n_features:	The number of features or distinct traits that can be used to describe each item in a quantitative manner.

The number of features must be fixed in advance. However it can be very high dimensional (e.g. millions of features) with most of them being zeros for a given sample. In this case we may use scipy.sparse matrices instead of numpy arrays so as to make the data fit in memory.

 A simple example: the iris dataset

Note

The information in this section is available in an interactive notebook 01_datasets.ipynb, which can be viewed using iPython notebook. An online static view can be seen here.

The machine learning community often uses a simple flowers database where each row in the database (or CSV file) is a set of measurements of an individual iris flower. Each sample in this dataset is described by 4 features and can belong to one of the target classes:

Features in the Iris dataset:
	sepal length in cm sepal width in cm petal length in cm petal width in cm
Target classes to predict:
	Iris Setosa Iris Versicolour Iris Virginica

scikit-learn embeds a copy of the iris CSV file along with a helper function to load it into numpy arrays:

>>> from sklearn.datasets import load_iris
>>> iris = load_iris()

The features of each sample flower are stored in the data attribute of the dataset:

>>> n_samples, n_features = iris.data.shape

>>> n_samples
150

>>> n_features
4

>>> iris.data[0]
array([ 5.1,  3.5,  1.4,  0.2])

The information about the class of each sample is stored in the target attribute of the dataset:

>>> len(iris.target) == n_samples
True

>>> iris.target
array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
       0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
       1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
       1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
       2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
       2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2])

The names of the classes are stored in the last attribute, namely target_names:

>>> list(iris.target_names)
['setosa', 'versicolor', 'virginica']

 Handling categorical features

Sometimes people describe samples with categorical descriptors that have no obvious numerical representation. For instance assume that each flower is further described by a color name among a fixed list of color names:

color in ['purple', 'blue', 'red']

The simple way to turn this categorical feature into numerical features suitable for machine learning is to create new features for each distinct color name that can be valued to 1.0 if the category is matching or 0.0 if not.

The enriched iris feature set would hence be in this case:

0. sepal length in cm
1. sepal width in cm
2. petal length in cm
3. petal width in cm
4. color#purple (1.0 or 0.0)
5. color#blue (1.0 or 0.0)
6. color#red (1.0 or 0.0)

 Extracting features from unstructured data

The previous example deals with features that are readily available in a structured dataset with rows and columns of numerical or categorical values.

However, most of the produced data is not readily available in a structured representation such as SQL, CSV, XML, JSON or RDF.

Here is an overview of strategies to turn unstructed data items into arrays of numerical features.

Text documents:	Count the frequency of each word or pair of consecutive words in each document. This approach is called Bag of Words Note: we include other file formats such as HTML and PDF in this category: an ad-hoc preprocessing step is required to extract the plain text in UTF-8 encoding for instance.
Images:	Rescale the picture to a fixed size and take all the raw pixels values (with or without luminosity normalization) Take some transformation of the signal (gradients in each pixel, wavelets transforms...) Compute the Euclidean, Manhattan or cosine similarities of the sample to a set reference prototype imagesaranged in a code book. The code book may have been previously extracted from the same dataset using an unsupervised learning algorithm on the raw pixel signal. Each feature value is the distance to one element of the code book. Perform local feature extraction: split the picture into small regions and perform feature extraction locally in each area. Then combine all the features of the individual areas into a single array.
Sounds:	Same strategy as for images within a 1D space instead of 2D

Practical implementations of such feature extraction strategies will be presented in the last sections of this tutorial.

PHP-MongoDB

To use mongodb with php you need to use mongodb php driver. Download the driver from the urlDownload PHP Driver. Make sure to download latest release of it. Now unzip the archive and put php_mongo.dll in your PHP extension directory ("ext" by default) and add the following line to your php.ini file:

extension=php_mongo.dll

Make a connection and Select a database

To make a connection, you need to specify database name, if database doesn't exist then mongodb creates it automatically.

Code snippets to connect to database would be as follows:

<?php
   // connect to mongodb
   $m = new MongoClient();
   echo "Connection to database successfully";
   // select a database
   $db = $m->mydb;
   echo "Database mydb selected";
?>

When program is executed, it will produce the following result:

Connection to database successfully
Database mydb selected

Create a collection

Code snippets to create a collection would be as follows:

<?php
   // connect to mongodb
   $m = new MongoClient();
   echo "Connection to database successfully";
   // select a database
   $db = $m->mydb;
   echo "Database mydb selected";
   $collection = $db->createCollection("mycol");
   echo "Collection created succsessfully";
?>

When program is executed, it will produce the following result:

Connection to database successfully
Database mydb selected
Collection created succsessfully

Insert a document

To insert a document into mongodb, insert() method is used.

Code snippets to insert a documents:

<?php
   // connect to mongodb
   $m = new MongoClient();
   echo "Connection to database successfully";
   // select a database
   $db = $m->mydb;
   echo "Database mydb selected";
   $collection = $db->mycol;
   echo "Collection selected succsessfully";
   $document = array( 
      "title" => "MongoDB", 
      "description" => "database", 
      "likes" => 100,
      "url" => "http://www.tutorialspoint.com/mongodb/",
      "by", "tutorials point"
   );
   $collection->insert($document);
   echo "Document inserted successfully";
?>

When program is executed, it will produce the following result:

Connection to database successfully
Database mydb selected
Collection selected succsessfully
Document inserted successfully

Find all documents

To select all documents from the collection, find() method is used.

Code snippets to select all documents:

<?php
   // connect to mongodb
   $m = new MongoClient();
   echo "Connection to database successfully";
   // select a database
   $db = $m->mydb;
   echo "Database mydb selected";
   $collection = $db->mycol;
   echo "Collection selected succsessfully";

   $cursor = $collection->find();
   // iterate cursor to display title of documents
   foreach ($cursor as $document) {
      echo $document["title"] . "\n";
   }
?>

When program is executed, it will produce the following result:

Connection to database successfully
Database mydb selected
Collection selected succsessfully
{
   "title": "MongoDB"
}

Update a document

To update a document , you need to use update() method.

In the below given example we will update the title of inserted document to MongoDB Tutorial. Code snippets to update a document:

<?php
   // connect to mongodb
   $m = new MongoClient();
   echo "Connection to database successfully";
   // select a database
   $db = $m->mydb;
   echo "Database mydb selected";
   $collection = $db->mycol;
   echo "Collection selected succsessfully";

   // now update the document
   $collection->update(array("title"=>"MongoDB"), array('$set'=>array("title"=>"MongoDB Tutorial")));
   echo "Document updated successfully";
   // now display the updated document
   $cursor = $collection->find();
   // iterate cursor to display title of documents
   echo "Updated document";
   foreach ($cursor as $document) {
      echo $document["title"] . "\n";
   }
?>

When program is executed, it will produce the following result:

Connection to database successfully
Database mydb selected
Collection selected succsessfully
Document updated successfully
Updated document
{
   "title": "MongoDB Tutorial"
}

Delete a document

To delete a document , you need to use remove() method.

In the below given example we will remove the documents that has title MongoDB Tutorial. Code snippets to delete document:

<?php
   // connect to mongodb
   $m = new MongoClient();
   echo "Connection to database successfully";
   // select a database
   $db = $m->mydb;
   echo "Database mydb selected";
   $collection = $db->mycol;
   echo "Collection selected succsessfully";
   
   // now remove the document
   $collection->remove(array("title"=>"MongoDB Tutorial"),false);
   echo "Documents deleted successfully";
   
   // now display the available documents
   $cursor = $collection->find();
   // iterate cursor to display title of documents
   echo "Updated document";
   foreach ($cursor as $document) {
      echo $document["title"] . "\n";
   }
?>

When program is executed, it will produce the following result:

Connection to database successfully
Database mydb selected
Collection selected succsessfully
Documents deleted successfully

In the above given example second parameter is boolean type and used for justOne field of remove()method.

Remaining mongodb methods findOne(), save(), limit(), skip(), sort() etc works same as explained in above tutorial.

Inserting data to mongoDB

The insert() Method

To insert data into MongoDB collection, you need to use MongoDB's insert() or save()method.

SYNTAX

Basic syntax of insert() command is as follows:

>db.COLLECTION_NAME.insert(document)

EXAMPLE

>db.mycol.insert({
   _id: ObjectId(7df78ad8902c),
   title: 'MongoDB Overview', 
   description: 'MongoDB is no sql database',
   by: 'tutorials point',
   url: 'http://www.tutorialspoint.com',
   tags: ['mongodb', 'database', 'NoSQL'],
   likes: 100
})

Here mycol is our collection name, as created in previous tutorial. If the collection doesn't exist in the database, then MongoDB will create this collection and then insert document into it.

In the inserted document if we don't specify the _id parameter, then MongoDB assigns an unique ObjectId for this document.

_id is 12 bytes hexadecimal number unique for every document in a collection. 12 bytes are divided as follows:

_id: ObjectId(4 bytes timestamp, 3 bytes machine id, 2 bytes process id, 3 bytes incrementer)

To insert multiple documents in single query, you can pass an array of documents in insert() command.

EXAMPLE

>db.post.insert([
{
   title: 'MongoDB Overview', 
   description: 'MongoDB is no sql database',
   by: 'tutorials point',
   url: 'http://www.tutorialspoint.com',
   tags: ['mongodb', 'database', 'NoSQL'],
   likes: 100
},
{
   title: 'NoSQL Database', 
   description: 'NoSQL database doesn't have tables',
   by: 'tutorials point',
   url: 'http://www.tutorialspoint.com',
   tags: ['mongodb', 'database', 'NoSQL'],
   likes: 20, 
   comments: [ 
      {
         user:'user1',
         message: 'My first comment',
         dateCreated: new Date(2013,11,10,2,35),
         like: 0 
      }
   ]
}
])

To insert the document you can use db.post.save(document) also. If you don't specify _id in the document then save() method will work same as insert() method. If you specify _id then it will replace whole data of document containing _id as specified in save() method.

Sentiment Analysis

In short, Sentiment Analysis is the process of detecting the contextual polarity of text. In other words, it determines whether a piece of writing is positive, negative or neutral.

An alternative term is opinion mining, as it derives the opinion, or the attitude of a speaker. A common use case for this technology is to discover how people feel about a particular topic.

For example, do people on Twitter think that Chinese food in San Francisco is good or bad?

Analyzing tweets for sentiment will answer this question for you. You can also learn why people think the food is good or bad, by extracting the exact word indicating why people did or didn't like the food. Example: "too salty"

This is the kind of insight one hopes to find when conducting market research. Now you know whether to expand your Chinese food empire to San Francisco or to keep it in Las Angeles.

Sentiment Analysis can be used to determine sentiment on a variety of levels. It will score the entire document as positive or negative, and it will also score the sentiment of individual words or phrases in the document.

For example, if someone writes a Facebook comment that reads:

"I love the summer in New York, but I hate the winter."

The individual scores would show "love the summer" as positive and "hate the winter" as negative. However, the sentiment for the entire comment would be neutral, because the positive sentiment for the word love would cancel out the negative sentiment for the word hate.

Because Sentiment Analysis can track a particular topic, many companies use it to track or monitor their products, services or reputation in general. For example, if someone is attacking your brand on social media, sentiment analysis will score the post as extremely negative, and you can create alerts for posts with hyper-negative sentiment scores.

Measuring sentiment accuracy

The accuracy of Sentiment Analysis can be measured in many ways, but the most common way is to score accuracy in comparison to a human. A study from the University of Pittsburgh shows that humans can only agree on whether or not a sentence has the correct sentiment, 80% of the time. So any natural language processing engine that can score around 80% is doing a great job with accuracy.

There are a few major challenges for an engine analyzing text for sentiment. One of the biggest issues is that it has trouble understanding irony. Even humans have trouble with someone who is being sarcastic. It is one of the most common mistakes a text analytics engine makes when trying to analyze text for sentiment.

Even humans have trouble, as they can analyze with 80% accuracy. Other problems are when words have multiple definitions. There are a few engines that use deep learning to help them understand context. For example, if someone is talking about excel, are they talking about the data software, the chewing gum, or the verb that describes how someone can be extremely good at something?

Data Types in MongoDB

MongoDB supports many datatypes whose list is given below:

• String : This is most commonly used datatype to store the data. String in mongodb must be UTF-8 valid.
• Integer : This type is used to store a numerical value. Integer can be 32 bit or 64 bit depending upon your server.
• Boolean : This type is used to store a boolean (true/ false) value.
• Double : This type is used to store floating point values.
• Min/ Max keys : This type is used to compare a value against the lowest and highest BSON elements.
• Arrays : This type is used to store arrays or list or multiple values into one key.
• Timestamp : ctimestamp. This can be handy for recording when a document has been modified or added.
• Object : This datatype is used for embedded documents.
• Null : This type is used to store a Null value.
• Symbol : This datatype is used identically to a string however, it's generally reserved for languages that use a specific symbol type.
• Date : This datatype is used to store the current date or time in UNIX time format. You can specify your own date time by creating object of Date and passing day, month, year into it.
• Object ID : This datatype is used to store the document’s ID.
• Binary data : This datatype is used to store binay data.
• Code : This datatype is used to store javascript code into document.
• Regular expression : This datatype is used to store regular expression