Machine Learning

Machine learning refers to a broad range of algorithms and computational techniques in which prior information is used to “teach” the algorithm how to predict future instances of a phenomenon. These techniques include neural networks, Bayesian networks, and such statistics for machine learning as clustering algorithms, factor analysis, principal components analysis, canonical correlation, and logistic regression. Broad types of algorithms rely on supervised learning, in which the program is presented with examples of inputs and corresponding outputs; unsupervised learning, in which the program is left to “learn” on its own; and semi-supervised learning, in which many cases are missing. Outputs in machine learning can be probability distributions, classifications into two or more categories, clustering, in which cases are grouped into clusters, and dimensionality reduction, as in factor analysis, canonical correlation, and principal components analysis.

As an example, we are going to look at a logistic regression which we will use to classify individual cases into one of two classes. The data we’ll use is Fisher’s iris dataset, which contains observations of iris sepal widths, sepal lengths, petal widths, and petal lengths for three species of iris. For simplicity, we’ll only include two of the three species in this example—versicolor and virginica. Moreover, an initial run fitting the logistic regression revealed that sepal width and length did not provide a statistically significant basis on which to distinguish these species, and so only petal length and petal width were retained as independent variables.

Whereas ordinary least squares regression must have a continuous, or real number, variable as its dependent variable, a logistic regression must have a categorical one. Letting l be petal length and w be petal width, the logistic regression we’re fitting would look be as in Equation (1) below:


A more meaningful version is the logit, which restates (1) as follows:


Fitting the model yields an output as shown below in Table 1:

Table 1. Output of a Logistic Regression Fit to Iris Data

Key Specs

Estimate Std. Error z value Pr(>|z|)
(Intercept) -45.272 13.61 -3.327 0.000879 ***
Petal.Length 5.755 2.306 2.496 0.012565 *
Petal.Width 10.447 3.755 2.782 0.005405 **
Signif. Codes: 0 ‘***’ 0.001 0.01 ‘*’ 0.05 ‘.’ 0.1

Thus, the logit is the following:


Using this equation, petal length and petal width of other irises can be used to identify species of iris.

Although this example may seem simplistic, the power of this and other machine learning techniques is powerful. One real world example concerned supply logistics within the Department of Defense. At a high echelon within the military, a special, or one-time, program would be established. Through failure to communicate its nature, the requisitions associated with the program would not be classified as SPR, meaning that this materiel would be included in forecasts of future needs. Since SPRs are, by definition, one-time events, including the materiel associated with them in forecasts is not desirable and would tend to inflate forecasts for those items. On the other hand, falsely assuming a particular requisition was associated with an SPR and not including it in a forecast would underestimate future needs. Using a set of correctly classified requisitions associated with SPRs, a logistic regression was fitted with which to classify unmarked requisitions that were suspected to be SPR related. A related benefit of this analysis was to explore the costs of either type of misclassification.

Machine learning online is available. Several MOOCs (massive open online courses) exist to teach it. For instance, Udacity offers nanodegrees in which students work to product course outputs and learn in the process. One nanodegree is Machine Learning Engineer Nanodegree, in which are given a problem and must identify an algorithm or tool with which to address the problem. They then create the system, analyze its effectiveness, and deploy it in a live environment. Another machine learning company is Coursera, which offers not only a full specialization in machine learning but also incorporates it in many other data analysis specializations.