Why Machines Learn: The Elegant Math Behind Modern AI

by Anil Ananthaswamy

Step 1. Store all instances of sample data. Each penguin is a vector [x1, x2], where x1=bill depth and x2=bill length. The entire dataset is stored in a matrix X, where X has m rows (number of penguins) and n columns (number of features). Each penguin is also associated with a label y, which is equal to -1 (Adélie) or 1 (Gentoo). So, y, which stores all the corresponding labels, is an m-dimensional vector. Step 2. Given a new data point, representing an unclassified penguin, in the form of a vector x with elements for bill depth and bill length [x1, x2], do the following: Calculate the distance of the new data point to each of the data points in the original dataset, X. This gives us a list of m distances, d. Sort the list d in the order of increasing distance (the first element has the minimum distance to the new point, and the last element has the maximum). As you sort d, simultaneously rearrange the elements of y, so that the appropriate labels (-1 or 1) remain associated with each penguin in the sorted list. Step 3. Take the first k elements of the sorted list d. These represent the k nearest neighbors. Collect the labels (-1 or 1) associated with each of these nearest neighbors. Count the number of 1s and the number of -1s. Step 4. If the 1s outnumber -1s, the new data point has the label 1 (Gentoo); else it’s an Adélie.