Unsupervised Learner
https://bigthyme.github.io/bookshelf/index.xml
Recent content on Unsupervised LearnerHugo -- gohugo.ioen-usThu, 23 Feb 2017 22:25:09 -0800Neural Networks for Novices
https://bigthyme.github.io/bookshelf/post/neural-network/
Thu, 23 Feb 2017 22:25:09 -0800https://bigthyme.github.io/bookshelf/post/neural-network/<p>Neural networks are a popular topic in the Machine Learning community but to the rest of us, they’re relatively unknown and somewhat unapproachable. What is a neural network? And what are they used for?
If you’ve had exposure to linear regressions, the same idea behind why we need such methodologies can be applied to why we need neural networks.
Imagine we have a hypothesis; given a list of data points like number of rooms and perhaps the size of a house can we figure out the price of similar houses. The basic premise of that problem suggests that each variable (number of rooms, size of house) has a binary relationship with the output; mainly the output can either be negatively affected by the variable or the opposite can be true. With neural networks, the variables have a dynamic relationship with the output. Neural networks on the other hand, are used for problems that might not be so obvious. Perhaps we want to find out out if a students test scores can be predicted based on hours spend studying and hours of sleep the night before the exam. It would be difficult to model that problem linearly, but we can start to see why we might need a more advanced technique.</p>
<p>In this post, we’ll dive into implementing a very simple neural network to show that the foundations of a neural network are fairly simple to
understand. As previously mentioned, I’m completing the <a href="http://anthonysing.com/post/regress/">Machine Learning</a> course with Professional Andrew Ng
and thus we’ll write in the language of <a href="https://www.gnu.org/software/octave/">Octave</a>.</p>
<p>To start, I’d like to look at the below graph to see if we can identify any patterns.</p>
<table>
<thead>
<tr>
<th>X₁</th>
<th align="center">X₂</th>
<th align="right">Output</th>
</tr>
</thead>
<tbody>
<tr>
<td>0</td>
<td align="center">0</td>
<td align="right">1</td>
</tr>
<tr>
<td>0</td>
<td align="center">1</td>
<td align="right">0</td>
</tr>
<tr>
<td>1</td>
<td align="center">0</td>
<td align="right">0</td>
</tr>
<tr>
<td>1</td>
<td align="center">1</td>
<td align="right">1</td>
</tr>
</tbody>
</table>
<p>If you’ll notice, when X₁ and X₂ are set to the same value (e.g. in the first and last row):</p>
<pre><code>[0 0 1]
[1 1 1]
</code></pre>
<p>This is what’s called an AND function where the bitwise operation performed on either numerical value produces <code>1</code>. We’ll try to further expand what this means and how it contributes to building a neural network.</p>
Linear Regression: NBA Edition
https://bigthyme.github.io/bookshelf/post/linear-regression/
Thu, 02 Feb 2017 23:24:00 -0800https://bigthyme.github.io/bookshelf/post/linear-regression/<p>As a longtime basketball fan, I’ve always been saddened at the eventual decline of once great players. I’ve often wondered if age does matter in the NBA or
do some coaches value experience moreso? In this post, I’d like to explore whether age and number of games played can predict future playing time.
My hypothesis is that players who are older tend to see less playing time.
With all good supervised Machine Learning alogirthms, we need good data. In this example, I query <code>http://stats.nba.com</code> with the help of the NBA’s API documentation from <a href="https://github.com/seemethere/nba_py/wiki/stats.nba.com-Endpoint-Documentation">seemethere</a>.
The data comes from the 2015-16 NBA season and the individiual data points are averages of every NBA player’s last 10 games.</p>
<p>To start, we should download the data and put it somewhere accessible</p>
<pre><code># In this example, I download the file to my ~/Downloads directory
$ cd ~/Downloads
$ curl "http://stats.nba.com/stats/leaguedashplayerstats?College=&Conference=&Country=&DateFrom=&DateTo=&Division=&DraftPick=&DraftYear=&GameScope=&GameSegment=&LastNGames=10&LeagueID=00&Location=&MeasureType=Base&Month=0&OpponentTeamID=0&Outcome=&PORound=0&PaceAdjust=N&PerMode=PerGame&Period=0&PlayerExperience=&PlayerPosition=&PlusMinus=N&Rank=N&Season=2015-16&SeasonSegment=&SeasonType=Regular+Season&ShotClockRange=&StarterBench=&TeamID=0&VsConference=&VsDivision=" > nba_players_2016.json
# Upon opening this data, you'll realize it's not a csv but I wrote nodejs code to parse it into one for your convenience
$ git clone git@github.com:bigthyme/ml-lessons.git
$ cd linear-regression
$ ls -al
</code></pre>
<p>Next, we should pick our modeling technique. We’ll be using the closed form <a href="http://mathworld.wolfram.com/NormalEquation.html">Normal Equations</a> and comparing that to the results from <a href="https://en.wikipedia.org/wiki/Gradient_descent">Gradient Decent</a>.
Because I’m learning from Professor Ng’s <a href="http://anthonysing.com/post/regress/">course</a>, I’ll be writing my algorithms in the language of <a href="https://www.gnu.org/software/octave/">Octave</a>. Thus, it may be prudent to follow
this <a href="https://www.youtube.com/watch?v=TWqYSOiSTEA">tutorial</a> on Octave prior to following my examples.</p>
<p>Let’s create a file named <code>normalEqn.m</code> where we will write out our normal equation in octave</p>
<pre><code>% declare a normalEqn function in Octave
function [theta] = normalEqn(X, y)
% initialize theta to a matrix of equal dimensions as expected result
theta = zeros(size(X, 2), 1);
% calculate the inverse matrix of the training data
theta = pinv(X)*y
end
</code></pre>
<p>For clarity, I’m going to create another file for calling the <code>normalEqn.m</code> function and adding useful logs</p>
<pre><code>printf('Loading up dataset for testing...\n');
data = csvread('player-defense_2015.csv');
% grab the proper columns from the csv
X = data(:, 1:2);
% grab the mins played column
y = data(:, 3);
m = length(y);
% Add intercept term to X
X = [ones(m, 1) X];
% Calculate the parameters from the normal equation
theta = normalEqn(X, y);
% Display the normal equation's result
fprintf('Theta computed from the normal equations: \n');
fprintf(' %f \n', theta);
% theta should be
% theta =
% 13.3900
% 9.5308
% 3.3927
</code></pre>
<p>Turns out it’s hard to predict this model as 13.39 minutes doesn’t explicitly tell us much about an individual player but we could say that 13 minutes in general is quite low
for a full-time NBA player. To improve this existing model, we could’ve tracked an individual player separately throughout their career and run the algorithm according to that individual’s dataset.
But let’s say we did infact have more explicit data on an individual player, how would we further validate our theta value given this model?
One way to validate the hypothesis is to use a different algorithm to double check our work. If we take the iterative approach of gradient descent
we can visualize the model and verify if the <a href="http://stats.stackexchange.com/questions/155580/cost-function-in-ols-linear-regression">cost function</a> is being
minimized. To use gradient descent, you have to make an educated guess as to what the learning rate (or alpha) might be along with
guessing the number of iterations a model needs to be useful. In the graph below, I used the following variables:</p>
<pre><code>alpha = 0.01;
number_of_iterations = 500;
</code></pre>
<p><img src="https://n31.imgup.net/gradient_cf771.png" alt="Gradient Descent Test" title="Gradient Descent" /></p>
<p>As you can see with our current data it does trend in a good direction but doesn’t in fact converge well
(read more about how to further debug results from this <a href="https://www.quora.com/OPTIMIZATION-What-are-some-tips-for-debugging-a-gradient-descent-algorithm-that-isnt-converging">technique</a>).
Again, we could’ve dramatically improved the model but I think it’s ok to run models and find out they don’t work well. As I continue to run this type of analysis I hope to
get better at invalidating models more quickly from a cursory glance.</p>
I Regress to Progress?
https://bigthyme.github.io/bookshelf/post/regress/
Sat, 31 Dec 2016 19:17:49 -0800https://bigthyme.github.io/bookshelf/post/regress/<p>I spent the last week brushing up on Linear Algebra in preparation for week 2 of Andrew Ng’s Machine Learning course offered via <a href="https://www.coursera.org/learn/machine-learning/home">Coursera</a>.
This is my second time attempting the course and I’m hoping I’ll be more motivated and prepared this time around. This blog serves as some accountability and perhaps
encouragement for others to join! I’m attempting to finish the course before the end date of March 5, 2017 and so far making good progress. My only advice from week 1 is to spend the time to go through matrices.
The course review covered the basics of addition, multiplication, transposition, and transformaiton too fast for my liking. If you’re like me and you’re not practicing matrix multiplication before bed every night,
I highly recommend watching khan academy’s <a href="https://www.khanacademy.org/math/precalculus/precalc-matrices/multiplying-matrices-by-matrices/v/matrix-multiplication-intro">review</a> of matrices.</p>
<p>If you have any insights on how to better prepare for week 2, i’m all ears!</p>
<p>Happy Coding,
Ant</p>
About
https://bigthyme.github.io/bookshelf/about/
Sat, 20 Jun 2015 14:02:37 +0200https://bigthyme.github.io/bookshelf/about/<p>Hi, my name is Anthony and I thoroughly enjoy learning new things.
If you’d like to chat more about anything, feel free to <a href="mailto:hi@anthonysing.com">hi@anthonysing.com</a> email me.</p>