Homework Five: Adaptive LTUs and More

Part 1: More with NeuralNetUnits

This week, you will extend your LTU so that it can learn a particular output function.

First, in case you didn't do this to start with, you should treat the threshold as just another weight with an artificial input that is held constant at -1. Thus, w0 is usually the effective threshold and is associated with a constant -1 input, x0. Then w1 through wn are the weights for the actual inputs to the LTU. When you compute the weighted sum from 0 through n, the sum will be greater than 0 when the weighted sum of the actual inputs exceed the threshold (weight on x0).

Second, in order to train an LTU, we need to distinguish the activation as weighted sum from the symbolic value we associate with a particular activation. For example, an activation of 0.91 may be treated as an output of "1" but we will need the 0.91 when it comes time to train the entire network (in which an LTU takes part).

Third, rather than using the step function we used before to determine a unit's output, we can use a sigmoid function that has certain desirable properties that you can learn about in CS150 Machine Learning. The commonly used function is: sigma(x)=1/(1+e-x), where x is the weighted sum for a given node. For example, a nodej, would have weighted sum Xj (including the threshold, x0), and would have an output, Yj = 1/(1+e-Xj).

At last, we get to training. An error signal for node k is the difference between the desired output and the actual output (as given by the sigmoid function). Thus, ek=dk-yk, where ek is the error for desired output dk and actual output yk. Now we want to use the error to follow the gradient toward the ideal weight settings for this node, k. So taking the derivative we get an error gradient, gk=yk (1-yk) ek.

Finally, during training, wjk gets replaced by wjk + a xj gk for weight wj and input xj of node k (the LTU in question). The variable a is a learning rate parameter that you can hold constant at some small value less than 1.0 (0.1 or 0.05 might be a good starting place).

Requirements

For this week's assignment, modify your MyLTU class to also implement the three methods from the NeuralNetUnit interface that we ignored in Assignment Four. These are: updateWeights, computeGradient, and getGradient. You should be able to write a main method (or a driver class) that starts with a new LTU having random weights (and threshold), and then trains it by looping so that it eventually computes a basic boolean function on the given inputs. We will test your program with a driver class that (a) creates an instance of your MyLTU with an arbitrary number of inputs, (b) sets those inputs to random values, and then (c) attempts to train your MyLTU on an arbitrary boolean function of our choice.

Part 2: Recursive Algorithms

Requirements

Write a public static method, lessThan, that takes as argument an array of ints and returns a boolean value. Your method must be recursive and may not contain any for or while loops. The method returns true only in the case where the array elements are in strictly increasing order starting with index 0 up to n-1 for an array, a, with n elements. That is, if a[0] < a[1] < a[2] < ... < a[n-2] < a[n-1], your method should return true; otherwise it should return false. You should place this method in a class, RecursiveLT, that simply contains this method. You should write a main method for your own testing as we will be grading you, in part, on your own testing. We will also base your grade on how your code works with our own driver that calls your static method directly (i.e., without creating an instance).

Submission instructions:

Do not forget to use the javadoc constructs for your documentation and include the html file produced by the javadoc tool in your final submission. Make sure that you create a folder named with your Westmont email name followed by "HW5". For example, someone with email address "cjones" would create a folder called "cjonesHW5". Make sure that inside that folder you have: for both parts of the assignment, java source files and html documentation files resulting from javadoc. Finally, either tar or zip the folder so that when we extract it, the folder "<emailname>HW5" will be created. Submit the packaged file for assignment 3 on Eureka.