Artificial Neural Networks for lua

[SirRanjid]

Hi was being asked to talk a bit more about my implementation of ANN here in lua.

Actually already wrote this but when I tried attching a file via drag&drop(over the ad file button instead of the text entry) so my browser made the tab a download tab for my local file and everything got deleted for my convenience I suppose.


Anyways I got this neural network algorithm to work in lua due to me being to dumb to compile a mainstream-nn-framework for love. No actually it took me months to understand it by myself and youtube while I wanted to do my own. And after months of lazy(when I had time and interest) youtube research and 3 attempts at this I finally got it working and reviewing my own code also grew my understanding of this.

Here are some videos that helped me:
https://www.youtube.com/watch?v=q555kfIFUCM --Si
https://www.youtube.com/watch?v=GlcnxUlrtek
https://www.youtube.com/watch?v=Ilg3gGewQ5U

This actually made me understand it but it's German: https://youtu.be/EAtQCut6Qno?t=4m49s

The main problem I had was the chainrule. Most videos presented me the chainrule as concept thats built into the network. And while I thought I got to implement the chainrule it was already included in the formula.
And that the backpropagation is basically the same as the forwardpropagation but reusing the values of the forward pass in the derivative function(here we got the chainrule).

After I built it I broke it again by applying random jitters to all kinds of updates to see what kind of effect it has and just playing around with it.

And here's the .love for you to play around with:

neural_net.love

(7.17 KiB) Downloaded 980 times

https://github.com/SirRanjid/lua-nn

PRESS SPACE to pause/unpause.


So the dots on the bottom are vertically one batch of errorvalues for different inputs. They ideally reach the bottom of the screen. Or do nothing when a local minimum is reached. (they are randomly initiated so sometimes they get it, sometimes not for this XOR-datasetlet)

Functionality:

Code: Select all

local nn = require("nn")
local x = 3   --amount of layers
local y = 5   --default amount of neurons per layer
local y_inputs = 2  --nodes of input layer (the original 5 of y become 2 for the first layer, *not* a new layer with 2 as input)
local y_outputs = 1  --nodes of output layer

local activation_func = nn.func.sig
local act_f_drv = nn.func.asig
local lear_rate = 0.02

new_net = nn(x,y,y_inputs,y_outputs,activation_func, act_f_drv, lear_rate) --generate a new_net
new_net:addlayer(y) --becomes new output layer
new_net:build() --make weights for all connections

local train_pairs = {

	{{0,0},{0}},
	{{1,0},{1}},
	{{0,1},{1}},
	{{1,1},{0}},
  
  --{{<inputs>},{<expected_outputs>}},
}
local batches = 10
local batchsize = 10

for I = 1, batches
  for J = 1, batchsize do
    new_net:smart_train(train_pairs,0.6) --0.6  60% chance of picking a random pair, 40% picking the one that gives least error
    --smart_train is experimental and picks a random pair from train_pairs and keeps track of the error
  end
  new_net:applybatch() --apply weight updates
end

And here some motivational nerdgasm videoson that topic:

[yintercept]

made the tab a download tab for my local file and everything got deleted for my convenience I suppose.

Now I feel like an asshole for having asked. Heres hoping you made
backups. Also that image is hilarious.

Anyways I got this neural network algorithm to work in lua due to me being to dumb to compile a mainstream-nn-framework for love

I share your pain.

That first link I literally watched a month after it was released, lol. Siraj does a really good job of it too.

If you get a chance, initialize a large hidden layer with
random values, and then attempt to just train the output layer.
Drop out is also an interesting topic to explore on that front.

applying random jitters to all kinds of updates to see what kind of effect it has and just playing around with it.

this is the fun part

And here's the .love for you to play around with:

op friggin delivers

the dots on the bottom are vertically one batch of errorvalues for different inputs.

have you explored generative adversarial networks yet? Might be the next step.

This is really good stuff.

[SirRanjid]

If you get a chance, initialize a large hidden layer with
random values, and then attempt to just train the output layer.
Drop out is also an interesting topic to explore on that front.

Sounds interesting. Dropout was actually something I wanted to implement.
At the moment I got momentum/velocity.

this is the fun part

YEEES

have you explored generative adversarial networks yet? Might be the next step.

This is really good stuff.

Yeah they are cool but I think a q-learner could be better for love. Actually my next step was linking those networks to automate feeding/backpropagation along multiple chained networks.

Also those long-short-term-memory models sound interesting. (https://www.youtube.com/watch?v=WCUNPb-5EYI ,actually was searching for a video of one of deepminds that guides you through your subway by taking in a photo of the subway-map)

[yintercept]

Yeah they are cool but I think a q-learner could be better for love.

Oh you'll really enjoy Q learners. You may learn better from video but I found http://mnemstudio.org/path-finding-q-le ... torial.htm this article was *entirely* painless and straightforward.

I've always got lost in this stuff and if you're willing to go outside of neural nets you'll lose days and weeks too. K means is a good place to start for clustering. And then go on to PCA which is a relaxed solution of k-means. And if you go in that direction, when you finish with principle component analysis (I finished k means but never got past PCA), self organizing maps are supposed to be a fun generalization of pca.

One day when I get the time I'm hoping to explore some of the simpler stuff like random forests, utility systems, and other ai more traditional to games. Theres lots of topics worth exploring. Sir, if you expand any more be sure to ping me!

[SirRanjid]

I bet^^. I think K-means is something I already got. I'm more about trying to make an AI that could really kick some ass and be fun to play against with.
Or imagine a simple zombie ai which is driven by a neural net. For each zombie just apply some small random jitter to the weights and it seems like you got a crowd of zombies that behave differently.

I could imagine using an neural network in combination with k-means to analyze and predict movement of troops/groups for strategy games. A bit like handwriting prediction.
Could also be combined with a GAN I think.

Hmm self organizing maps also sound interesting... (looks like it could also aid a strategy ai or map generation)

Thanks for your input these are definetly interesting topics.

[yintercept]

> I'm more about trying to make an AI that could really kick some ass

If you want 'fun' AI that is also relatively skilled, if you are fascinated by emergence, you should look into Utility AI and Behavior Trees.

I've done toy examples (on paper) to demonstrate the concept to kids, and both behavior trees and utility ai are fairly easy to grasp.

Always wanted to try capturing the output of a utility ai and transform it into a behavior tree.

If you hook up an NN as the utility function for the utility ai, or just the jitter method you wrote then you could experiment with weight manipulation for the ai in question. Unlike behavior trees, utility has to be recalculated every step so it can be expensive to track a lot of elements (read the articles about the ai in FEAR2 for more), but if there was someway to capture that and 'freeze' the utility values
as a behavior selector then some of these calculations might be mitigated.

I've seen examples that go in this direction, but where the utility ai is a node in the behavior tree, and not the other way around. All and all though, it seems like utility is the 'peanutbutter' to decision and behavior tree 'jelly' and I wish I had the time to explore it more.
Researchers have said the future of AI is through ensemble methods and from what I can tell this is a largely unexplored direction.


Edit: You may also enjoy http://trainsportedgame.no-ip.org/media.php a game about using AI to manage a rail network. Built with love2d I believe.

[SirRanjid]

https://love2d.org/imgmirrur/Zyq0LFc.mp4

I updated it bit to make it color different training pairs differently on the plotter instead of coloring all by their error.
Also added sin and cos functions, some jitter and a new net for eyecandy.
The nn code itself now also outputs the index of the used training-pair.

[yintercept]

What have you trained it on so far?

[SirRanjid]

Nothing haha. Actually I like to watch it do stuff like shown in the image and just alter stuff like jitter and such. I concluded that a fixed jitter gets problematic the closer it gets to the error value or when the jitter is bigger than it. So maybe a dynamic/relative jitter is better (as with what you achieve when back-propagating the squared error instead of the error alone(the error value gets even smaller when the error decreases, which lets it converge nicely towards a minimum(hopefully the global))).
I find it to be more fun to play with the algorithm itself.

Other than that I can't really think of useful stuff I can train it on also in terms of performance of an neural net written in a c-interpreter. The next problem is useful data. Training it on those freely available databases seems a little boring to me.

But facebook has automatic descriptive captions for each photo you upload. (if you check the page's source it tells you whats seen of the profile pictures in text) So one could use facebook-profile-pictures and their already generated descriptions to get a similar network. That somehow sounds fun when having an automatic agent that collects this stuff and feeds it to a network.

Maybe I'll make the training look better and sporadically throw out some animations that look cool and give you a visual hint of how it works.

I got myself a nvidia jetson nano and want to use some more advanced libraries like tensorflow and maybe train it on stuff. I'm really interested in those networks that compute depth from just one video-stream for each pixel and build some kind of SLAM robot from it (or with open-cv). And generate a octree map from the point-cloud. I already thught of the part that turns the point-cloud into a useful octree map. Maybe I can have a neural net that guesses the position in that octree map and from that position run an algorithm that gets it more precise and then give feedback to the neural net... etc. (Maybe I could train a neural network on finding interesting points to track for the computer-vision part. If its good at finding and recognizing the same spots I could try finding the position in a more hash-table manner i think)
The bigger challenge then would be to make it run in a useful speed on the jetson. So much interesting thoughts....
But I'm just beginning to learn how to use the gpio pins which requires me to learn about the electric components in order for me not have a burning jetson at some point. But it will take some years as side-hobby and i just occasionally have interest in doing stuff on it. Only downside is that I dislike python. Shame on google for not going with lua. Or I dive more into linux and make everything work with lua which costs even more time.


tl;dr:
Nothing really trained on. It's more another hobby in my hobby-rotation.

[SirRanjid]

I made it look nicer.

https://love2d.org/imgmirrur/Ha0uuNR.mp4
https://love2d.org/imgmirrur/W7SyIOn.mp4
https://love2d.org/imgmirrur/VUU2Zmd.mp4

The last one has different activation functions on each layer.