Dueling neural networks
Neural networks aren’t limited to just learning data; they can also learn to create it. One of the classic machine learning papers is Generative Adversarial Networks (GANs) (2014) by Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, and others. GANs take the form of two opposing neural networks one learning to generate fake samples while the other tries to separate the real samples from the fakes.
Sophisticated GAN models like VQGAN and others generate everything from fake landscapes, faces, and even Minecraft worlds.
These are my notes on the classic paper that first introduced GANs.
Setup
GANs are fundamentally a two-part binary…
The simple power of [[links]]
The problem with traditional notes
There’s a stack of old notebooks sitting on my shelf, each filled with notes from books, random thoughts, to-do lists, and ideas. The notes and ideas seemed important enough to capture at the time, but I couldn’t tell you what’s in them. Today, they’re more like a treasure hunt without a map. Short of leafing through them page by page, I have no way of knowing what’s inside or easily finding old ideas.
I expect I’m not the only person with this problem. There’s a wonderful feeling of capturing our thoughts with pen and paper, but that’s often where they…
Working through attention in deep learning
I’ve started to go through classic papers in machine learning, inventions that shifted the state of the art or created an entirely new application. These are my notes on the Transformer introduced in Attention is All You Need. The transformer addressed problems with recurrent sequence modeling in natural language processing (NLP) but has since been applied to vision, reinforcement learning, audio, and other sequence tasks.
Why attention?
Recurrent models built from RNNs, LSTMs, or GRUs were developed to deal with sequence modeling in neural networks because they can include information from adjacent inputs as well as the current input. …
Machine Learning | Natural Language Processing
An AI alternative history with natural language processing
My top priority is to reduce carbon emissions. The more we can reduce our dependence on foreign oil, the more energy we can produce. This is why I want the U.S. to keep growing as fast as possible, to do whatever it takes to bring about the transformation of our economy. The world, in its most powerful, strongest sense, needs a plan to deliver change.
No, you didn’t miss a debate between Presidents Obama and Trump. The quote above was generated entirely by an AI, trained to replicate the speeches of President Obama! …
In my last article on RNNs, we looked at how the purpose of RNNs, how an RNN cell is built, and how to implement RNNs in PyTorch for sequence prediction. In this post, we’ll look at a more complex form of recurrent networks, the Long Short-term Memory network (LSTM) and why they’re better suited than vanilla RNNs for things like language processing.
LSTMs
In basic RNNs, the network is able to consider previous inputs in the form of a hidden state when computing the output of the network at the current time. However, this means that inputs occurring several steps behind…
Neural networks with memory
Introduction
A typical feed-forward neural network maps inputs to outputs with no consideration of previous computations or where the current input fits in relation to others. The network applies the same function to each input regardless of sequence. This is fine for many applications, but often the context of an input has some relevance to the target output. One way to address this problem is to use a recurrent neural network (RNN). An RNN is a network with ‘memory.’ The network maintains information about previous inputs allowing its current output to be generated with consideration about the past. There are a…
Intro
Mountain Car is a classic reinforcement learning problem where the objective is to create an algorithm that learns to climb a steep hill to reach the goal marked by a flag. The car’s engine is not powerful enough to drive up the hill without a head start so the car must drive up the left hill to obtain enough momentum to scale the steeper hill to the right and reach the goal.
Q-Learning
We will use a reinforcement learning technique called Q-Learning to solve this problem. Q-Learning is an algorithm that attempts to learn a function or policy which takes an…
I think one of the best ways to learn a new topic is to explain it as simply as possible so that someone with no experience can understand it (aka The Feynman Technique). This post is an attempt to do that with policy gradient reinforcement learning.
I’m new to reinforcement learning so if I made a mistake or you have a question, let me know, so I can correct the article or try and provide a better explanation.
Intro
CartPole
We’ll be using the OpenAI Gym environment CartPole where the object is to keep a pole balanced vertically on a moving cart…
