Table of Contents
IntroductionVadim Karpusenko introduces the course, and gives an overview of materials that will be needed to complete the course.
Jupyter Notebook & TensorFlowVadim introduces Jupyter Notebook, and demonstrates how to access local files on the platform.
Machine Learning, Neural Networks & Deep Learning
Machine Learning DemoVadim demonstrates how to use PoseNet, a machine learning model which allows for real-time human pose estimation in the browser.
Deep Learning, Machine Learning, & Neural Networks ResourcesVadim shares the three books the workshops references, and points out that each book has a Github repository with its corresponding source code.
Statistics & Machine LearningVadim discusses the relationship between statistics and machine learning, and explains the concept of feeding. Feeding is when machine learning takes practical information and measurements, and finds distributions and corresponding values.
Neurons & Neuron LayersVadim analyzes what the relationship between machine learning and neurons is, and explains that machine learning is replicating the processing of information of neurons, it associates neural channels with specific weights, and adds multiple signals together.
Deep LearningVadim describes the process of how deep learning is generated, which is from multiple layers of neurons being part of a machine learning model.
Neurons & Model TrainingVadim explains that each neuron within a network has its own weight, demonstrates that reinforcement is learned by a neural network by penalizing or rewarding the activated neural channels, and shrinking down or increasing the weight associated with each neuron.
Training ModelsVadim explains that in machine learning, data is fed to a model, processed through a CPU or GPU, then the data is frozen or stored once the training ends. Afterwards, a new image is fed to the model, and with proper training the model should be able to recognize what the image represents.
Deep Learning & AIVadim explains that when thinking about artificial intelligence, engineers assume that the computer is passing the Turing test and is able to be creative. Once information is processed through deep learning, artificial intelligence determines how to use the data. An example is given of an image's style being processed through deep learning and AI applying that style to another image.
AI Modeling DemoVadim highlights other examples where AI modeling changes the style of an image after being given data processed through deep learning.
Combining Neural Networks & AIVadim demonstrates how neural networks can be trained to work on different tasks, and have their work combined.
Jupyter Notebook SetupVadim demonstrates how to import libraries and run code on the Jupyter Notebook.
Hardware & CompilerVadim gives an overview of how hardware and the compiler work together as a premise to better understanding how tensorflow functions under the hood.
Tensorflow ArchitectureVadim gives an overview of how Tensorflow is built, explains that Tensorflow used computational graphs to allocated memory, and is now switching to use eager execution which means Tensorflow has access to the code right when it is executed.
Plotting Uniform DataVadim demonstrates how to use the Matplotlib library to depict uniform data. The following sections iterate over the same linear regression model.
Plotting Data with NoiseVadim demonstrates how to add noise to a normal distribution, and how to use both the Matplotlib and the numpy libraries for histograms using the linear regression example from the previous section.
Predicting Data with TensorFlowVadim demonstrates how to use TensorFlow to calculate the error between two different plot lines: one representing the solution, and the other being a prediction of the solution. The loss function calculates the difference between the solution and the prediction. This section introduces different types of helper functions used to predict data using TensorFlow.
Visualizing Learning Rate & LossVadim explains that in order to improve the loss function, one has to improve the neural network learning rate by modifying the weights and biases within the loss function.
Learning Rate & GradientsVadim demonstrates how to use gradients and learning rates to better the training process of a model and get closer to the solution. The more the learning rate increases, the closer the model will get to the line representing the linear regression.
Learning Rate vs StepsVadim explains that learning rates and steps are independent hyper parameters that, when combined, lead to more accuracy, and reviews the concept of loss function and how it was applied to this section's linear regression example.
Image Processing & Model Training
Image Processing with KerasVadim demonstrates how to build training data, use a dataset, and visualize its images using the Matplotlib library.
Creating a 2D Image ModelVadim demonstrates how to add layers to model by first creating a sequential model one layer and allocating a specific amount of neurons to the layer.
Sigmoid vs. ReLUVadim compares sigmoid to the rectified linear unit, and explains that calculating one sigmoid value requires iteration, whereas using the rectified linear unit is one quick operation. ReLU is preferred to sigmoid because it calculates faster and does not perform exponential operations.
SoftmaxVadim explains that the softmax function has an output equal to one. Normalizing the sum of output to one permits to look into neurons as probabilities.
Understanding the ModelVadim describes the different layers available in a model, the difference between a flatten and dense layer, and discusses how pixels and neurons interact within each layer.
Defining the Solver & Loss FunctionVadim explains that, in a model, a solver contains information about how to get to the solution and determines what the loss function should be.
Training a ModelVadim explains that, after defining the solver and the loss function, the model needs to be trained. To train a model, data is given to the model, the model iterates through multiple trainings, called epochs, and then the model compares its estimations to the validated data.
Comparing Different ModelsVadim explores modifying the number of layers and neurons, and compares the accuracy of each model. A model is divided into a training dataset, a test dataset, and a validation dataset, which has images that were never seen by the model. The validation dataset determines which model is the best one.
Principal Component AnalysisVadim explains that principal component analysis is a mathematical technique that looks at all of the provided data, and analyzes how much useful information is contained per degrees of freedom.
Convolution & Pooling
Convolutions & Pooling DefinitionsVadim explores two important methods in image processing: convolutions and pooling. Convolution is only partially connected neural networks to pixels. Pooling only takes in consideration the largest pixel within a pool of pixels from a given image.
Adding Convolutions & PoolingVadim demonstrates how to add multiple convolutional layers and how to add a callback function containing an accuracy rate. The accuracy rate is useful when when the correct number of epochs is unknown.
Diagramming Convolutions & PoolingVadim demonstrates how to plot a model accuracy and a model loss for both the training and the test functions.
Visualizing Convolutional OperationsVadim demonstrates how convolutional operations change an image using a filter that modifies its pixels.
Predicting, Saving & Loading ModelsVadim provides different steps that one can take after training a model, accesses individual layers, starts predictions, and visualizes neural networks.
Neural Network AttentionVadim explores where the convolutional network looks for data when making predictions, introduces the concept of transfer of learning, and answers a question about the use of mini bashes.
Bias in Training Data SetsVadim explores training my the model into recognizing a new image, and focuses on where the attention of the neural network is.
Detecting Good & Bad ReviewsVadim explains that a dataset can be overfed with data, and demonstrates how to spot overfeeding and prevent it.
Natural Language ProcessingVadim explains that in language processing, each word is assigned particular coordinates. Tthis way negative and positive words are differentiated. Word embedding is also discussed.
Sentiment AnalysisVadim demonstrates how to train a model to predict if comments are positive or negative, and explains that predictability depends on the given dataset and the preprocessing of the text.
Long Short Term MemoryVadim explores Long Short Term Memory or LSTM ,one of the classes of neural networks , explains how one has to build intuition when working in machine learning and deep learning, highlights the importance of keeping a clean training dataset, and trains a model that recognizes the author of a quote.
Reinforcement LearningVadim explains what reinforcement learning is, introduces different reinforcement learning example, and introduces the open AI gym, a toolkit for developing and comparing reinforcement learning algorithms.