Table of Contents
IntroductionScott Moss introduces and provides an overview of the course notes and code repository.
Course OverviewScott talks about the current state of AI and provides an overview of the course content, including building a chat experience, semantic search, document QA, and function calling. The requirements for the course are Node.js and having an OpenAI API key. The course will mainly focus on using OpenAI but will discuss on other models and services.
Intro to LLMsScott introduces Large Language Models (LLMs) and explains what they are and how they work. The role of these models in the AR world and the use cases for LLMs, such as writing, content creation, customer support, research, and education, are also discussed.
Setup Node & OpenAI APIScott sets up the project and installs the necessary dependencies. They then explain how to set up the API key using a .env file and demonstrate how to import and use the OpenAI library. They show how to make a chat request to the AI and log the response. The instructor also discusses the concept of tokens and how they affect the cost and response of the AI. They demonstrate how to send messages to the AI and receive responses, and explain the limitations of chat models in terms of memory and non-determinism. Finally, they introduce the concept of a chatbot and explain the difference between the code they have written and a chatbot.
Simple AI Chat in Node.jsScott creates a Node command-line chat interface using the OpenAI GPT-3 model. The readline module reads input from the command line, and a recursive function continuously prompts the user for input and generates responses from the AI model.
Scaling Chat & AI TemperatureScott explains the scaling issues that come with chat-based applications. Token limits, memory constraints, and the need for creative solutions to handle these limitations are discussed.
Search & Langchain
Search & Langchain OverviewScott introduces semantic search, which goes beyond simple string matching to understand the meaning and context of words. Scott shares some examples of semantic search in action, such as autocomplete suggestions and recommendation systems. Langchain is introduced, and the semantic search feature in the documentation is demonstrated.
Embeddings & VectorsScott introduces word embeddings and vectors and describes how embeddings are collections of vectors that represent the meaning behind words. Embeddings are useful for storing and comparing the meanings of words and are essential for semantic search.
Creating a Semantic SearchScott provides an example of how semantic search can be used to find movies with similar themes. A movie recommendation semantic search application is created, and the initial required libraries are imported. A createStore method is written to return an in-memory vector store from a list of movie documents.
Performing a Similarity SearchScott codes the search function, which converts a query into an embedding and uses cosine similarity to compare it with other embeddings. The score of each search result is displayed demonstrating the proximity of each result.
Scaling Semantic SearchScott discusses the challenges and considerations when deploying and scaling semantic search. Challenges include handling large volumes of data, updating and maintaining indexes and embeddings, ensuring query accuracy, optimizing for speed and accuracy, and using caching and feedback loops.
Question Answering Systems
Document QA SystemsScott introduces document QA (Question Answering) and compares it with semantic search. Document QA systems provide direct answers to questions, rather than just returning links like search engines. Use cases for document QA systems are discussed as well as benefits including efficiency, accuracy, and scalability.
Creating a Document QAScott begins coding a document QA application. A vector store is created and the documents providing data for the application are a YouTube podcast video and a PDF of the XBox instructions.
Youtube & PDF Document LoadersScott loads the YouTube video and PDF into the application and explains how chunking avoids exceeding the token limit and improves search accuracy.
Document QA Query FunctionScott implements the query function, which performs the similarity search and sends the query to the GPT model. The similarity search determines which source should be used for determining the answer. The context from the source along with the question are then sent to the model for an answer.
Logging & Testing the QueryScott logs the results generated by the AI model and displays the sources from which the answer was derived. The possibility of having the AI model itself provide the source it used is also discussed along with scaling the document QA system.
AI-Driven Function CallingScott explains function calling in AI models like GPT. Function calling allows the AI to call specific user-created functions to interact with the outside world and obtain up-to-date information. Significant challenges and considerations are associated with function calling, such as latency, accuracy, reliability, and security concerns.
Creating a FunctionScott creates the application for demonstrating AI function calling. The advanced-math NPM package is installed and a calculate function is created which will call the math.evaluate method inside the package.
Getting CompletionsScott codes the getCompletion function, creating the completion with a specified GPT model. In addition to the model, temperature and messages, a functions object is included with names, descriptions, and parameters.
Generating Images with DALL·EScott adds an additional function to the application, which allows the AI to generate an image based on a prompt. The function uses the DALL•E OpenAI API to generate the image and return the URL.
Scaling Function CallingScott discusses the challenges of scaling function calling in production.
Wrapping UpScott wraps up the course by providing additional recommendations and suggestions to further explore OpenAI APIs. Topics including diffusion models and training models on GPUs are discussed. Students are encouraged to continue building on the knowledge gained in the course and to experiment with different ideas and applications.