Symbolic AI - The collection of all methods in artificial intelligence research that are based on high-level symbolic (human-readable) representations of problems, logic and search
Generative AI - A subset of artificial intelligence that uses generative models to produce text, images, videos, or other forms of data
Causal AI - A technique in artificial intelligence that builds a causal model and can thereby make inferences using causality rather than just correlation
AI Programming Languages
Mojo - The programming language for all AI developers
Supervised learning - A paradigm in machine learning where algorithms learn from labeled data
Decision tree learning - The method using a decision tree as a predictive model to go from observations about an item to conclusions about the item's target value
Ensemble learning - The method using multiple learning algorithms to obtain better predictive performance than could be obtained from any of the constituent learning algorithms alone
Random forest - An ensemble learning method for classification, regression and other tasks that operates by constructing a multitude of decision trees at training time
Support vector machine - The supervised learning models with associated learning algorithms that analyze data for classification and regression analysis
Classification - The problem of identifying which of a set of categories (sub-populations) a new observation belongs to, on the basis of a training set of data containing observations
Logistic regression - A statistical model that models the probability of an event taking place by having the log-odds for the event be a linear combination of one or more independent variables
ROC curve - A graphical plot that illustrates the diagnostic ability of a binary classifier system as its discrimination threshold is varied
Naive Bayes classifier - A family of simple probabilistic classifiers based on applying Bayes' theorem with strong (naive) independence assumptions between the features
Regression - A set of statistical processes for estimating the relationships between a dependent variable and one or more independent variables
Ordinary least squares - A type of linear least squares method for choosing the unknown parameters in a linear regression model
ARIMA model - A generalization of an autoregressive moving average (ARMA) model, fitted to time series data either to better understand the data or to predict future points in the series
Unsupervised learning - A type of machine learning in which models are trained using unlabeled dataset and are allowed to act on that data without previous training
K-means clustering - A method of vector quantization that aims to partition n observations into k clusters in which each observation belongs to the cluster with the nearest mean
Reinforcement learning - An area of machine learning concerned with how intelligent agents ought to take actions in an environment in order to maximize the notion of cumulative reward
Markov decision process - The mathematical framework for modeling decision making in situations where outcomes are partly random and partly under the control of a decision maker
Multi-armed bandit - A problem in which a fixed limited set of resources must be allocated between competing (alternative) choices in a way that maximizes their expected gain
Value function - A function used in mathematical optimization and reinforcement learning that assigns a measure of desirability to states or actions
Concepts & Techniques
Hyperparameter - A parameter whose value is used to control the learning process
Hyperparameter optimization - The problem of choosing a set of optimal hyperparameters for a learning algorithm
Embedding - A representation learning technique that maps complex, high-dimensional data into a lower-dimensional vector space of numerical vectors
Early stopping - A form of regularization used to avoid overfitting when training a learner with an iterative method, such as gradient descent
Cross-validation - Any of various similar model validation techniques for assessing how the results of a statistical analysis will generalize to an independent data set
Anomaly detection - The identification of rare items, events or observations which raise suspicions by differing significantly from the majority of the data
One-class classification - The technique trying to identify objects of a specific class amongst all objects, by primarily learning from a training set containing only the objects of that class
Recommender system - An information filtering system that seeks to predict the 'rating' or 'preference' a user would give to an item
Related Fields
Mathematical model - An abstract description of a concrete system using mathematical concepts and language
Mathematical optimization - The selection of a best element, with regard to some criteria, from some set of available alternatives
Frameworks, Platforms & Tools
scikit-learn - A free software machine learning library for the Python programming language
Neural network - The computational models used in machine learning for finding patterns in data
Tensor - The mathematical objects represented as multidimensional arrays used in machine learning
Sigmoid function - A mathematical function having a characteristic 'S'-shaped curve or sigmoid curve
Softmax function - A function that converts a vector of K real numbers into a probability distribution of K possible outcomes
Backpropagation - A widely used algorithm for training feedforward neural networks
Autoencoder - A type of artificial neural network used to learn efficient codings of unlabeled data (unsupervised learning)
Vanishing gradient problem - The difficulty encountered when training artificial neural networks with gradient-based learning methods and backpropagation, where gradients shrink as they back-propagate
Deep Learning Concepts & Training
Deep Learning - A part of a broader family of machine learning methods based on artificial neural networks with representation learning
Stochastic gradient descent - An iterative method for optimizing an objective function with suitable smoothness properties
Fine tuning - An approach to transfer learning in which the weights of a pre-trained model are trained on new data
LoRA (machine learning) - A parameter-efficient fine-tuning technique for adapting pre-trained models to specific tasks with significantly fewer computational resources
Key Architectures
Recurrent neural network - A class of artificial neural networks where connections between nodes can create cycles, allowing output from some nodes to affect subsequent input to the same nodes
LSTM - An artificial neural network used in the fields of artificial intelligence and deep learning, distinguished by feedback connections
Attention - A technique in the context of neural networks that mimics cognitive attention, enhancing the important parts of the input data and fading out the rest
Transformer - A deep learning architecture based on the multi-head attention mechanism
Core Frameworks
TensorFlow - An end-to-end open source platform for machine learning
TFDS - The collection of datasets ready to use with TensorFlow or other Python ML frameworks like Jax
Keras - The Python Deep Learning API designed for human beings, not machines
PyTorch - An open source machine learning framework that accelerates the path from research prototyping to production deployment
Deep Learning, MIT Press - The textbook intended to help students and practitioners enter the field of machine learning in general and deep learning in particular
AttentionViz - A Global View of Transformer Attention
BertViz - A tool for visualizing Attention in NLP Models
GraphRAG - A data pipeline and transformation suite that is designed to extract meaningful, structured data from unstructured text using the power of LLMs
Vertex AI - A machine learning (ML) platform for training and deploying ML models and AI applications
Amazon Bedrock - A fully managed service offering a choice of high-performing foundation models
Microsoft Foundry - A platform for building and deploying AI applications, with a portfolio of services and models
Azure OpenAI Service - The service providing REST API access to OpenAI's powerful language models
Azure Machine Learning - An enterprise-grade machine learning service to build and deploy models faster
Amazon SageMaker - The service to build, train, and deploy machine learning (ML) models for any use case with fully managed infrastructure, tools, and workflows
Local LLM Deployment
Ollama - A tool designed for deploying and managing large language models (LLMs) locally
LM Studio - A desktop app for developing and experimenting with LLMs locally on your computer
LocalAI - The free, Open Source OpenAI alternative
Standards
Model Formats
GGUF - A file format for storing models for inference with GGML and executors based on GGML
ONNX - An open format built to represent machine learning models
Safetensors - A simple format for storing tensors safely