An A-Z glossary of artificial intelligence terms and definitions

 

 

 

Annotation: The process of tagging language data by identifying and highlighting any grammatical, semantic, or phonetic elements in language data.

 

 

Adaptive learning: Used in generative AI to define a model that adjusts learning processes based on feedback it receives. Adaptive learning models enhance their ability to generate accurate outputs.

 

 

Adversarial example: Inputs deliberately designed to confuse or tick a generative AI model into producing incorrect outputs. They are often indiscernible to the human eye.

 

 

Adversarial network: Often referred to as Generative Adversarial Networks (GANs). That means two neural networks (a generator and a discriminator) are trained simultaneously and competitively to improve the quality of generated outputs.

 

 

Algorithm: An algorithm is an exact list of instructions that conduct specified actions step-by-step in software- or hardware-based routines. Algorithms are often used as specifications for performing data processing and play a major role in automated systems.

 

 

Algorithmic bias: Systematic errors in an algorithm that generates unfair outcomes, often privileging one group of users over another.

 

 

Algorithmic creativity: Describes the capability of generative AI systems to create art, music, literature, or other creative works without direct human input.

 

 

Application programming interface (API): Defined rules that enable applications to communicate with each other. It acts as an intermediary layer that processes data transfers between systems, letting companies open their application data and functionality to third parties.

 

 

Artificial intelligence (AI): The simulation of human intelligence in machines that are programmed to think like humans. Check out: An ultimate guide to artificial intelligence.

 

 

Attention mechanism: A technique used in generative models, especially language generation, which helps focus on parts of input date, improving the relevance and coherence of generated content.

 

 

Autoencoder: A neural network used in generative AI that is trained to encode input data as representations, then decode representations back to the original data.

 

 

Auto-classification: Using ML, NLP, and alternative AI-guided techniques to classify text in an increasingly fast, cost-effective, and accurate manner.

 

 

Autonomous generation: The ability of a generative AI system to create content or make decisions independently, without specific guidance or rules set by humans.

 

 

B

 

Batch normalisation: A supervised learning method of training neural networks. It works by standardising inputs to a network, which stabilises learning, saves time, and improves accuracy.  Batch normalisation can be quite complex, so for more details check out: Batch normalisation guide.

 

 

Bias: Systematic errors in data or the model that can lead to skewed or prejudiced outputs. Platforms must address bias to ensure fairness and accuracy in generated content.

 

 

Bidirectional encoder representations from transformers (BERT): A transformer-based ML technique for NLP. BERT is often the baseline of NLP experiments after being introduced in 2018 by researchers at Google. BERT was pretrained on language modelling and next sentence prediction.

 

 

Big data: Describes large volumes of data, structured and unstructured, that are typically too large or complex for traditional data-processing software. For more, check out: A guide to using big data.

 

 

Binary classification: A machine learning algorithm that categorises new observations into two classes, such as taking a generated image and determining whether real or fake.

 

 

Blending: The technique of combining features or aspects from multiple sources or datasets to create new, hybrid outputs. Typically used in generative AI, blending can lead to deeper and more complex forms of analyses. Altair provide a great explanation: What is Data Blending?

 

 

C

 

Categorisation: Categorisation is an NLP function that assigns a category to a document.

 

 

Chatbot: A computer programme designed to simulate conversation with human users, typically employed on the internet and used to streamline customer services. For more information, check out: How to humanise a chatbot.

 

 

Clustering: The grouping of similar data points in a dataset. Often used in generative AI to understand underlying patterns in the data, which can inform the generation of new, similar data.

 

 

Composite AI: The combined application of AI techniques to broaden the level of knowledge representations and to solve a wider range of problems in a more efficient manner. Check out Gartner’s helpful definition for more detail: Composite AI.

 

 

Computational creativity: The study and development of AI systems that can perform tasks traditionally considered to require creativity, such as composing music, painting, writing poetry, and so on. The field encompasses both the theoretical and practical aspects of creativity in AI.

 

 

Computational linguistics: Computational linguistics is an interdisciplinary field concerned with the computational modelling of natural language. Computational linguists apply skills to develop and analyse computer applications of translation, voice recognition, text analysis, and so on.

 

 

Computational semantics: Computational semantics studies the automation of the construction and reasoning of meaning representations of natural language expressions. Semantics is concerned with computing the meaning of linguistic objects and combines insights from computational linguistics.

 

 

Computer vision: A field of computer science that focuses on enabling computers to identify and understand objects and people in images and videos. For a more detailed explanation, check out Microsoft’s helpful guide: What Is Computer Vision?

 

 

Conditional generation: A process in generative AI where the model generates data based on certain conditions or parameters, such as generating images of a specific category.

 

 

Content generation: The use of AI, particularly generative models, to create content, including text, images, music, and videos. For a guide, check out: How to master generative AI content.

 

 

Contextual embedding: The representation of words or features in the context of their surrounding words or features, improving the relevance and coherence of the generated content.

 

 

Conversational AI: Used by developers to build conversational user interfaces, chatbots, and virtual assistants. They offer integration into chat interfaces such as messaging platforms, SMS, and websites.

 

 

Convolutional neural network (CNN): A type of deep neural network commonly used in generative AI, especially for image generation and analysis. CNNs can detect patterns and features in images.

 

 

Creativity metrics: Measures used to assess the novelty, quality, and diversity of the content produced by AI models.

 

 

Curriculum learning: A training strategy in generative AI where models are gradually exposed to increasingly complex data, improving their ability to generate complex outputs.

 

 

Cycle-consistency loss: A concept used in Generative Adversarial Networks, particularly in tasks like image-to-image translation. It ensures that the original input can be recovered after a series of transformations.

 

 

D

 

Data: Values that convey information, such as quantity, quality, facts, or other meaning. For more information, and to understand the analysis of data, check out: Data analytics 101.

 

 

Data augmentation: The process of increasing the amount and diversity of data by modifying existing datasets. This is particularly useful in generative AI for improving model performance and robustness.

 

 

Dataset: A collection of data derived from a single source or collected for a single project.

 

 

Decoder: In the context of generative models, a decoder is a network or a part of a network that converts compressed data (like feature vectors) back into a more readable or usable format.

 

 

Deepfake: Videos in which the face and/or voice of a person, most commonly a public figure, has been manipulated by AI software in a way that makes the video seem authentic. See: What are deepfakes and how to protect against them.

 

 

Deep learning (DL): A subset of machine learning involving neural networks with many layers. DL is integral to generative AI models, especially in processing complex inputs like images and natural language.

 

 

Denoising: A technique in generative AI where the model learns to remove noise or imperfections from data, often used in image and audio processing.

 

 

Did you mean (DYM): DYM is an natural language processing function [JS1] used in search applications to identify typos in a query or suggest similar queries that could produce results in the search database being used.

 

 

Digital twin: A digital twin uses real world data to create simulations that can predict how a product or process will perform.

 

 

Discriminator: In Generative Adversarial Networks, the discriminator is the part of the network that distinguishes between real and generated data, helping to improve the quality of the generated output.

 

 

Disentangled representation: A representation where individual dimensions capture single, independent factors of variation in the data, aiding in more controlled generation.

 

 

Distribution matching: A technique in generative AI where the model learns to generate data that matches the distribution of the training dataset, ensuring realistic and diverse output.

 

 

Domain adaptation: The ability of a generative model to adapt to new, unseen domains of data, maintaining performance even when the new data differs from the training data.

 

 

Dynamic time warping (DTW): An algorithm in generative AI for measuring similarity between two temporal sequences, which may vary in speed. It is particularly useful in speech recognition.

 

 

E

 

Elman network: A type of recurrent neural network where connections between units form a directed cycle. The architecture is useful in generative models for tasks that involve sequential data, like language modelling.

 

 

Embedding: A representation of data where elements with similar meanings are mathematically close to each other. In generative AI, embeddings are crucial for handling complex data like text and images.

 

 

Encoder: In generative AI, an encoder is a part of a model, often a neural network, which compresses data into a more compact representation, typically used in autoencoder architectures.

 

 

Encryption: The process of protecting information or data by using mathematical models to scramble it in such a way that only the parties who have the key to unscramble it can gain access.

 

 

Ensemble learning: A technique where multiple models are trained and predictions are combined. In generative AI, ensemble learning can improve the quality and reliability of the generated content.

 

 

Entropy: The measure of randomness or uncertainty in the model’s predictions. It is a key concept in understanding and controlling the diversity of generated content.

 

 

Epoch: An epoch in machine learning, including generative AI, is one complete pass through an entire training dataset. More epochs lead to better learning but can increase the potential of overfitting.

 

 

Evolutionary algorithm: A subset of AI algorithms inspired by the process of natural selection. These algorithms iteratively select, mutate, and breed candidate solutions. They are sometimes used in generative AI to evolve novel solutions to problems.

 

 

Explainable AI: Branch of AI focused on making the decision-making processes of AI systems transparent and understandable to humans. Explainable AI is becoming more important in generative AI models, as people require trust and transparency. For more, see: The importance of ethics.

 

 

Extrapolation: The generative AI model’s ability to generate outputs or make predictions for data points that lie outside the range of its training data.

 

 

F

 

Feature extraction: The process of identifying and isolating significant patterns or attributes in input data. In generative AI, extraction is crucial for understanding and reproducing complex data structures.

 

 

Feedforward neural network: A type of neural network where connections between nodes do not form cycles. The architecture is commonly used in simpler generative models.

 

 

Fine-tuning: The process of making adjustments to pre-trained models to adapt them to specific tasks.

 

 

Fitness function: In evolutionary algorithms, a fitness function quantitatively evaluates how close a given design solution is to achieving the set aims. In generative AI, this concept can be used to guide the generation process towards desired outcomes.

 

 

Forward propagation: The process in neural networks, including generative models, in which inputs are passed forward through the network layers to generate an output.

 

 

Frame generation: The use of generative AI to create individual frames in video and animation, often for the purpose of creating smooth transitions or filling in missing frames.

 

 

Fuzzy logic: A form of many-valued logic where truth values can be any real number between 0 and 1, representing degrees of truth. In generative AI, it is used to handle imprecise or uncertain information. For more information, check out: A very brief introduction to Fuzzy Logic and Fuzzy Systems

 

 

G

 

Generative adversarial network (GAN): Neural network architecture used in unsupervised machine learning, involving two parts: the generator, which generates data, and the discriminator, which evaluates it.

 

 

Genetic algorithm: A type of evolutionary algorithm that mimics the process of natural selection, using methods like mutation, crossover, and selection to optimise problems.

 

 

Generative AI: An AI model that can generate new data instances. These models learn to capture the probability distribution of the input data so they can produce data similar to their training data.  

 

 

Generative pre-trained transformer (GPT): A type of language processing AI model designed to generate human-like text. It is perhaps the most prominent example of a generative AI model.

 

 

Graph neural network (GNN): A type of neural network which directly runs on the graph structure. GNNs are used in generative models that deal with graph-structured data.

 

 

Ground truth: The accuracy of the training set’s classification for supervised learning. It is the benchmark against which the performance of generative models is measured.

 

 

H

 

Haptic technology: Tech that simulates the sense of touch through force, vibrations, or motions to the user. Often used in generative AI to create virtual environments or to enhance user interactions.

 

 

Hashing: Converts input data into a fixed-size string of bytes, typically used for data retrieval. Hashing can play a role in generative AI for data indexing or ensuring the uniqueness of generated content.

 

 

Hebbian learning: A theory of how brain neurons adapt during learning process, often summarised as “cells that fire together wire together.” Inspires approaches and algorithms in neural network training.

 

 

Heuristic: An approach to problem-solving that may not be perfect or best but is sufficient for reaching an immediate goal. In generative AI, heuristics can guide the generation process or model training.

 

 

Hill-climbing algorithm: A mathematical optimisation technique that starts with an arbitrary solution to a problem and iteratively makes minor changes to the solution, each improving it a bit more.

 

 

Hyperplane: In machine learning, including generative AI, a hyperplane is a decision boundary that helps to classify data points. It is especially relevant in models that involve classification tasks.

 

 

I

 

Image-to-image translation: A process in generative AI where the model transforms an input image into an output image, keeping the core structure but altering its style or specific characteristics.

 

 

Imbalanced data: Situations in machine learning and generative AI where some classes of data are much more frequent than others. Handling imbalanced data is crucial for training effective generative models.

 

 

Inference: The process in generative AI where a trained model applies what it has learned to new data, generating predictions or outputs based on its training.

 

 

Information bottleneck: A method used in deep learning and generative AI to find a compact representation of the input data that preserves the most relevant information about the output task.

 

 

Intelligent agent: An autonomous entity in AI that sees and acts upon an environment and directs its activity towards achieving goals. In generative AI, such agents can be designed to generate responses or actions based on environmental inputs. For more: What is an intelligent agent?

 

 

Instance-based learning: When an AI model memorises training examples and uses them to make predictions. The approach can be utilised in generative AI for applications where examples resemble test cases.

 

 

Isomorphic mapping: The creation of a one-to-one correspondence between elements of two sets, which can be used in tasks like graph generation and structure-preserving transformations.

 

 

Integration testing: In the context of generative AI development, refers to testing where individual units or components of a model are combined and tested as a group to evaluate their interactions.

 

 

Interpolation: AI model generating data that falls within the range of its training dataset. Opposed to extrapolation where the model generates data outside the range of its training data.

 

 

Iterative learning: An approach where the model gradually improves over time, often by repeatedly processing data and refining algorithms. Common method for generative models.

 

 

J

 

Jensen-Shannon divergence: Measures similarity between two probability distributions. Used in generative AI to analyse difference between the generated data distribution and real data distribution.

 

 

Jitter: A technique used in data augmentation where small, random perturbations are added to data (like images or audio) to create a more robust dataset for training generative models.

 

 

Joint distribution: The probability distribution over all combinations of values in a set of variables. Important for generating data that captures the relationships between variables.

 

 

Joint learning: A training approach where a model is trained to perform multiple tasks simultaneously, which can lead to more generalised and robust generative models.

 

 

Jump connections: Also known as skip connections, these are connections in a neural network that skip one or more layers. In generative models, they help in preserving information and mitigating the vanishing gradient problem.

 

 

Jupyter notebook: Part of Project Jupyter, Jupyter notebook is an open-source web application that allows the creation and sharing of documents containing live code, equations, visualisations, and narrative text. It is widely used in generative AI for experimentation, data analysis, and model development. See more: How to Use Jupyter Notebooks: The Ultimate Guide.

 

 

Just-in-time compilation: The compilation of certain parts of the code during runtime, which can optimise the performance of AI models, particularly in terms of speed and resource usage.

 

 

K

 

K-means clustering: A method of vector quantisation used for cluster analysis in data mining, aiming to partition data points into clusters with similar characteristics.

 

 

K-nearest neighbours: A simple, non-parametric algorithm used for classification and regression in machine learning, often applied in generative AI for tasks like anomaly detection.

 

 

Keras: A high-level neural networks application programming interface in Python, used for prototyping deep learning models, including generative models like Generative Adversarial Networks and Variational Autoencoders (VAEs).

 

 

Kernel: In machine learning, a kernel is a function used to transform data into a higher-dimensional space, enabling better classification when data is not linearly separable in the original space.

 

 

Keyframe: In animation and film, a keyframe defines the starting and ending points of a smooth transition. Keyframes can be used in generative AI for animation and frame interpolation.

 

 

Kullback-Leibler divergence: A measure of divergence between probability distributions, often used in generative AI to quantify the difference between generated and training data distributions.

 

 

Knowledge base: A collection of data, information, and rules used for knowledge management. Generative AI can utilise knowledge bases to generate contextually relevant outputs.

 

 

Knowledge extraction: The process of extracting valuable knowledge from large datasets, a technique used in generative AI to improve understanding and generation capabilities.

 

 

Knowledge graph: A network of real-world entities and their relationships, used in generative AI for context-aware content generation.

 

 

Knowledge representation: The way in which intelligent agents, like generative models, represent knowledge about the world to make decisions or generate content.

 

 

L

 

Label smoothing: A regularisation technique used in training generative AI models to prevent overconfidence in predictions by penalising overly confident outputs, encouraging more diverse and calibrated predictions.

 

 

Language generation: The process of using generative AI to create human-like text or speech, often employed in chatbots, virtual assistants, and in content generation.

 

 

Large language model (LLM): Type of AI algorithm that uses deep learning techniques and massively large data sets to understand, summarise, generate, and predict new content. For more information, check out: What’s a large language model?

 

 

Latent space: The abstract, high-dimensional space in which the model represents data. It is essential in models like Generative Adversarial Networks  and variational autoencoders.

 

 

Layer normalisation: A technique used to normalise activations in neural network layers. It can improve the stability and training speed of generative models.

 

 

Leveraged generative models: These models combine generative AI techniques with reinforcement learning, enabling the generation of sequences with specific desired characteristics.

 

 

Linear regression: Statistical technique used for modelling the relationship between a dependent variable and independent variables. While not generative, it is a foundational concept in machine learning.

 

 

Log-likelihood: A measure of how well a generative model’s predictions match the actual data distribution. It is used to assess the quality of generated content.

 

 

Logistic regression: Statistical model used in classification, where the output is binary. While not generative in nature, logistic regression can be part of pipelines that include generative models.

 

 

Loss function: A mathematical function used in training generative AI models to quantify the difference between the model’s predictions and the actual data. It guides the model towards better performance.

 

 

Long short-term memory (LSTM): Recurrent neural network architecture well-suited for sequential data, making it valuable in tasks like text generation, language modelling, and speech recognition.

 

 

M

 

Machine Learning (ML): Use and development of computer systems able to learn and adapt without following explicit instructions, by using algorithms and statistical models to analyse and draw inferences from patterns in data. For more information, check out: What is Machine Learning?

 

 

Machine translation: The task of automatically translating text or speech from one language to another using generative AI models, such as neural machine translation systems.

 

 

Markov chain: A stochastic model describing a sequence of events where the probability of each event depends only on the state attained in the previous event. Markov chains are used in generative models.

 

 

Memory augmented neural networks: Neural network architectures that incorporate external memory modules, allowing them to store and access information over extended sequences. These networks are valuable in generative tasks with long-term dependencies.

 

 

Meta-learning: A machine learning technique where a model learns how to learn. In generative AI, meta-learning can help models adapt quickly to new tasks or generate content with diverse styles.

 

 

Minibatch: A subset of data used during training in generative AI. Minibatch training is an efficient way to update model parameters and manage large datasets.

 

 

Mixture density network (MDN): A neural network architecture used for modelling probability distributions over continuous variables. MDNs are employed in generative AI to manage uncertainty and generate probabilistic outputs.

 

 

Motion capture: A technology used to record the movement of objects or humans. In generative AI, motion capture data can be used to create realistic animations and simulations.

 

 

Model ensemble: A technique in which multiple generative AI models are combined to improve performance. Ensemble methods can enhance the diversity and quality of generated content.

 

 

Multi-modal generation: The generation of content that involves multiple modes of input or output, such as generating both text and images together. Multimodal generative AI models are used in tasks like image captioning.

 

 

Multi-task prompt tuning (MPT): An approach that configures a prompt representing a variable — that can be changed — to allow repetitive prompts where only the variable changes.

 

 

N

 

Natural language processing (NLP): A field of AI focussing on the interaction between computers and human language, enabling generative AI models to understand, generate, and manipulate natural language text.

 

 

Natural language understanding (NLU): The ability of generative AI models to comprehend and interpret human language, enabling them to generate contextually relevant responses and content.

 

 

Natural language generation (NLG): Solutions that automatically convert structured data, such as that found in a database, an application, or a live feed, into a text-based narrative.

 

 

Negative sampling: A technique used in training generative AI models to sample negative examples, improving their ability to distinguish between positive and negative instances.

 

 

Neighbourhood generation: A process in generative AI where data points are generated in the vicinity of existing data points, often used for data augmentation, and enhancing model robustness.

 

 

Neural architecture search (NAS): A method in which generative AI models are used to automatically search for the optimal neural network architecture for a given task.

 

 

Neural differential equations: A framework that uses neural networks to solve differential equations.

 

 

Neural network: Neural networks are a subset of machine learning that rely on training data to learn and improve accuracy over time, allowing models to classify and cluster data at a high velocity.

 

 

Neural style transfer: A technique in generative AI that combines the content of one image with the artistic style of another, producing visually appealing images that merge content and style.

 

 

Neuromorphic computing: A type of computing inspired by the structure and function of the human brain. In generative AI, neuromorphic computing can lead to more efficient and brain-like AI systems.

 

 

Noisy data: Data that contains errors, outliers, or irrelevant information. Managing noisy data is essential in generative AI to ensure the quality of generated content.

 

 

Non-Markovian models: Generative AI models that do not rely on the Markov property, allowing them to capture more complex dependencies between data points.

 

 

O

 

Objective function: A mathematical function that the generative AI model aims to optimise. It quantifies the model’s performance and guides it towards better generation.

 

 

Object detection: A computer vision task where generative AI models identify and locate objects within images or videos, often used in applications like autonomous driving and image captioning.

 

 

Ontology: A formal representation of knowledge or concepts and their relationships in a specific domain. Generative AI can use ontologies to generate more contextually relevant content.

 

 

OpenAI: A research organisation and company known for developing innovative AI models, including GPT (Generative Pre-trained Transformer) models. OpenAI has made significant contributions to the field of generative AI.

 

 

Optical character recognition (OCR): Tech used in generative AI to convert printed or handwritten text into machine-readable text, allowing models to generate or process textual data from images.

 

 

Oracle model: An oracle model is a hypothetical model that has access to perfect information or an idealised version of the training data. It serves as a benchmark for evaluating generative models.

 

 

Out-of-distribution (OOD) detection: Technique in generative AI used to identify data points that are different from the training data. OOD detection helps improve the reliability of generative models.

 

 

Overfitting: An issue in machine learning where a generative AI model learns to perform well on the training data but fails to generalise to new, unseen data. Preventing overfitting is crucial for generative model performance.

 

 

Overparameterisation: A strategy in deep learning where a generative AI has more parameters than necessary. It can lead to improved model performance but may require careful handling to prevent overfitting.

 

 

P

 

Parallel computing: A method of performing multiple computations simultaneously, used in generative AI to speed up training and inference processes by leveraging multiple processors or graphics processing units.

 

 

Parallel data: In generative AI for translation and language tasks, parallel data consists of text or speech in one language and its corresponding translation or interpretation in another language.

 

 

Perception network: A neural network component in generative AI models that processes sensory inputs, such as images or audio, and extracts relevant features for further processing or generation.

 

 

Pixel recurrent neural network (Pixel RNN): A type of generative AI model that generates images pixel-by-pixel, considering the dependencies between pixels in the image.

 

 

Policy gradient methods: A class of reinforcement learning algorithms used to train models to make sequential decisions and generate content, often employed in tasks like game playing and robotics.

 

 

Pre-trained model: A generative AI model that has been trained on a large dataset for a specific task and can be fine-tuned or used as a starting point for related tasks.

 

 

Principal component analysis (PCA): A dimensionality reduction technique used in generative AI for data compression and feature extraction, particularly in image and video generation.

 

 

Probabilistic graphical models: A framework that represents probabilistic relationships between variables using graphs, commonly used in generative AI for modelling complex dependencies.

 

 

Progressive GAN (ProGAN): An improvement of the Generative Adversarial Network architecture that gradually increases the resolution of generated images during training, resulting in high-quality images.

 

 

Pruning: A technique in generative AI model optimisation that involves removing unnecessary or low-impact neural network connections or parameters to reduce model size and improve efficiency.

 

 

Q

 

Quality assessment: Quality assessment in generative AI involves evaluating the quality of generated content, such as images, text, or audio, to ensure that it meets certain criteria and standards.

 

 

Quantisation: The process of mapping continuous values to a discrete set of values. Often used in tasks like image compression, where continuous colour values are mapped to a limited palette of colours.

 

 

Query expansion: A technique used in information retrieval and generative AI to improve search results. Involves expanding queries with related terms or synonyms to retrieve relevant information.

 

 

QuickDraw dataset: A publicly available dataset containing millions of hand-drawn sketches contributed by users. Often used in for tasks like sketch generation and recognition.

 

 

R

 

Random forest: A machine learning ensemble method that combines multiple decision trees to improve generative AI model performance, particularly in classification and regression tasks.

 

 

Reciprocal recommender: A technique that considers both user preferences and item properties in recommendation systems, enhancing the quality of recommendations.

 

 

Recommender system: A system that provides personalised recommendations to users, commonly used in applications like movie recommendations and product suggestions.

 

 

Recurrent generative model: A type of generative AI model that incorporates recurrent layers to generate sequences of data, making it suitable for tasks like text generation and music composition.

 

 

Recurrent neural network (RNN): A type of neural network architecture used in generative AI for processing sequential data, making it suitable for tasks like text generation and speech recognition.

 

 

Reinforcement learning (RL): A machine learning paradigm where agents learn to make decisions through interactions with an environment. Used in generative AI for tasks like game playing and robotic control.

 

 

Representation learning: The process of learning meaningful representations of data that can be used for generative AI tasks. It involves capturing essential features from raw data.

 

 

Residual network (ResNet): A deep neural network architecture known for its skip connections, allowing it to train very deep models. ResNet is used in generative AI for various computer vision tasks.

 

 

Restricted Boltzmann machine (RBM): A generative AI model used in unsupervised learning for feature learning and data representation. RBMs are building blocks in deep belief networks.

 

 

Retrieval Augmented Generation (RAG): Retrieval-augmented generation (RAG) is an AI technique for improving the quality of LLM-generated responses by including trusted sources of knowledge, outside of the original training set, to improve the accuracy of the large language model’s output.

 

 

Robustness: The ability of generative AI models to perform consistently and accurately in the presence of noise, variations, or adversarial inputs, ensuring reliable content generation.

 

 

S

 

Self-attention mechanism: A key component in transformers, a type of neural network architecture used in generative AI, which enables models to weigh the importance of distinct parts of input sequences when generating content.

 

 

Self-supervised learning: A variant of supervised learning in generative AI where models are trained to predict certain parts of their input data without explicit labels, often used for pre-training.

 

 

Semantic segmentation: A computer vision task in generative AI where models classify each pixel in an image into a specific category, enabling tasks like object recognition and scene understanding.

 

 

Sentence embeddings: In generative AI, these are fixed-length representations of sentences, often used to capture semantic information for various natural language processing tasks.

 

 

Sentiment analysis: Sentiment analysis is an natural language processing function that identifies the sentiment in text. This can be applied to anything from a business document to a social media post. Sentiment is typically measured on a linear scale (negative, neutral, or positive), but advanced implementations can categorize text in terms of emotions, moods, and feelings.

 

 

Seq2Seq (Sequence-to-Sequence): A neural network architecture used in generative AI for tasks that involve sequential data, such as machine translation and text summarisation.

 

 

Stochastic gradient descent (SGD): An optimisation algorithm used in training generative AI models that iteratively adjusts model parameters to minimise a loss function.

 

 

Style transfer: A generative AI technique that involves altering the style of an input, such as converting a photograph into the style of a famous artist’s painting.

 

 

Style-generative adversarial network (StyleGAN): A generative AI model known for its ability to generate high-quality images with fine-grained control over style attributes.

 

 

Subject-Action-Object (SAO): SAO is an natural language processing function that identifies the logical function of portions of sentences in terms of the elements that are acting as the subject of an action, the action itself, the object receiving the action (if one exists), and any adjuncts if present.

 

 

Supervised learning: A machine learning paradigm where generative AI models are trained on labelled data, where each input is associated with a corresponding desired output or label.

 

 

Symbolic methodology: A symbolic approach designs a system using specific, narrow instructions that guarantee the recognition of a linguistic pattern. Rule-based solutions tend to have a high degree of precision, though they may require more work than machine learning-based solutions to cover the entire scope of a problem, depending on the application.

 

 

Synthetic data: Artificially generated data used in generative AI for tasks like data augmentation, model training, and improving model robustness.

 

 

T

 

Temporal generative models: Generative AI models designed to oversee sequential data with a temporal aspect, such as time series forecasting and video generation.

 

 

TensorFlow: An open-source machine learning framework developed by Google that is widely used in generative AI for model development and training.

 

 

Text generation: A generative AI task where models generate human-readable text based on input data or prompts. Text generation is used in applications like chatbots, content generation, and language translation.

 

 

Text-to-Speech (TTS): A technology in generative AI that converts text into spoken language, enabling the generation of natural-sounding speech from written text.

 

 

Tokens: A unit of content corresponding to a subset of a word. Tokens are processed internally by large language models and can also be used as metrics for usage and billing.

 

 

Tokenisation: The process of splitting text into individual units, or tokens, which can be words, subwords, or characters. Tokenisation is a crucial step in generative AI for text processing.

 

 

Topic modelling: A generative AI technique used to identify topics or themes within a collection of documents. It is commonly applied in tasks like document clustering and content recommendation.

 

 

Top-k sampling: A text generation strategy in generative AI where the model selects from the top-k next tokens at each step, controlling the diversity of generated text.

 

 

Top-p (nucleus) sampling: A text generation strategy in generative AI where the model selects from a subset of the tokens with cumulative probability less than or equal to a threshold “p.” This strategy also controls text diversity.

 

 

Transfer learning: A technique in generative AI where a pre-trained model is fine-tuned for specific tasks or domains. Transfer learning allows models to leverage knowledge from one task to improve performance on another.

 

 

Treemap: Display copious amounts of hierarchically structured (tree-structured) data. The space in the visualisation is split up into rectangles that are sized and ordered by a quantitative variable. The levels in the hierarchy of the treemap are visualised as rectangles containing other rectangles.

 

 

Transformer: A neural network architecture widely used in generative AI for various natural language processing tasks, known for its self-attention mechanism and ability to capture contextual information in text.

 

 

Turing test: A test for intelligence that requires that a human being should be unable to distinguish the machine from another human being by using the replies to questions put to both. For more information, check out: What is the Turing Test?

 

 

U

 

Uncertainty estimation: The process of quantifying the uncertainty associated with model predictions. It is crucial for understanding the reliability of generated content.

 

 

Underfitting: A common issue in machine learning where a generative AI model is too simple to capture the underlying patterns in the data, resulting in deficient performance.

 

 

Universal approximation theorem: A mathematical concept stating that neural networks with enough parameters can approximate any continuous function.

 

 

Unsupervised learning: A machine learning paradigm where models learn patterns and structures in data without the use of explicit labels. It is often used for tasks like clustering and dimensionality reduction.

 

 

Upsampling: A technique in generative AI that involves increasing the resolution or size of data, such as images or audio, often used in tasks like image generation and super-resolution.

 

 

V

 

Variational autoencoder (VAE): A model combining elements of autoencoders and variational inference to learn probabilistic representations of data, often used for generating new samples.

 

 

Vector quantisation (VQ): A technique used to discretise continuous data, commonly applied in tasks like speech and image compression.

 

 

Virtual reality (VR) simulation: The use of generative AI to create immersive and interactive virtual environments, enabling experiences in virtual reality.

 

 

Visual question answering (VQA): A task where models answer questions about images or visual data, requiring both vision and language understanding.

 

 

Voice cloning: A technology in generative AI that allows for the replication of a person’s voice, often used in applications like voice assistants and voice synthesis.

 

 

W

 

Wave Generative Adversarial Networks (WaveGAN): WaveGAN is a generative AI model designed for generating audio waveforms, particularly in tasks like speech synthesis and music generation.

 

 

Weakly supervised learning: A type of learning in generative AI where models are trained with limited or noisy supervision, often used when obtaining precise labels or annotations is challenging.

 

 

Weight initialisation: The process of setting initial values for neural network weights. Proper weight initialisation can significantly impact the training and performance of generative AI models.

 

 

Windowing: A method that uses a portion of a document as metacontext or metacontent.

 

 

Word2Vec: A technique used to learn word embeddings from large text corpora. It is commonly used for capturing semantic similarities between words.

 

 

Word Embeddings: Vector representations of words that capture semantic relationships between words in a continuous vector space. They are essential for various natural language processing tasks.

 

 

X

 

Xavier initialisation: A weight initialisation technique in generative AI that helps set initial weights in neural networks to ensure efficient training and improved model convergence.

 

 

XML generation: XML generation involves creating structured data in XML format from various sources, such as text or databases, for tasks like document generation and data interchange.

 

 

XOR problem: A classic problem in machine learning and generative AI where models need to learn to perform an exclusive OR operation on binary inputs, often used to evaluate the capabilities of neural networks.

 

 

Y

 

YAML ain’t markup language (YAML): YAML is a human-readable data serialisation format used in generative AI and other fields to represent data structures in a readable and concise manner.

 

 

Yield: The process of producing or generating output, often used in the context of code or functions that generate data or results.

 

 

You only look once) (YOLO): YOLO is a real-time object detection algorithm in generative AI that can detect and locate multiple objects within images or video frames with high accuracy and speed.

 

 

Yule-Simon distribution: A probability distribution in generative AI to model distribution of number of attempts required to observe a new distinct event, often used in certain text generation models.

 

 

Z

 

Z-score (standard score): A statistical measure used in generative AI for data normalisation and outlier detection by measuring how many standard deviations a data point is from the mean.

 

 

Zero-inflated model: A statistical model used in generative AI to account for data with an excess of zero values, often applied in count data and rare event prediction.

 

 

Zero-shot image generation: An advanced generative AI task where models are trained to generate images of objects or concepts they have never seen during training.

 

 

Zero-shot learning: A machine learning and generative AI technique where models are trained to recognize and generate data or perform tasks for which they have not seen examples during training.