These are questions I’ve had, or that I thought others who are exploring AI and the most recent advances in language models. What I hope these become are answers put into the simplest possible terms while backfilling what is often presumed to be common knowledge in the field.

Each section will be kept short, and then link to more details elsewhere.

Transformers

1. What is the transformer model?

Vector databases

1. What is a vector database?
2. What are word vectors?
3. What is one-hot encoding?

Embeddings

1. What are embeddings?
2. How are embeddings stored?
3. What does it mean to shrink an embedding?

Attention

1. What is attention?
2. What is an attention head?
3. Why do we need multiple attention heads?

Mamba

1. What is Mamba?

What is NLP?

1. What is a token?