7 concepts behind large language models are explained in 7 minutes
Image of the author | Idiom
If you’ve been using large language models like GPT-4 or Claude, you might be wondering how they write actual code, explain complex topics, and even help you debug your morning coffee routine (just kidding!).
But what actually happened under the hood? How do these systems turn simple cues into sometimes almost human-centric, contextual responses?
Today, we will learn more about the core concepts that make the big language model work. Whether you’re a developer who integrates LLM into your application, a product manager trying to understand the features and limitations, or just a curious person, this article is for you.
1. Tokenization
Before any text reaches a neural network, it must be converted to a numerical representation. Tokenization is this translation process, which is more complex than splitting on spaces or punctuation only.
Tokenizer uses algorithms such as byte pair encoding (BPE), text, or sentences to create vocabulary to balance efficiency and representation quality.
Image of the author | picture (draw.io)
These algorithms construct sub-word vocabulary by starting with a single character and gradually combining the most common pairs. For example, “unhappiness” may be symbolized [“un”, “happy”, “ness”]allowing the model to understand prefix, root and suffix respectively.
This subword method solves many basic problems. It processes words outside of singing by breaking it down into known works. It manages a rich morphological language in which words have many variations. Most importantly, it creates fixed vocabulary sizes, a model that is often available for modern LLM’s 32K to 100K tokens.
The tokenization method determines the efficiency of the model and the calculation cost. Effective tokenization shortens the sequence length, thus reducing processing requirements.
GPT-4’s 8K context window allows 8,000 tokens, which is approximately equal to 6,000 words. When you build applications that handle long documents, token counting is critical to managing costs and maintaining limitations.
2. Embed
You may have seen articles or social media posts about embedding and popular embedding models. But what are they? Embedding converts discrete tokens into vector representations, usually in hundreds or thousands of dimensions.
This is where things get interesting. Embedding is a dense vector representation that also captures semantic meanings. Embeddings place them in a multidimensional space where similar concepts are brought together, rather than treating words as arbitrary symbols.
Image of the author | picture (draw.io)
Imagine a map where “king” and “queen” are close neighbors, but “king” and “bike” are intercontinental separated. Essentially, this is the appearance of the embedded space, just that hundreds or thousands of dimensions occur at the same time.
Embedding is your secret weapon when you build a search feature or recommendation system. Two texts with similar embeddings are semantically related, even if they do not share exact words. This is why modern search can understand that “car” and “car” are essentially the same thing.
3. Transformer architecture
Transformer architecture revolutionized natural language processing (yes, literally!) brought attention. Instead of processing text in sequence, transformers can view all parts of a sentence at the same time and find out which words are most important to know each other, instead of processing the text in sequence, you can view all parts of a sentence at the same time.
When dealing with “the cat sits on the mat because it is comfortable”, attention mechanism helps the model understand that “it” refers to “mat”, not “cat”. This strengthens the connection between related words through learning focus weights.
For developers, this translates into a model that can handle remote dependencies and complex relationships within text. This is why modern LLMs can maintain coherent dialogue between multiple paragraphs and understand the context across the entire document.
4. Training phase: Pre-training and fine-tuning
LLM development takes place at different stages, each stage has a different purpose. Language models learn patterns from large datasets through pre-training, which is an expensive and computationally intensive phase. Think of it as teaching a model to understand and generate human language.
Next is fine-tuning, where you will pre-train models specifically for specific tasks or domains. Rather than learning the language from scratch, you teach a model that already has the capability to do well in specific applications such as code generation, medical diagnosis, or customer support.
Image of the author | picture (draw.io)
Why is this method effective? You don’t need a lot of resources to create a powerful professional model. The company is building domain-specific LLMs by fine-tuning existing models using its own data and achieving impressive results with relatively moderate computational budgets.
5. Context window
Each LLM has a context window – the maximum amount of text that can be considered at once. You can conceptualize it as an operational memory of the model. From a model perspective, nothing except this window exists.
This can be very challenging for developers. How do you build a chatbot that remembers conversations with multiple sessions when the model itself does not have persistent memory? How do you deal with documents that are longer than context windows?
Some developers maintain conversation summary, sending them back to the model to maintain the context. But that’s just one way to do that. Here are some possible solutions: using memory in LLM systems, retrieval capability enhanced generation (RAG) and sliding window technology.
6. Temperature and sampling
Temperature helps balance randomness and predictability in responses generated by language models. At temperature 0, the model always selects the most likely token, resulting in consistent but potentially duplicate results. Higher temperatures introduce randomness, making the output potentially more creative but easier to predict.
In essence, temperature determines the probability distribution of model vocabulary. At low temperatures, the model is strongly favorable tokens with high probability. At high temperatures, it gives a lower probability token choice opportunity.
Sampling technologies such as TOP-K and Nucleus sampling provide other control mechanisms for text generation. TOP-K sampling will limit the selection to k highest probability token, while kernel sampling can adaptively determine the settings of candidates by using the cumulative probability threshold.
These technologies help balance creativity and coherence, providing developers with fine-grained control over model behavior.
7. Model parameters and proportions
Model parameters are learning weights for encoding everything that LLM knows. Most large language models usually have millions of parameters, while larger models push it to trillions of dollars. These parameters capture patterns in language, from basic syntax to complex reasoning capabilities.
More parameters usually mean better performance, but the relationship is not linear. Scaling the size of the model requires larger computing resources, datasets, and training durations.
For actual development, parameter counting affects inference costs, latency and memory requirements. The 7 billion parameter model may run on consumer hardware, while the 70 billion parameter model requires an enterprise GPU. Understanding this trade-off helps developers choose the right model size for their specific use cases and infrastructure restrictions.
Summarize
The concepts we introduce in this article form the technical core of each LLM system. Then what is next?
Go to build something that can help you understand language models better. Try to do some reading on the go, too. Start by starting with “all attention you need” (such as “attention”), explore embedding techniques, and then try different tokenization strategies on your own data.
Set up the local model and observe how temperature changes affect the output. Use configuration files across different parameter sizes. Happy experiment!
