Language models are by far the most visible AI technology right now. They write emails, summarize contracts, and answer customer questions. But under the hood, it is less mysterious than marketing language makes it sound.
What a language model does
To the point: It predicts which word comes next. Nothing more. If you write “The capital of Germany is”, the model provides the most plausible continuation — “Berlin”.
Whole answers, texts, code, and translations emerge from word-for-word predictions. What sounds simple is surprisingly powerful — as long as the question fits patterns that the model has seen before.
How it learned it
From gigantic amounts of text — books, Wikipedia, forum posts, manuals, code repositories. A modern model has internalized patterns from several billion sentences.
It is like an applicant who has read every book in the city library — but does not remember any single source exactly. They can say something plausible about almost everything, but cannot provide a verbatim citation for every sentence.
Tokens and context windows
Language models do not see “words” — they see tokens. These are pieces consisting of two to five letters. A German word like “Werkstattauftrag” is often broken down into three or four tokens.
The context window is the amount of tokens that the model can read at once. Modern models manage 8,000 to 128,000 tokens — ranging from a short email to an entire stack of contracts.
Well-known models, briefly categorized
GPT-4, Claude, Gemini
Cloud models from the largest providers. Very powerful, broadly trained. But: data goes to the provider, often to the USA. Practical for routine tasks with non-critical content — problematic with sensitive data.
Llama, Mistral, Qwen
Open-weight models: the file is freely available, the model runs on your own hardware. Several sizes — from 7 to 405 billion parameters. For most office tasks, 8 to 70 billion parameters are completely sufficient.
Specialized models
For code, law, or medicine, there are domain-specific trained models. Often also usable locally and, in a narrowly defined area, better than generalists.
What a 70-billion-parameter model like Llama 3.1 can do locally covers about 80 percent of what GPT-4 can do in the cloud — with full data control and without ongoing per-query costs.
What you should remember
- Language models guess — very well, but not guaranteed to be correct
- Cloud models: powerful, but data leaves your premises
- Local models: fully controlled, often good enough for routine tasks
- Larger is not always better — smaller models are faster and often sufficient. With RAG, even smaller models become useful for your business.
- The right model depends on the task, not on marketing
Which model fits your business?
We recommend local or cloud — depending on what you want to do with it.
Schedule an initial consultation
Frequently asked questions
What does a language model do?
How did a language model learn?
What are tokens and the context window?
Legal notice and privacy policy: Impressum & Datenschutz (German only; the German version is binding).