Rules for Understanding Language Models

After millions of years of evolution, humans understand each other pretty well. But now, confronted with machines that talk, we cannot assume they will act like humans, or act for the same reasons as humans. If we don’t understand how language models (LMs) will behave or the general principles behind that behavior, it’s easy to fall into common pitfalls and create more work than we save by using them for inappropriate tasks or settings. I will draw on my own research and other findings in the modern science of AI to explain 5 general principles of language model behavior that drive their errors and their differences from human behavior:

1. An LM memorizes when it can
2. An LM acts like a population, not a person
3. An LM aims to please
4. An LM leans on subtle associations
5. An LM learns only what's written down