A model's knowledge lives in millions or billions of numbers, usually stored at high precision. Quantization stores them with fewer bits, shrinking the model's memory footprint and letting it run faster, often with only a small loss in quality.
The art is choosing how aggressively to reduce precision without the answers degrading noticeably. Done well, it is what allows sizeable models to run on phones and laptops rather than requiring a server for every request.