Ai Model's Benchmarking
You must have heard or seen benchmarking of LLM when they released on some standards aspects.
Why Benchmarking is Important?
Computer scientist discovered various way to measure intelligence, capacity, capability, using various checks for LLM model and standardized them.
you will have seen some models when given update like gemini, grok, Qwen, Deepseek, Sarvam etc.
below is one snippet which is released recently.
Below is the benchmark report of the Gemini 3.1 Pro Preview released by Google yesterday
Now as you can see their multiple Benchmarks each one has specific things which validates.
You can also see for one of popular opensource model developed by qwen
Now you would have observed there are multiple ways of benchmarking, and every model have to go through these tests.
So, a question arises here why you should you be bothered about these bench marking methodologies.?
Now let's take analogy of buying any product, you should have purchased any items for instance take example of buying a laptop what you do.
You research and take note of
you gather what you want (like what work you want to do example gaming, video-editing)
you check what are the criteria that laptop should have like higher Ram, must have external Graphics card, higher CPU model other aspects too.
So, idea behind this analogy is to see what the things are required for your requirement it gives you a all idea about choosing a product which suits you.
That is why it is required to understand the benchmarking and model attributes, that will help you to choose models for your specific work or requirement when you build any product.
Now wrapping it up for now dropping you a question here
As you know these benchmarks are standardized then any model trainer would train them to pass those particular benchmarks then how the credibility of the model is met in large context?
Leaving you with that.
Comment your thoughts on that.
In the upcoming blogs we will go through each popular benchmarking models one by one with their evolution
