Focus Beats Size: A 4B Model on GSMA’s Open Telco Leaderboard

A 4-billion-parameter model, built by a small team with a fraction of the usual computing power, now sits at rank 7 on the public GSMA Open Telco Leaderboard. Synapt’s telecom-domain model, telco-slm-4B, ranks directly behind two of Google’s Gemini models and just ahead of SoftBank’s LTM. The leaderboard also includes general-purpose models built for a much wider range of work than this benchmark covers.

A result like this needs context to be read properly. This piece answers three questions: what GSMA is and why its leaderboard matters, why a smaller model can compete with larger ones, and what the benchmark tests. Once those questions are answered, the result can be assessed on its merits.

What GSMA is and why its leaderboard is credible?

GSMA is the organization that represents mobile network operators globally. It is the body behind Mobile World Congress, the industry’s largest annual event, and it helps set shared technical standards across telecom. A GSMA benchmark is not one company’s claim about its own AI. It is a shared, public, independently administered test that competitors submit their models to, because comparison against others is how credibility is established in this industry.

The GSMA Open Telco Leaderboard, run jointly with Khalifa University and Huawei’s Paris Research Centre, is that kind of shared test. It is now on its third major release, and a recent snapshot listed 85 different AI models, including telecom-specific models and the main general-purpose models of several major AI labs. Each of those labs submitted its model knowing it would be scored against models built specifically for telecom, so the board is not easy to top. It includes some of the most capable AI models available.

Model size and compute

The leaderboard’s top-ranked model, built by AT&T, has 31 billion parameters and was trained on roughly 400 billion telecom-specific words across about 530 GPUs over an extended period, a large undertaking for AT&T, one of the world’s largest telecom operators.

telco-slm-4B has 4 billion parameters, roughly a twelfth the size, built using a fraction of that computing power: on the order of 2,000 times less compute than AT&T’s leading model. It was trained with distillation, in which a smaller model learns from a larger “teacher” model’s reasoning on telecom problems instead of from the full breadth of general knowledge. The difference resembles training a surgeon for one specific operation versus training a doctor to practice general medicine across every specialty. The specialist does not need the broader training to perform that one operation well.

The size and compute figures matter as much as the rank: a team with a small fraction of the usual budget and hardware produced a model that performs competitively on a hard, shared, public test, alongside models built with far larger budgets. It reached rank 7 with roughly a twelfth of the parameters and a fraction of a percent of the compute of the leaderboard leader.

What the leaderboard tests

The GSMA leaderboard scores models across seven telecom tasks, including interpreting 3GPP technical specifications, answering questions about radio access network standards and solving telecom-specific math problems. telco-slm-4B’s overall average across all seven is 74.0, placing it directly behind two Google Gemini models and just ahead of SoftBank’s LTM.

telco-slm-4B scored 96.4 on TeleLogs, the single highest score on that benchmark across the entire leaderboard. TeleLogs gives the model a real network log and a fixed list of eight possible root causes, and asks which one explains the issue. The answer is not stated directly in the log. It requires careful reading of technical log content and reasoning about cause and effect in a specialized domain.

The path to 96.4 was uneven. At one point the model’s TeleLogs score was as low as 31%, and investigation traced the cause to how the model read the answer options. It had partly learned to guess based on where an answer option sat in a fixed list, rather than reading what the option said.

To test this, the same 120 questions were run twice, once with the answer options in their normal order and once shuffled, with the content unchanged. The model changed its answer on 42.5% of questions because of the shuffle: clear evidence it was partly reading position and not content.

The fix was to train on both the normal and shuffled versions of every example, so that position was no longer a usable shortcut. That change alone raised the TeleLogs score by over 20 points. Further iteration beyond that fix took it to 96.4, each round finding what was wrong and fixing exactly that.

What does Synapt’s GSMA leaderboard result demonstrate?

The result should not be read as evidence that larger models are overrated. General-purpose models perform strongly across a very wide range of tasks that telco-slm-4B was not trained for: writing code, holding an open-ended conversation and reasoning across unrelated domains. They are built for breadth, and that breadth requires a large amount of compute.

When a task is well defined and the domain is specific, a smaller model built for that exact task, trained on the right data, iterated on carefully, with failure modes such as positional bias diagnosed and fixed, can match or exceed a general-purpose model on that task, because the breadth of a general-purpose model isn’t needed here. This is a narrower claim than “small models are just as good”, and it is the one the data supports.

What this means for choosing a model

For organizations evaluating AI vendors in a technical domain (telecom, healthcare, finance, law), this result bears on a practical question: does the job require a large, general-purpose model running on someone else’s cloud infrastructure, with your data leaving your own systems? Or can a smaller, purpose-built model, running entirely on infrastructure you control, do the job you need done, at lower compute cost and without your data leaving your own environment?

This result illustrates the tradeoff: a model a twelfth the size of the leaderboard leader, built for one domain instead of every domain, ranking alongside some of the strongest general-purpose AI models on the tasks that domain covers. General-purpose and domain-specific models each have a place, and when choosing between them it helps to ask what the job in front of you needs.

Written by

Bibek Behera · SLM and Graph Team Lead

I lead the SLM and Graph team at Synapt AI, incubated at Prodapt, where we build domain-specific small language models and knowledge-graph intelligence for enterprise AI. My focus is making AI systems trustworthy, from extraction and entity resolution to graph-aware retrieval and evaluation. I’m also the Founder & CEO of Flomoney, and previously built technology at Alibaba Group and Sears. I studied at IIT Bombay. I’m passionate about agentic AI, efficient models, and lean AI-native teams, and I enjoy mentoring early-career engineers. Always happy to connect with anyone building in agentic AI, SLMs, or knowledge graphs.