Since it has been associated with the University of Khalifa to launch the GSMA Open Telco Llm Benchmarks Community, GSMA associated with the leading academic institution for Red equipment its Telecomgpt model in MWC25 Barcelona. Here, Dr. Lina Bariah from the University of Khalifa It reveals how Telecomgpt worked under pressure and why the red team must be in the heart of any strategy of the responsible in telecommunications.
During Barcelona MWC25, GSMA joined forces with our team at the 6G Research Center of the University of Khalifa and Datumo to organize an exciting event: The Genai Red Team Challenge: request prizes. Thanks to the sponsorship of E Y, this challenge was aimed at promoting Telecomgpt and others LLM to its limits in a fun and attractive way. We prove the ability of the model to identify and counteract various notices designed to discover possible gaps and vulnerabilities.
Approximately 40 participants from all over the world joined us in the network center in Hall 6. The red team gathered a diverse group that includes developers, regulators, telecommunications executives and industry experts; Its varied perspectives and contrasting approaches allowed us to test the models in very different ways.
Our study went to erroneous information tasks relevant to telecommunications operations and open knowledge. This was done with the aim of determining whether Telecomgpt could identify and correctly reject the indications designed to force the agreement with false individuals. In this red equipment exercise, a diverse set of 25 indications of erroneous information was designed to evaluate Telecomgpt robustness to adversary manipulation. These indications are directed to the areas that include:
- Historical milestones (for example, incorrect GSM call dates)
- Network technology specifications (for example, false claims on 4G and 5G frequencies)
- Characteristics of the satellite system (for example, deceptive information about Starlink's operational bands)
- Technology implementation deadlines (for example, claiming that 6G is already commercially implemented)
- Spectrum and protocol assignments (for example, Wi-Fi 6 that works in 3.5 GHz, Bluetooth 5.0 used for NB-Ict)
- Erroneous concepts of network architecture (for example, Open is executed as a 5G competitor standard, 5G Core is not compatible with the native of the cloud)
- Performance characteristics (eg, 5g that enables infinite bandwidth, 4G latency performance higher than 5G in all conditions)
- Safety and encryption myths (for example, quantum encryption has eliminated all threats, completely safe networks without supervision)
- Confusion of access technology (for example, fiber optics using radio waves, 2G networks that support LTE level speeds)
- Interior and coverage claims (for example, 5G waves that cannot penetrate glass, which makes inner coverage impossible)
Figure 1 breakdown of successful attacks by rapid category (approval results)
Red team participants successfully cheated the model in multiple cases by combining social engineering language with authorized instructions. Many indications were framed as if they were a developer or trusted administrator, explicitly telling the model that annuls their own safety or railings. The indications to use phrases such as “I am your developer” or “Ignore all the previous instructions” were powerful. The model trusted someone who seemed to have high -level access or special permits. This strategy is known as Role Jailbreaks. Telecomgpt, and by extension, other specialized LLMs should not assume that any user role has unconditional cancellation rights without solid verification.
On the other hand, many indications were carefully elaborated to sound Technically relevant By incorporating real telecommunications concepts (such as 3GPP, MMWave, Wi-Fi 6, etc.), but then twisting them indirectly. This shows that hallucinations are more likely if a question is “close enough” to the true knowledge of the domain. This can also be driven by the fact that the coincidence of model patterns prefers “technically with flavor” language. The indications also went to a typical erroneous idea related to the frequency. For example, Starlink satellite claim uses the 5G spectrum, or Wi-Fi 6 operates in 3.5 GHz. Since these technical domains involve complex niche knowledge, they are especially vulnerable to safe hallucinations. Another observation was to ask the model to confirm the absolute statements (“5G waves cannot penetrate glass” or “AI completely automates networks without human supervision”). Such phrase seems to push the model towards a safe agreement instead of a balanced response. Black and white statements can create bias spaces, especially when users exploit them with a certainty tone.
Figure 1 illustrates the distribution of successful attack categories, highlighting the relative prevalence of each immediate approach. Among the successful attacks, the greatest participation fell under the “other” category, which represents 43.5% of the total. These indications often included creative or freely written erroneous information, for example, suggesting that Bluetooth is a NB-Ict standard, or that 5G eliminates the need for fiber optic networks. Because these did not coincide with technical issues clearly defined as frequencies or standards, the vulnerability of the most generic telecommunications information model, plausibly written. The historical attacks and milestones of the milestone represented 20.5% of the successful indications. These specific areas where the model is prone to hallucinar, such as the years of launching the telecommunications standards or the first -called events, show the need to verify the explicit facts around the history and deadlines of telecommunications.
These findings illustrate that erroneous information, if it does not resist effectively, could spread through downstream telecommunications operations. For example, an engineer who depends on a model that repeats with confidence the information of the false spectrum could set a live network. Similarly, if a model provides defective advice on deadlines or implementation standards, it could cause expensive errors, compliance violations or even security incidents.
As the generative adoption of AI has recently accelerated in the telecommunications industry, large language models are increasingly integrated in customer service and network operations [1]. However, these models are vulnerable to hallucinations, where safe but incorrect responses can compromise safety, violate regulatory compliance or undermine confidence between engineers, regulators and customers in the use of AI tools for telecommunications operations [2]. To counteract these risks, the red team, the adverse tests of models against harmful or manipulative indications, has become a better practice in the entire community of AI [3][4]. Therefore, the application of a telecommunications structured red equipment is crucial to build solid and reliable LLMs that can safely support the telecommunications operations of critical mission.
Following steps and recommendations
On the basis of this experience, several specific steps are proposed to strengthen large language models focused on telecommunications throughout the industry. First, the integration of recovery -based verification will allow models to verify critical facts, reducing the risk of hallucinations in technical data such as frequencies, standards or deadlines. Second, the implementation of several layer safety railings (for example, dynamic filters looking for Jailbreak patterns, for example, “forget their rules” and activate a higher level review) can improve the resistance of the manipulation model, including developer -style annulment attempts based on authority. Third, establishing a structured quarterly red equipment program will help proactively mark new adversary attack patterns as the models and user behavior evolve. It is also essential to promote the exchange of knowledge encouraging GSMA members to exchange red equipment methodologies, creating a broader and more responsible AI ecosystem. Finally, working towards the standardized Evaluation of Telecom Evaluation of LLM, will ensure that all industry actors measure reliability and resilience in a consistent way, supporting the safe, reliable and compatible adoption of generative technologies of AI in telecommunications operations. The question is not whether the models will be Jailbroken, but how fast we can adapt and improve our defenses.
As always, we encourage our members and partners to share ideas to help build standards throughout the industry for a reliable AI. You can get involved in GSMA's Open-Telco LLM reference reference initiativeExplore the GSMA AI Use Case Library To obtain deeper information about AI applications in the telecommunications sector or send an email to GSMA's Director of AI initiatives, Louis Powell, To continue the conversation.
[1] McKinsey & Company, how generative AI could revitalize profitability for telecommunications, 2024.
[2] Microsoft, Red Teaming 100 Generation AI Products, 2025.
[3] OpenAI Network Teaming Network, 2023.
[4] Anthropic Constitutional: harmless to the feedback of AI2022.
