The reference points GSMA Open-Telco LLM is launched to advance in Telecoms

The new community provides an open source framework to evaluate large language models for capacity, energy efficiency and security.

February 25, 2025, London -The GSMA Foundry, GSMA's innovation hub, today announced the GSMA Open-Tan-Telco LLM reference points, an open source community aimed at improving the performance of large language models (LLM) for specific applications of specific applications of Telecommunications. The community provides a first industry frame to evaluate AI models in cases of use of real world telecommunications and is compatible with the launch when hugging FACE, the University of Khalifa, the Linux Foundation and a large number of operators and suppliers mobile network leaders.

As the adoption of AI is accelerated in telecommunications, the LLMs have demonstrated significant deficiencies in the management of technical knowledge of telecommunications, regulatory compliance and the solution of network problems. In recent tests, GPT4[1] He scored less than 75% in Teleqna[2] [3]An integral data set adapted to evaluate the knowledge of the LLM in the field of telecommunications, and less than 40% in the 3GPptDOCS classification[4]A data set based on 3GPP standards documentation. Microsoft Phi2[5]A much smaller model, obtained only 10% in Math500[6] [7]A reference point of 500 general mathematics questions.

These results highlight the current limitations of AI models to address specific telecommunications consultations. The GSMA Open-Ten-Telco LLM reference points will address these spaces by providing transparent and open evaluations of AI models through capacities, energy efficiency and safety.

“Today's models fight with specific telecommunications consultations, often producing inaccurate, misleading or little practical recommendations“Said Louis Powell, head of IA initiatives, GSMA.”By creating a set of reference points throughout the industry, we are not only improving the performance of the model, but also ensuring that AI in telecommunications is safe, reliable and aligned with the operational needs of the real world. “

Mobile networks that admit the launch of the reference points GSMA Open-Telco LLM include Deutsche Telekom, LG Uplus, SK Telecom and Turkcell and Technology Supplier, Huawei.

The GSMA OPEN-TELCO LLM reference community allows mobile networks, researchers and artificial intelligence developers who send use cases, data sets and models for evaluation. A standardized comparative evaluation framework ensures that all AI models are evaluated against real -world challenges in areas such as knowledge of telecommunications domain, mathematical reasoning, energy consumption and safety. The resulting reference points will stay in embraceing the face to guarantee transparency and encourage community participation.

Mobile network operators, suppliers, new companies and researchers are encouraged to contribute, sending interest use cases and LLM, to [email protected] And for more information, visit www.gsma.com/get-involved/gsma-foundry/gsma-open-telco-llm-benchmarks.

Partners quotes:

Hugged face

Jeff Boudier, product manager and growth, hugging his face, said: “FACE is the leading open platform for AI builders, and we are delighted to support and organize the GSMA Open-Telco LLM reference points to advance the adoption and innovation of the AI ​​of Telecommunications.”

University of Khalifa

“The Academy plays a crucial role in the advance of AI for telecommunications by guaranteeing rigorous benchmarking and scientific integrity. At the University of Khalifa, we are proud to support the reference initiative GSMA Open-Telco LLM. This effort will promote innovation and improve the reliability of AI models in real -world telecommunications applications,“Said Professor Meroune Debbah, director, 6G Research Center, University of Khalifa.

LG UPLU

Sangyeob Lee, technology director, LG Uplus, said: “We stand at a turning point, which we go to the coexistence of human and AI agents, and telecommunications will play a vital role in establishing safe and autonomous connections between them. LG Uplus undertakes this innovation of AI agents through the progress of LLM and welcomes the reference points of Gsma Open-Telco LLM as our guide light towards the safe intelligence services we are looking for. “

The Linux Foundation

“The launch of the Open-Telco LLM reference points of GSMA marks a significant milestone in the advance of the adoption of AI throughout the telecommunications industry.” said Arpit Josfipra, General Manager, Networks, Edge and IoT, the Linux Foundation. “When establishing open and standardized reference points, this initiative brings very necessary performance and transparency information, allowing the operators and partners of the ecosystem to implement a specific domain AI with confidence. The Linux Foundation supports this effort, since it aligns with our vision of open collaboration to boost innovation and efficiency in telecommunications networks worldwide. “

SK Telecom

Eric Davis, head of the AI ​​Tech collaboration office, SK Telecom said: “The introduction of the reference points GSMA Open-Telco LLM marks a fundamental milestone for the telecommunications industry in its search for benefits of tangibles. By establishing a standardized evaluation framework, we are simultaneously promoting innovation and ensuring that the solutions of the robustness, reliability and precision required by our sector in rapid evolution. “

The launch follows the commitment of the entire industry last year to explore the cases of use of AI of Telecommunications in an ethically and sustainably, whose maturity road map responsible for the AI ​​of the GSMA, which helps the Mn To ensure that the principles of the best practice are applied from the beginning through evolution.

AI in MWC25 Barcelona

The 'Gen AI Summit: Experimentation to the transformation' in MWC25 Barcelona will have a variety of sessions aimed at exploring practical applications and the transformative potential of the generative AI within the telecommunications sector. The key sessions will include discussions on the optimization of the AI ​​promoted network, custom client's personalized experiences and the integration of the generative AI in 5G and beyond. Renowned speakers such as Harry Singh, BT digital director; Harrison Lung, Group's Strategy Director in E &; Kaniz Mahdi, director Technology Aws Industries at Amazon Web Services; and Laurent Leboucher, Cto in Orange; He will share his ideas and experiences to take advantage of AI for the advance of the industry.

Other most prominent aspects of the AI ​​speakers in MWC25 Barcelona include Keynote 7, 'Tech Game Changers', where Arthur Mensch, an outstanding researcher at AI and CEO from Mistral AI, will take the stage to discuss the latest developments and real world applications that meet the telecommunications industry. In addition, Keynote 10, “why the agents of AI will change everything,” they will see Bret Taylor, CEO and co -founder of Sierra and member of the OpenAi Board, will discuss how AI agents are prepared to transform companies and companies.

-Ess-

Editors notes