PanGu (Huawei)

From Systems analysis Wiki
Jump to navigation Jump to search

Huawei PanGu (Chinese: 盘古) is a family of ultra-large pre-trained artificial intelligence models (foundation models) developed by Huawei Cloud. The name "PanGu" refers to Pangu, the mythological first living being in Chinese culture who created the world[1]. The PanGu family covers various domains, including natural language processing (NLP), computer vision (CV), multimodal analysis, predictive modeling, and scientific computing. Since 2025, the openly released members of the family have been distributed under the name openPangu.

History and Development

PanGu-α (2021)

The first model in the family, PanGu-α (PanGu-Alpha), was introduced in April 2021. With 200 billion parameters, it became the largest Chinese language model at the time, surpassing the size of GPT-3 (175 billion) from OpenAI[2].

The model was developed by the Huawei Cloud team in collaboration with the Noah's Ark lab and trained on a cluster of 2048 specialized Huawei Ascend 910 processors using the MindSpore framework[3]. The training corpus consisted of 1.1 TB of high-quality Chinese text data. PanGu-α demonstrated strong performance on the CLUE (Chinese Language Understanding Evaluation) benchmark, achieving first place in the overall rankings[1].

PanGu-Σ (2023): Toward a Trillion Parameters

In March 2023, Huawei researchers published a paper introducing PanGu-Σ (PanGu-Sigma), a sparse language model with 1.085 trillion parameters — roughly an order of magnitude larger than PanGu-α[4]. Built on the MindSpore framework and trained on a cluster of 512 Ascend 910 accelerators over roughly 100 days using 329 billion tokens drawn from more than 40 natural and programming languages, PanGu-Σ inherited its dense parameters from PanGu-α and extended them into a sparse architecture through a mechanism called Random Routed Experts (RRE).

Unlike conventional MoE routing, RRE uses two-level routing in which experts are first grouped by domain or task, and tokens are then randomly and uniformly assigned within each group, without a learnable gating function. Combined with an Expert Computation and Storage Separation (ECSS) technique, this yielded a reported 6.3-fold increase in training throughput over a comparable MoE model with the same hyper-parameters, and allowed compact sub-models to be extracted for downstream tasks such as dialogue, translation, and code generation[4].

PanGu 3.0 (2023): A Platform Approach

In July 2023, Huawei unveiled the PanGu 3.0 platform, marking a shift from a single model to a multi-layered "5+N+X" architecture focused on industrial applications[5].

  • L0 (Base Layer): Five fundamental models (NLP, CV, multimodal, predictive, and scientific computing).
  • L1 (Industry Layer): N industry-specific models, fine-tuned from the base models for specific sectors (government, finance, manufacturing, etc.).
  • L2 (Scenario Layer): X models for specific application scenarios (virtual assistant, typhoon track prediction, etc.).

This hierarchical approach allows customers to either use ready-made solutions or fine-tune industry models on their own data, significantly simplifying and reducing the cost of adaptation.

PanGu 5.0 (2024)

On June 21, 2024, at the Huawei Developer Conference (HDC 2024), Huawei Cloud CEO Zhang Ping'an announced PanGu 5.0, an upgrade emphasizing a full range of model sizes, stronger multimodal understanding, and improved reasoning[6]. The release spanned a family of models ranging from around one billion parameters for on-device use up to trillion-parameter models for cross-domain enterprise tasks, and added the ability to process modalities such as text, images, video, radar, infrared, and remote-sensing data. PanGu 5.0 also powered an upgraded Xiaoyi (Celia) assistant on Huawei's HarmonyOS devices[6].

PanGu 5.5 (2025): Mixture-of-Experts Architecture

On June 20, 2025, at HDC 2025, Huawei announced PanGu 5.5, aimed at solving advanced industrial problems. Five foundation models — for NLP, computer vision, multimodal processing, prediction, and scientific computing — were upgraded together[7]. The centerpiece was a new NLP "deep thinking" model built on a Mixture-of-Experts (MoE) architecture with 256 expert sub-networks and a total of 718 billion parameters[8].

Because the MoE design dynamically activates only a portion of the model for a given task — and combines "fast" and "slow" thinking depending on problem difficulty — Huawei reported an eight-fold increase in overall inference efficiency compared with the previous generation[9]. The update also introduced DeepDiver, an agent module able to carry out multi-step research tasks of more than ten steps in under five minutes and generate reports exceeding 10,000 words[7].

openPangu 2.0 (2026)

On June 12, 2026, at the Huawei Developer Conference (HDC 2026), Huawei introduced openPangu 2.0, an open-weight generation of the family presented in two Mixture-of-Experts variants: Pro, with 505 billion total parameters and 18 billion activated per token, and Flash, with 92 billion total and 6 billion activated parameters[10]. Both variants support a 512K-token context window and are optimized for Ascend hardware and the HarmonyOS ecosystem; Huawei claimed roughly double the single-card inference throughput of other mainstream open-source models on Ascend[10].

Beginning June 30, 2026, Huawei started to open-source the model in stages on the GitCode "Ascend Tribe" community, releasing the openPangu-2.0-Flash weights, inference code, and operators first, with the larger Pro variant and additional components to follow[11]. The models were trained on Huawei's Ascend AI accelerators and optimized for its software stack, in line with the company's strategy of offering an AI platform independent of Nvidia hardware[11].

Key Architectural and Technical Solutions

The PanGu models are built on a GPT-like transformer architecture but include several innovations for training ultra-large models. To control the generation process, a special Query Layer was introduced, which helps to induce the desired output during the pre-training stage[3].

From PanGu-Σ (2023) onward, the family increasingly adopted sparse, Mixture-of-Experts designs — first through Random Routed Experts, and later through the MoE architectures of PanGu 5.5 and openPangu 2.0 — in order to scale total parameter counts while keeping the compute activated per token comparatively low.

The training and deployment of PanGu models are tightly integrated with Huawei's own hardware and software platform:

  • Ascend 910 Processors: Specialized AI accelerators that form the basis of the computing clusters.
  • MindSpore Framework: An open-source deep learning platform that supports auto-parallel technology, which combines five types of parallelism (data, model, pipeline, optimizer, etc.) to efficiently distribute computations across thousands of nodes[3].

Specialized Models and Their Applications

PanGu-Weather

One of the most well-known models in the family is PanGu-Weather, a global meteorological model based on deep learning. In July 2023, a paper describing it was published in the prestigious scientific journal Nature, reported to be the first Nature paper authored solely by employees of a Chinese technology company[12].

The model has demonstrated the ability to match or surpass the accuracy of traditional numerical weather prediction methods from the European Centre for Medium-Range Weather Forecasts (ECMWF) at a significantly higher speed. Generating a 24-hour global forecast takes the model mere seconds instead of the hours required by supercomputer-based numerical prediction, representing a speed-up of approximately 10,000 times[13]. From mid-2023, PanGu-Weather forecasts were made available on the ECMWF website, and the model was included in ECMWF's evaluation of data-driven weather forecasting[14].

Industrial Applications

PanGu models have been implemented in over 500 scenarios across 30 industries. Some examples include:

  • Agriculture: The Chinese Academy of Agricultural Sciences (CAAS) used PanGu to develop a breeding model, which helped cultivate an experimental rice variety with improved lodging resistance[8].
  • Oil and Gas Industry: CNPC uses a PanGu model (the "Kunlun" large model) for the automatic detection of pipeline defects with sub-millimeter accuracy, increasing identification efficiency by ~40%[15].
  • Public Administration: In Shenzhen, an intelligent assistant named "Xiaofu" was created, which provides citizens with information about public services based on a corpus of over 200,000 local documents[5].
  • Pharmacology: The PanGu Drug Molecule Model is used to accelerate the screening of drug candidates. Using it, a team led by Professor Liu Bing at Xi'an Jiaotong University reported designing a broad-spectrum antimicrobial compound in about one month; Huawei describes it as a candidate expected to become the first new class of antibiotics with novel protein targets in roughly 40 years, though the drug remains in development[5][16].

Open-Source Release

In late June 2025, Huawei began open-sourcing part of the PanGu family under the openPangu name, publishing the models on the Chinese developer platform GitCode[17]:

  • PanGu (openPangu) Dense Model 7B (7 billion parameters).
  • PanGu (openPangu) Pro MoE Model 72B (72 billion parameters), described by Huawei as the first large-scale model trained entirely on its own Ascend chips.

The move was aimed at stimulating innovation and building an open ecosystem around the Huawei Ascend hardware platform as a strategic response to global competition in AI[17].

Allegations of derivation from Alibaba's Qwen (2025)

Days after the open-source release, in early July 2025, an entity calling itself HonestAGI posted a paper on GitHub arguing that the PanGu Pro MoE 72B model showed an "extraordinary correlation" with Alibaba's Qwen 2.5 14B model, based on "model fingerprinting" of attention-parameter distributions, and suggesting the model may not have been trained from scratch[18]. Huawei's Noah's Ark Lab quickly rejected the claim, stating that the model was independently developed on Ascend hardware and not based on incremental training of other companies' models, and that any third-party open-source code had been used in compliance with the relevant licenses[18]. The original repository containing the allegations was subsequently deleted. Reuters and other outlets reported the dispute as an illustration of intensifying competition and intellectual-property scrutiny within China's AI industry[18].

Toward a fully open-source stack (2025)

At Huawei Connect 2025 (Shanghai, September 2025), Huawei committed to open-sourcing its entire Ascend AI software stack by December 31, 2025, including the CANN compute architecture, the Mind-series (MindSpore) toolchains, and the openPangu foundation models — a strategy explicitly framed as building a global alternative to Nvidia's CUDA ecosystem[19].

Literature

  • Zeng, W.; et al. (2021). PanGu‑α: Large‑Scale Autoregressive Pretrained Chinese Language Models. PDF.
  • Ren, X.; et al. (2023). PanGu‑Σ: Towards Trillion Parameter Language Model with Sparse Heterogeneous Computing. arXiv:2303.10845. Online preprint.
  • Huawei (2021). HDC.Cloud 2021: Huawei Releases Six Ground‑breaking Products to Supercharge the Cloud and Intelligent Transformation of Business. Online news.
  • Huawei Cloud (2023). Reshaping Industries with AI: Huawei Cloud Launches PanGu Models 3.0 and Ascend AI Cloud Services. Online news.
  • Bi, K.; et al. (2023). Accurate Medium‑Range Global Weather Forecasting with 3D Neural Networks. Nature, 620, 560–566. DOI:10.1038/s41586‑023‑06185‑3.
  • Technology Magazine (2025). What Huawei PanGu 5.5 Models Mean for Industrial AI. Online article.
  • Huawei Cloud (2025). Huawei Cloud Announces Pangu Models 5.5 and All‑new AI Cloud Service. Online news.
  • MindSpore Team (2021). MindSpore: An All‑Scenario Deep Learning Computing Framework (White Paper v1.1). PDF.
  • Zhang, S.; et al. (2024). Ascend 910 NPU SoC Architecture for Large‑Scale AI Training. arXiv:2407.11888. Online preprint.
  • AIbase News (2025). Huawei Open Sources Dense PanGu 7B and Mixture‑of‑Experts PanGuPro 72B. Online news.
  • Reuters (2025). Huawei's AI lab denies that one of its Pangu models copied Alibaba's Qwen. Online report.
  • Huawei (2025). Huawei Connect 2025 keynote — open‑sourcing CANN, MindSpore and openPangu by 31 Dec 2025. Online news.
  • PANews (2026). Huawei Releases Open‑Source Large Model openPangu 2.0 (Pro 505B / Flash 92B, 512K context). Online news.
  • CNPC & Huawei Cloud (2024). Kunlun: Large‑Scale AI Model for Oil and Gas Pipeline Defect Detection. Online case study.
  • MindSpore Docs (2024). Automatic Parallel — Five‑Mode Hybrid Strategy in MindSpore. Online documentation.
  • Press, O.; et al. (2021). Train Short, Test Long: Attention with Linear Biases Enables Input‑Length Extrapolation. arXiv:2108.12409.
  • Law, M. (2025). How Huawei PanGu 5.5 AI Models Transform Industry Operations. AI Magazine. Online article.

References

  1. ↑ 1.0 1.1 “HDC.Cloud 2021: Huawei Releases Six Groundbreaking Products to Supercharge the Cloud and Intelligent Transformation of Business”. Huawei. [1]
  2. ↑ Wodecki, Ben (27 Apr 2021). «Huawei has created the world's largest Chinese language model». AI Business. [2]
  3. ↑ 3.0 3.1 3.2 Zeng, Wei, et al. (Apr 2021). «PanGu-α: Large-scale Autoregressive Pretrained Chinese Language Models». Technical Report. [3]
  4. ↑ 4.0 4.1 Ren, Xiaozhe, et al. (20 Mar 2023). «PanGu-Σ: Towards Trillion Parameter Language Model with Sparse Heterogeneous Computing». arXiv:2303.10845. [4]
  5. ↑ 5.0 5.1 5.2 «Reshaping Industries with AI: Huawei Cloud Launches Pangu Models 3.0 and Ascend AI Cloud Services». HUAWEI CLOUD. 7 Jul 2023. [5]
  6. ↑ 6.0 6.1 «Huawei Showcases AI-Native Cloud Innovations, Highlighting Industry Applications of Pangu 5.0 Models». BigDATAwire / HPCwire. 20 Sep 2024. [6]
  7. ↑ 7.0 7.1 «Huawei Cloud Announces Pangu Models 5.5 and All-new AI Cloud Service, Positioned as the AI Pioneer in Industries». HUAWEI CLOUD. 20 Jun 2025. [7]
  8. ↑ 8.0 8.1 Law, Marcus (23 Jun 2025). «What Huawei Pangu 5.5 Models Mean for Industrial AI». Technology Magazine. [8]
  9. ↑ «How Huawei Pangu 5.5 AI Models Transform Industry Operations». AI Magazine. [9]
  10. ↑ 10.0 10.1 «Huawei releases open-source large-scale model Pangu 2.0: up to 505 billion parameters and 512K context». PANews. 12 Jun 2026. [10]
  11. ↑ 11.0 11.1 «Huawei open-sources the openPangu-2.0-Flash model». KuCoin / PANews. 30 Jun 2026. [11]
  12. ↑ «Prestigious science journal Nature publishes paper about Pangu Weather AI Model authored by HUAWEI CLOUD researchers». Huawei News. 6 Jul 2023. [12]
  13. ↑ Bi, Kaifeng, et al. (2023). «Accurate medium-range global weather forecasting with 3D neural networks». Nature. [13]
  14. ↑ «AI Weather Prediction: More Accurate, in Just Seconds». HUAWEI CLOUD. [14]
  15. ↑ «CNPC and Huawei Cloud Jointly Launch the "Kunlun" Model for the Oil and Gas Industry».
  16. ↑ «Huawei Cloud's Pangu Drug Molecule Model Accelerates the Discovery of Super Antibiotics». HUAWEI CLOUD. 19 Sep 2022. [15]
  17. ↑ 17.0 17.1 «Huawei Open Sources Dense Pangu 7B and Mixture of Experts Model with 72B Parameters». Albase News. 30 Jun 2025. [16]
  18. ↑ 18.0 18.1 18.2 «Huawei denies Pangu model was based on Alibaba's Qwen». Developer Tech (reporting Reuters). Jul 2025. [17]
  19. ↑ Xu, Eric (keynote) (Sep 2025). «Groundbreaking SuperPoD Interconnect: Leading a New Paradigm for AI Infrastructure». Huawei. [18]
Retrieved from "https://systems-analysis.info/eng/index.php?title=PanGu_(Huawei)&oldid=298"