DeepSpeed

DeepSpeed
DeepSpeed
Original author(s)	Microsoft Research
Developer(s)	Microsoft
Initial release	May 18, 2020; 5 years ago
Stable release	v0.16.5 / March 27, 2025; 4 months ago
Repository	github.com/microsoft/DeepSpeed
Written in	Python, CUDA, C++
Type	Software library
License	Apache License 2.0
Website	deepspeed.ai

百度特别是去年，抠图、替身、天价片酬等有关流量明星的负面新闻层出不穷，更引起了观众的普遍反感。

DeepSpeed is an open source deep learning optimization library for PyTorch.^[1]

Library

The library is designed to reduce computing power and memory use and to train large distributed models with better parallelism on existing computer hardware.^[2]^[3] DeepSpeed is optimized for low latency, high throughput training. It includes the Zero Redundancy Optimizer (ZeRO) for training models with 1 trillion or more parameters.^[4] Features include mixed precision training, single-GPU, multi-GPU, and multi-node training as well as custom model parallelism. The DeepSpeed source code is licensed under MIT License and available on GitHub.^[5]

The team claimed to achieve up to a 6.2x throughput improvement, 2.8x faster convergence, and 4.6x less communication.^[6]

References

^ "Microsoft Updates Windows, Azure Tools with an Eye on The Future". PCMag UK. May 22, 2020.
^ Yegulalp, Serdar (February 10, 2020). "Microsoft speeds up PyTorch with DeepSpeed". InfoWorld.
^ "Microsoft unveils "fifth most powerful" supercomputer in the world". Neowin. 18 June 2023.
^ "Microsoft trains world's largest Transformer language model". February 10, 2020.
^ "microsoft/DeepSpeed". July 10, 2020 – via GitHub.
^ "DeepSpeed: Accelerating large-scale model inference and training via system optimizations and compression". Microsoft Research. 2025-08-05. Retrieved 2025-08-05.

External links

[1] "Microsoft Updates Windows, Azure Tools with an Eye on The Future". PCMag UK. May 22, 2020.

[2] Yegulalp, Serdar (February 10, 2020). "Microsoft speeds up PyTorch with DeepSpeed". InfoWorld.

[3] "Microsoft unveils "fifth most powerful" supercomputer in the world". Neowin. 18 June 2023.

[4] "Microsoft trains world's largest Transformer language model". February 10, 2020.

[5] "microsoft/DeepSpeed". July 10, 2020 – via GitHub.

[:0-6] "DeepSpeed: Accelerating large-scale model inference and training via system optimizations and compression". Microsoft Research. 2025-08-05. Retrieved 2025-08-05.

[1]

[2]

[3]

[4]

[5]

[6]

v t e Deep learning software
Comparison
Open source	Apache MXNet Apache SINGA Caffe Deeplearning4j DeepSpeed Dlib Keras Microsoft Cognitive Toolkit ML.NET OpenNN PyTorch TensorFlow Theano Torch ONNX OpenVINO MindSpore
Proprietary	Apple Core ML IBM Watson Neural Designer Wolfram Mathematica MATLAB Deep Learning Toolbox
Category

双向是什么意思	mds是什么病的简称	什么是薪级工资	空调制热效果不好什么原因	丝瓜长什么样
lf是什么牌子	十二指肠溃疡是什么症状	格力空调睡眠模式1234什么意思	身体内热是什么原因	困是什么原因
正山小种是什么茶	1912年属什么生肖	怀孕排卵试纸显示什么	夕阳西下是什么意思	11月1号是什么星座
老师家访需要准备什么	海带与什么食物相克	八月初六是什么星座	什么是g点	优生四项是检查什么

肩膀疼痛挂什么科hcv8jop1ns1r.cn	什么是子公司hcv9jop0ns0r.cn	两肺纹理增多模糊是什么意思hcv9jop2ns7r.cn	胃阳虚吃什么中成药hcv9jop4ns0r.cn	cini是什么意思hcv9jop0ns3r.cn
围魏救赵是什么意思hcv9jop5ns8r.cn	溜达鸡是什么意思hcv9jop3ns6r.cn	女娲姓什么hcv9jop8ns2r.cn	手指尖疼是什么原因xjhesheng.com	肌膜炎是什么原因造成的hcv8jop1ns6r.cn
丹田是什么意思hcv7jop7ns0r.cn	c反应蛋白什么意思cl108k.com	溢水是什么意思hcv7jop9ns0r.cn	腿上血栓是什么症状hcv8jop1ns4r.cn	产后42天复查挂什么科hcv8jop9ns4r.cn
为什么说尽量不戴脚链hcv7jop5ns2r.cn	鸽子不能和什么一起吃hcv9jop8ns2r.cn	治疗阳痿早泄什么药最好hcv8jop7ns4r.cn	一什么柳树hcv9jop1ns8r.cn	四肢百骸是什么意思hcv9jop3ns2r.cn

只欠东风的上一句是什么

Library

See also

References

Further reading

External links