在AWS部署多大型多語言語言模型BLOOM

多語言大型語言模型BLOOM(bloom-176B)

Model Architecture and Objective
Modified from Megatron-LM GPT2
Decoder-only architecture
Layer normalization applied to word embeddings layer (StableEmbedding)
ALiBI positional encodings, with GeLU activation functions

176,247,271,424 parameters:
    3,596,615,680 embedding parameters
    70 layers, 112 attention heads
    Hidden layers are 14336-dimensional
    Sequence length of 2048 tokens used (see BLOOM tokenizer, tokenizer description)
    Objective Function: Cross Entropy with mean reduction.

於AWS部署Bloom-176B

部署系統需求

1. AWS 帳戶。
2. 加入 Amazon SageMaker Studio。(通常需要花費 10 分鐘左右執行初始設定)
3. 對於 BLOOM-176B，建議使用 ml.p4d.24xlarge。

使用Amazon SageMaker

1. 在選擇的AWS區域中打開Amazon SageMaker Studio。在選擇AWS區域時，請考慮可用的機型。此模型需要使用ml.p4d.24xlarge。
2. 在Amazon SageMaker Studio中，複製amazon-sagemaker-examples。
3. 到路徑inference/nlp/realtime/llm/bloom_176b/，並打開notebook “djl_deepspeed_deploy.ipynb”。
4. 執行筆記本中的所有cell，只需注意最後5個單元格將進行資源清理，可以在結束使用前執行即可，並且注意notebook中的兩個可設置選項：

• 指定從Hugging Face Hub下載模型

• 使用VpcConfig指定創建端點

BLOOM模型應用

%%time

smr_client.invoke_endpoint(
    EndpointName=endpoint_name,
    Body=json.dumps(
        {
            "input": [
　　　　　　　　"Cloud computing advances", 
　　　　　　　　"AWS is the best" 
　　　　　　　　],
            "gen_kwargs": {
                "min_length": 20,
                "max_new_tokens": 100,
                "temperature": 0.8,
                "num_beams": 5,
                "no_repeat_ngram_size": 2,
            },
        }
    ),
    ContentType="application/json",
)["Body"].read().decode("utf8")

CPU times: user 18.6 ms, sys: 0 ns, total: 18.6 ms
Wall time: 11.8 s

'[\n  "Cloud computing advances in the last few years have made
 it possible to store and process large amounts of data in a 
cost-effective manner. Cloud computing is a model for enabling
 ubiquitous, convenient, on-demand network access to a shared 
pool of configurable computing resources (e.g., networks, 
servers, storage, applications, and services) that can be 
rapidly provisioned and released with minimal management 
effort or service provider interaction. This cloud model 
promotes availability and is composed of five essential 
characteristics, three service models",\n  "AWS is the best
 cloud computing service provider in the world. It provides a
 wide range of services to its customers. The services provided
 by the company are as follows:\\nThe company has a large number 
of data centers in different parts of the globe. This is done to
 ensure that the services are available to the customers at all 
times."\n]'