Load testing by Python Locust

Locust 介紹

An open source load testing tool. Define user behaviour with Python code, and swarm your system with millions of simultaneous users.

如果你也是python 使用者，這邊介紹一個效能測試的工具locust，他是直接用python code去寫測試script，自由度比較高，不像老牌測試工具jmeter 是需要GUI操作，假設我們有一個登入情境，密碼是需要RSA加密，那用python 可以去引用一些第三方library去完成這個任務，或是一些比較複雜行為，如果是jmeter沒有在他功能範圍內，就不知道要怎麼使用。

•Official Site: https://locust.io/

•Docs: http://docs.locust.io/en/stable/

•Installation

pip install locust

Write a locust script

這邊一個範例是先登入拿取token後去測試/files API

from locust import HttpUser, task, between

import json

from test_data import *

import random





class WebsiteTestUser(HttpUser):

    wait_time = between(0.5, 1)

    host = "http://host_address"



    def on_start(self):

        """ on_start is called when a Locust start before any task is scheduled """

        if len(USER_CREDENTIALS) > 0:

            account, password = USER_CREDENTIALS.pop()

            self.access, self.refresh = self.login_to_get_token(account, password)



    def on_stop(self):

        """ on_stop is called when the TaskSet is stopping """

        pass



    def login_to_get_token(self, account, password):

        path = '/login'

        headers = {'Content-Type': 'application/json'}

        payload = {

            "email": account,

            "password": password

        }

        resp = self.client.post(url=path, headers=headers, data=json.dumps(payload))



        return resp.json()['access_token'], resp.json()['refresh_token']



    def refresh_token(self, token):

        url = "/refreshToken"

        headers = {'Content-Type': 'application/json'}

        payload = {"refresh_token": token}

        resp = self.client.post(url, headers=headers, data=json.dumps(payload))

        self.access = resp.json()['access_token']

        self.refresh = resp.json()['refresh_token']



    @task(1)

    def get_file(self):

        uuid_list = [

            '7ca74e17-ac8a-48fc-84ed-0e947cb4c333',

            '666e4a5b-6704-41e9-b08d-b18dd1d50c0f',

            'c8c17345-d533-41b0-bbd5-5e39d023fcb6',

        ]

        path = f'/files/{random.choice(uuid_list)}'

        headers = {'Content-Type': 'application/json', 'Authorization': f'Bearer {self.access}'}

        with self.client.get(url=path, headers=headers, catch_response=True) as response:

            if response.status_code == 401 and response.json()['status'] == "UnauthorizedError":

                self.refresh_token(self.refresh)

他就是一個正常的python module，我們可以直接import 需要的packages

from locust import HttpUser, task, between

import json

from test_data import *

import random

這邊define一個class 繼承HttpUser，讓我們可以用來發送 HTTP requests去測試我們要測試的API或系統。between 讓simulated users在一個隨機等待時間範圍區間發送requests，也可以用constant去固定每個task中間的等待時間。

class WebsiteTestUser(HttpUser):

    wait_time = between(0.5, 1)

@task decorator 可以對要測試的task設定權重比例，Locust會creates a greenlet (micro-thread)讓每一個模擬的user去呼叫我們主要要被測試的task methods，其他沒有@task decorator的internal methods如範例中的refresh_token 就只會在我們要呼叫的時候被使用，self.client attribute 可以用來發送 HTTP calls，也會被記錄在locust報告的統計資訊中。

@task(1)

def get_file(self):

    ...

    with self.client.get(url=path, headers=headers, catch_response=True) as response:

    ...

Run the script with GUI

locust -f locust_file.py

然後可以打開browser 到預設的port 8089， http://localhost:8089/ 去設定預期要模擬的人數跟每一秒要增加多少人到預期的量。

Running without the web UI

如果不用web UI也可以用terminal帶一些需要用到的flag去跑，--headless 代表不用web UI，-u代表要幾個users，-r代表spawn rate，-t 代表要跑多少時間，--html代表要產生的html report。

locust -f locust_file.py --headless -u 5 -r 1 -t 360s --html test_report.html

也可以用設定config file的方式去跑

locust --config=/load_testing/config.conf

config file example

locustfile = /load_testing/locust_file.py

headless = true

host = http://example.com

users = 10

spawn-rate = 1

run-time = 10s

html = locust_report.html

Trigger locust test with Gitlab CI

也可以利用Gitlab CI 設定variables在config file上，去跑測試

config file example

locustfile = /builds/project_name/load_testing/RUN_TARGET.py

headless = true

host = https://example.com

users = RUN_USERS

spawn-rate = 1

run-time = RUN_TIME

html = locust_report.html

Gitlab CI Yaml example

stages:

  - test

locust:

  stage: test

  variables:

    TARGET: ""

    USERS: "1"

    TIME: "10s"

  before_script:

    - cat /builds/project_name/load_testing/__locust.conf

    - sed -e 's/RUN_TARGET/'$TARGET'/1' -e 's/RUN_USERS/'$USERS'/1' -e 's/RUN_TIME/'$TIME'/1' /builds/project_name/load_testing/__locust.conf > /builds/project_name/load_testing/locust_run.conf

    - cat /builds/project_name/load_testing/locust_run.conf

  script:

    - locust --config=/builds/project_name/load_testing/locust_run.conf

  rules:

    - if: $TARGET

  allow_failure: true

  artifacts:

    when: always

    paths:

      - locust_report.html

    expire_in: 1 week

Another Gitlab CI Yaml example 不用config file也不用variables

stages:

  - test

locust:

  stage: test

  image: /runner-image/qa-python:3.7.7-20220719

  variables:

    GIT_SSL_NO_VERIFY: "1"

  script:

    - locust -f load_testing/test1.py --headless -u 10 -r 1 -t 360s --html report1.html

    - locust -f load_testing/test2.py --headless -u 10 -r 1 -t 360s --html report2.html

    - locust -f load_testing/test3.py --headless -u 10 -r 1 -t 360s --html report3.html

  rules:

    - changes:

        - requirements.txt

  allow_failure: true

  artifacts:

    when: always

    paths:

      - report1.html

      - report2.html

      - report3.html

    expire_in: 1 week

總結

這邊簡單介紹效能測試工具locust，也介紹了幾種方式去跑這個測試工具，透過效能測試我們可以知道我們測試的系統大概可以承受多少的交易量，反應時間是否有達到使用者都能順暢使用的標準，或是業務上應該要達到的標準，超出負荷的時候，系統是怎麼樣回應，下降回可承受的量是否能回復正常運行，在跑長時間測試的時候看看回應是不是都是很穩定，是否有不如預期的行為產生，看看是硬體資源(CPU/Memory/Disk IO)是否需要調整，看看網路端是否異常，DB連線數量上面是否被限制，又或是程式碼效能需要再調整，總之效能測試是一種很重要的非功能性測試，開發人員或測試人員可以在開發好一個API的時候就先測試看看，比較底層的API或服務，可能dependency比較少比較好釐清問題，早點測試早點發現問題先處理成本也會比較低，祝福大家系統的品質透過測試都能越來越好。