Create and Automate Grafana K6 API load testing through GitHub Action

 If you are new to Grafana K6 API load testing then I would recommend going through my initial post on API load testing using k6.

In this article here, I’m going to talk about automating your K6 load test APIs GitHub Action pipeline and integrating with Grafana cloud to publish the test run result.

Create your Grafana K6 Cloud Account

First visit Grafana K6 Cloud and create your free account which we will use for monitoring our test results on the cloud. Once you registered, grab an API Cloud Token from the setting under Testing & synthetics=> Performance blade from the left-hand side menu.

Next, let's create a project structure in an organized way to automate it through the pipeline, here I’m going to use my favorite GitHub Action. Follow the steps to do so.

Create K6 Project

Step 1: Create your project folder and two folders as env and tests and a javascript file main.js using your favorite IDE tool. I’m using again my favourite Microsoft Visual Studio Code.

Step 2: First we will set up env-related settings. Let’s create a folder as ‘test’ inside k6=>env and add two JSON files for settings related to tests, here I would define load and stress tests and then a file as settings.json which will have Tests related settings:

1. config.load.json: In this file, we will add settings for required VUs, duration, tags, etc for our tests.

{
    "vus": 1,
    "duration": "60s",
    "tags": {
      "test_type": "load",
      "api_name": "Test API"
    }
}

2. config.stress.json

{
    "vus": 10,
    "duration": "60s",
    "tags": {
      "test_type": "stress",
      "api_name": "Test API"
    }
}

3. settings.json

  {
    "SETTINGS": {    
      "baseUrl": "#{APIURL}#"
    },
    "TEST_FILTERS": {
      "enabled": false,
      "startsWith": null,
      "endsWith": null,
      "contains": null,
      "regex": null
    },
    "ENVIRONMENT": {
      "execution": "test",
      "optionsSet": "load",
    },
    "AZUREAD": {
      "tenantId":"#{TENANTID}#",
        "clientId": "#{CLIENTID}#",
        "clientSecret":"#{CLIENTSECRET}#", 
        "userName":"#{USERNAME}#", 
        "password":"#{PASSWORD}#"
    }
  }

in settings.json, I have defined four categories as URL settings, test filters (to filter tests to exclude/include in the run), environment ( to choose between load/test, and based on that one of the above two files config.load/stress.json will be loaded for VUs related settings and in last AzureAD category to define settings required to pull Azure AD token.

Step 3: create a folder ‘çommon’ inside k6=>tests and add two utility methods to get the token and another one to create an HTTP object for API calls.

1. getAuthToken.js.

import http from 'k6/http';
import { check } from 'k6';
import exec from 'k6/execution';

export function GetToken(tenantId, clientId, clientSecret, userName, password)
{   
  const authUrl = `https://login.microsoftonline.com/${tenantId}/oauth2/v2.0/token`;

  const payload = {
    client_id: clientId,
    client_secret: clientSecret,
    scope:`api://${clientId}/api.scope`,
    grant_type: 'password',
    username: userName,
    password: password
  };

  let options = {
    vus: 5,
    duration: "10s",
    thresholds: {  'http_req_duration{scenario:default}': [   `max>=0`,     ],    },
    }
   const headers = { 'Content-Type': 'application/x-www-form-urlencoded' };    

  const res = http.post(authUrl, payload, { headers, },  { options });
   if (!check(res, { 'Token Fetched': r => r.status == 200, }))
   { 
       console.log("Getting Auth token failed:",res);
        exec.test.abort("Abort Tests: Error! Getting Auth Token", res);
       return '';
   }
   
   console.log("your token is ---------- ", res.json()); 
   console.log("your token is ---------- ", res.json()["access_token"]); 
  
  return  res.json()["access_token"]; 
};

2. utility.js

//import modules
import { Httpx } from 'https://jslib.k6.io/httpx/0.0.1/index.js';
import { check } from "k6";
import { Rate } from 'k6/metrics';
//Error Rate is an object for representing a custom metric keeping track of 
//the failures
export const errorRate = new Rate('errors');

export function GetHttpObject(data) {
    //create httpx session
    const session = new Httpx({
        baseURL:  data.SETTINGS.baseUrl,
        timeout: 10000 // 10s timeout. 
    });
    //set headers
    session.addHeaders(
        {
            'Authorization': `Bearer ${data.JWT_TOKEN}`, 
            'User-Agent': 'My k6 custom user agent', 
            'Content-Type': 'application/json',
        }
    );
    return session;
}

Step 4: We will add a test by creating a module module-wise folder inside k6=> tests. For example, I added a folder called ‘user’ and user-tests.js inside the user folder. i.e. k6=>tests=>user=>user-tests.js with below code:

import http from 'k6/http';
import { check, sleep, group } from 'k6';

export let UserTests = [
    GetUserAccess, GetUserInterest
];

function GetUserAccess(httpsession) {
    group('GET User Access', function () {
        let res = httpsession.get(`/user/access`);

        check(res, {
            'status was 200': (r) => r.status == 200,
            "Got response in 2 seconds": (r) => r.timings.duration < 2000,
        });
    });
}

function GetUserInterest(httpsession) {
    group('Get User Interest', function () {
        let res = httpsession.get(`/user/interest`);
        check(res, {
            'status was 200': (r) => r.status == 200,
            "Got response in 2 seconds": (r) => r.timings.duration < 2000,
        });
    });
}

The above code, has first import statement for the dependencies and then a variable with an array of all defined method-test names followed by test code for methods/API where we have two assert statements, one to check the successful response and another for response time as less than 2 secs). You can add more assert statements based on your needs.

The variable ‘UserTests’ we define here will be used in the main file to list and run defined tests in this file.

Step 5: Now we add the main.js file, which is the starting/source file to be executed through the test command, to the project folder. i.e. K6=>main.js with below code.

import { sleep } from 'k6';
import {GetHttpObject} from './tests/common/utility.js';
import {GetToken} from './tests/common/getAuthToken.js';
//import tests
import { UserTests } from './tests/user/user-tests.js';

let ENVIRONMENT = {};
ENVIRONMENT.execution = "local";//default
if (__ENV.EXECUTION) {
    ENVIRONMENT.execution = __ENV.EXECUTION;
}

ENVIRONMENT.optionsSet = "load";//default option
if (__ENV.OPTIONS_SET) {
    ENVIRONMENT.optionsSet = __ENV.OPTIONS_SET;
}

//load tests
 let TESTS = [...UserTests];


// Load k6 Run Options
let optionsFile = `./env/${ENVIRONMENT.execution}/config.${ENVIRONMENT.optionsSet}.json`;
export let options = JSON.parse(open(optionsFile));

// Load test settings
let DATA = JSON.parse(open(`./env/${ENVIRONMENT.execution}/settings.json`));
DATA.ENVIRONMENT = ENVIRONMENT;

// Filter the tests to run
let TESTS_TO_RUN = [];

if (__ENV.TEST_FILTERS) {
    DATA.TEST_FILTERS.enabled = true;
    let tokens = __ENV.TEST_FILTERS.split('|');
    DATA.TEST_FILTERS[tokens[0]] = tokens[1];
}

if (DATA.TEST_FILTERS.enabled) {
    TESTS_TO_RUN = TESTS.filter(t => {

        console.debug(`Test filters. Probing ${t.name}`);
        // starts with
        if (DATA.TEST_FILTERS.startsWith != null &&
            t.name.startsWith(DATA.TEST_FILTERS.startsWith))
            return true;
        // ends with
        if (DATA.TEST_FILTERS.endsWith != null &&
            t.name.endsWith(DATA.TEST_FILTERS.endsWith))
            return true;
        // contains
        if (DATA.TEST_FILTERS.contains != null &&
            t.name.indexOf(DATA.TEST_FILTERS.contains) != -1)
            return true;
        // regex
        if (DATA.TEST_FILTERS.regex != null &&
            t.name.match(DATA.TEST_FILTERS.regex))
            return true;
        return false;
    });
}
else {
    TESTS_TO_RUN = [...TESTS];
}

export function setup() {
    if (__ENV.K6_CLOUD_TOKEN) {
        console.log('Reading from environment variable');
        DATA.JWT_TOKEN = __ENV.K6_CLOUD_TOKEN; //reading from environment variable.
    }
    else { //fetch token through Microsoft graph api.
        console.log('Fetching token from Microsoft Graph API call');
        DATA.JWT_TOKEN = GetToken(DATA.AZUREAD.tenantId, DATA.AZUREAD.clientId, DATA.AZUREAD.clientSecret, DATA.AZUREAD.userName, DATA.AZUREAD.password);
    }
    return DATA;
}
export default function (data) {
    const httpsession = GetHttpObject(data);
    TESTS_TO_RUN.forEach(t => { t(httpsession); sleep(1); });
    sleep(1);
}

export function teardown(data) {
    // teardown code
}

In the above file here, after the import statements where we imported the dependencies and the tests variable created in the user-tests.js file, we define the environment local/test/prod (the way you name it, in this case, we created only one environment as ‘test’) and test options as load/stress.

In the next steps, we read the options for load/stress tests and variables from the settings file which will be de-tokenized through the pipeline before execution, and then we have a logic to apply test filters to include/exclude tests if we define any such test filter token in settings.json file inside TEST_FILTERS section.

Next, we have defined the setup() method to perform the necessary setup for the test run. for example, setting up an auth token, etc. Note, that all variables are available on the DATA variable which includes settings for the settings.js file as well.

Next, we have a default method that will initiate API calls and tests one by one for each API module defined or filtered and available as part of the TESTS_TO_RUN variable.

Till here our project is fully ready and next, we will create a GitHub action pipeline to automate this.

Step 6: Create a GitHub action pipeline ‘k6_api_tests.yml’ in your .github folder for your GitHub project location.

name: K6 API performance test automation. 

on:
  push:
    branches:
      - main
    paths:
      - 'API/**'
  workflow_dispatch:
  
env:
  WORKING_DIRECTORY: './k6'

jobs:
  k6-test-execution:
    runs-on: ubuntu-latest
    environment: testenv
    defaults:
      run:
        working-directory: ./k6

    steps:
      - uses: actions/checkout@v4

      - name: Test env Settings Variable Substitution
        uses: microsoft/variable-substitution@v1
        with:
          files: "**/settings.json"
        env:
          SETTINGS.baseUrl: ${{secrets.API_PATH}}
          AZUREAD.tenantId: ${{ secrets.MSALTenantID}}
          AZUREAD.clientId: ${{ secrets.MSALClientID}}
          AZUREAD.clientSecret: secrets.MSALClientSecret}}
          AZUREAD.userName: ${{ secrets.AutomationTestUserName}}
          AZUREAD.password: ${{ secrets.AutomationTestUserPassword}}

      - name: Run k6 local test
        uses: grafana/k6-action@v0.3.1
        with:
          filename: ./k6/main.js
          flags: -e OPTIONS_SET="load" -e EXECUTION="test"
          cloud: true
          token: ${{ secrets.K6_CLOUD_API_TOKEN }}
          

In the above file, we are first de-tokenizing the variables in the settings.js file in the step ‘Test env Settings Variable Substitution’.

Finally, we have step ‘Run k6 local test’ where we use the ‘grafana/k6-action@v0.3.1’ GitHub action which is from Grafana. This action has four params as:
filename: your main.js file path,
flags: any flags like env variables input from the command i.e. OPTIONS_SET and EXECUTION in our case.
cloud: boolean variable, true means it will expect the K6 cloud API token to publish the result.
token: your K6 Cloud API Token which you grabbed from the Grafana K6 Cloud portal.

With all this, you are done. Now you run the pipeline and you will see the result in the Grafana K6 cloud dashboard as below:

Hope you enjoyed the content, follow me for more like this, and please don’t forget to LIKE it. Happy programming.