ℹ️ Documentation for Qase API and Reporters is at https://developers.qase.io/docs/

Qase CLI orqliis Qase's command line app build in Go Language, which can be used to start and complete test runs, and it can publish results to Qase in several formats.

View source code on GitHub here: qase-tms/qasectl

How to install the app?

Install via Homebrew (macOS/Linux)

brew tap qase-tms/tap
brew install qasectl

Install via go install

go install github.com/qase-tms/qasectl@latest

Make sure to add $GOPATH/bin to your $PATH environment variable to be able to run the qasectl command. The binary is named qasectl, you may want to create an alias, as qli

Build from source

Clone the repository

git clone https://github.com/qase-tms/qasectl.git && cd qasectl

Build the binary

make build

You will find the binary in the build/ directory.

Try creating a test run by executing the binary -

./build/qli testops run create --project QD --token <your-token> --title "Run created from Qase-cli" --description "Hello, from qase-cli" --environment <env-slug> --verbose

Terminal cloning qasectl, running make build, then creating a test run with ./build/qli testops run create

Use a Docker image

Pull the Docker image

docker pull ghcr.io/qase-tms/qase-cli:latest

Run the Docker container

docker run --rm ghcr.io/qase-tms/qase-cli:latest <add-your-command-here>

Using Qase CLI

Please find the complete documentation of the available options here -

https://developers.qase.io/docs/qase-cli#/

Configuring Qase CLI in your CI workflow

You can use qli in cases where your test runner generates run reports in one of these formats (Junit; Qase; Allure; XCTest; Maestro).

Examples

GitHub

An example github.yml, that uses Qase GitHub Actions.

name: Run Playwright Tests and Report to Qase

on:
  push:
    branches:
      - docker-qli

jobs:
  test:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Node.js
        uses: actions/setup-node@v4
        with:
          node-version: "18"

      - name: Install dependencies
        run: npm install

      - name: Run Playwright tests
        run: npx playwright test
        continue-on-error: true

      - name: Get current date
        id: get-date
        run: echo "date=$(date +'%Y-%m-%d %H:%M:%S')" >> $GITHUB_OUTPUT

      - name: Create a Qase test run
        uses: qase-tms/gh-actions/run-create@v1
        id: qase-run-create
        with:
          token: ${{ secrets.QASE_API_TOKEN }}
          project: ${{ secrets.QASE_PROJECT_CODE }}
          title: "Playwright Test Run ${{ steps.get-date.outputs.date }}"
          description: "Automated Playwright test run"
          environment: "dev-web-app"
          verbose: true

      - name: Upload test results to Qase
        uses: qase-tms/gh-actions/report@v1
        continue-on-error: true
        with:
          token: ${{ secrets.QASE_API_TOKEN }}
          project: ${{ secrets.QASE_PROJECT_CODE }}
          id: ${{ steps.qase-run-create.outputs.id }}
          format: junit
          path: results/test-results.xml
          batch: 100
          verbose: true

      - name: Complete a Qase test run
        uses: qase-tms/gh-actions/run-complete@v1
        id: complete
        # use always() to run even if test step failed or job was canceled
        # but don't run if creating a test run failed before this step
        if: always() && steps.qase-run-create.outcome == 'success'
        with:
          token: ${{ secrets.QASE_API_TOKEN }}
          project: ${{ secrets.QASE_PROJECT_CODE }}
          id: ${{ steps.qase-run-create.outputs.id }}
          verbose: true

GitLab

An example .gitlab-ci.yml, that uses Qase CLI in a docker container.

stages:
  - test
  - publish

variables:
  PROJECT_CODE: ${QASE_PROJECT_CODE}
  API_TOKEN: ${QASE_API_TOKEN}
  RESULTS_FILE: "results.xml"
  TEST_RUN_TITLE: "Test Run $(date +'%Y-%m-%d_%H-%M-%S')"
  TEST_RUN_DESCRIPTION: "Test run description"
  OUTPUT_FILE: "qase.env"

test_and_report:
  stage: test
  image: ubuntu:20.04
  before_script:
    - apt-get update && apt-get install -y curl gnupg
    - curl -fsSL https://deb.nodesource.com/setup_18.x | bash -
    - apt-get install -y nodejs
    - node -v && npm -v
    - npm install
    - mkdir results

  script:
    - npx playwright test --reporter=junit --output=/results/${RESULTS_FILE} || true
  artifacts:
    paths:
      - /results/${RESULTS_FILE}

publish:
  stage: publish
  image: docker:24.0.5
  services:
    - docker:24.0.5-dind
  script:
    - |
      docker run --rm -v "$(pwd)/results:/results" \
      ghcr.io/qase-tms/qase-cli:latest testops run create \
      --project ${PROJECT_CODE} \
      --token ${API_TOKEN} \
      --title "${TEST_RUN_TITLE}" \
      --description "${TEST_RUN_DESCRIPTION}" \
      --output /results/${OUTPUT_FILE}

    # step to parse the RUN_ID from the output file
    - RUN_ID=$(cat results/${OUTPUT_FILE} | grep QASE_TESTOPS_RUN_ID | cut -d'=' -f2)

    # publishes results to Qase
    - |
      docker run --rm -v "$(pwd)/results:/results" \
      ghcr.io/qase-tms/qase-cli:latest testops result upload \
      --project ${PROJECT_CODE} \
      --token ${API_TOKEN} \
      --id ${RUN_ID} \
      --format junit \
      --path /results/${RESULTS_FILE} \
      --verbose

    # marks the Qase test run as complete.
    - |
      docker run --rm \
      ghcr.io/qase-tms/qase-cli:latest testops run complete \
      --project ${PROJECT_CODE} \
      --token ${API_TOKEN} \
      --id ${RUN_ID} \
      --verbose

Maestro

Qase CLI can upload the results of a Maestro 2.x run (Maestro 1.x is not supported). This needs Qase CLI v0.6.0 or later. Run your flows so the debug output and the JUnit report are saved to the same folder, then upload that folder with --format maestro:

maestro test --debug-output results/maestro-run --flatten-debug-output --format junit --output results/maestro-run/report.xml .maestro/
qli testops result upload --project PROJ --token <your-token> --id <run-id> --format maestro --path results/maestro-run --verbose

Each flow becomes one test result, and its commands become steps. Screenshots are attached to their steps, and logs are attached to failed results. The --path folder must hold one run only.

To link a flow to existing test cases, add qase.id to the flow's properties. Several IDs go in one comma-separated string, for example qase.id: "123, 456". The full list of Maestro options is in the developer documentation.

Qase CLI App