Quality Supervisor is an open-source plugin that adds QA analysis skills to your AI client. It reads your Qase project and answers questions about coverage, failures, flakiness, and release readiness, and shows the QQL query behind every number it reports.

It runs in Claude Code, Claude Desktop, Cowork, and other MCP-capable clients such as Cursor, VS Code, or Atlassian Rovo. It reaches Qase through the Qase MCP Server, so there is nothing to install on the Qase side.

The plugin analyses; Qase stores. Findings only become cases, defects, or tags in Qase when you approve them.

ℹ️ Official repository and full documentation:github.com/qase-tms/qase-quality-supervisor

What's inside

Component	Name	What it does
Skill	`finding-coverage-gaps`	Finds untested requirements, empty suites, and un-automated critical paths. Drafts the missing cases if you ask.
Skill	`triaging-test-failures`	Clusters a run's failures by error signature, then separates product bugs from automation, environment, and flaky noise. Files and links defects on approval.
Skill	`analyzing-test-flakiness`	Finds cases that both pass and fail, confirms by re-run, and recommends quarantine or fix.
Skill	`assessing-release-readiness`	A five-dimension go / no-go gate for a milestone, test plan, or set of runs.
Skill	`reporting-quality-pulse`	An HTML report card for a period you choose: execution, pass rate, defect backlog, case activity, active users.
Skill	`analyzing-change-impact`	Diffs a git branch, traces what it reaches, and returns a QA impact analysis plus a regression checklist mapped to your existing cases.
Agent	`quality-supervisor`	Routes a broad quality question to the right skills and combines the results.
Command	`/quality-supervisor:quality-report`	A read-only sweep across coverage, flakiness, triage, and readiness in one pass.

Each skill works on its own. The agent and the command exist for when you want more than one analysis at a time.

Requirements

Requirement	Detail
Qase plan	Teams and Enterprise. The analyses run on QQL, which the free plan does not include.
Qase MCP Server	Version 2.1.1 or newer. The hosted endpoint needs Enterprise; on Teams tier, you can run the server yourself.
Project code	The uppercase prefix on your case IDs. In `DEMO-42`, the code is `DEMO`.
Your source Git repository	Only for `analyzing-change-impact`, and only on a branch that differs from the default branch. It is the one skill that reads code.

Note: Quality Supervisor does not consume Qase AI credits. The model runs in your own AI client, so that client's provider bills the compute. Qase serves the data.

Install

The repository is both the plugin and its marketplace. In Claude Code:

/plugin marketplace add qase-tms/qase-quality-supervisor
/plugin install quality-supervisor@quality-supervisor

Restart the session, then run /plugin to check it loaded.

In Cowork, install the packaged plugin file directly, or add the repository as a marketplace if your build supports it.

Connect to Qase

Hosted, the default. The plugin ships pointing at https://mcp.qase.io/mcp. The first time a skill needs Qase, your client runs the OAuth flow and you sign in with your normal Qase login. No API token is created or stored.

Self-run. Use this if you're on the Teams plan, or in a headless session where no browser can open. Create a token in Qase under Settings, then API tokens, set it in your environment, and replace the plugin's .mcp.json with:

{
  "mcpServers": {
    "qase": {
      "command": "npx",
      "args": ["-y", "@qase/mcp-server"],
      "env": { "QASE_API_TOKEN": "${QASE_API_TOKEN}" }
    }
  }
}

Keep the server name qase. The plugin's safety hooks match tool names by that prefix, and renaming the server disables them without warning.

Run your first report

/quality-supervisor:quality-report DEMO

Add a milestone, plan, or run to scope the release gate. Without one, the readiness dimension is skipped.

/quality-supervisor:quality-report DEMO Release 1.0

The sweep is read-only. It covers coverage, flakiness, triage of the latest run, and readiness if you scoped it.

💡The full sweep is not quick.On a project with 149 cases, 3 milestones, and 78 defects it takes about 5 minutes and 56 tool calls on a large model. Cost scales with the number of dimensions you ask for, not with project size. Use it before a release or as a weekly check, and ask a single question the rest of the time.

Ask in your own words

You do not need the command. These reach the right skill on their own:

Question	Skill
"Where are our coverage gaps in DEMO?"	coverage gaps
"Which cases are still manual?" or "which suites are empty?"	coverage gaps
"Triage the latest run in DEMO and tell me what's a real bug."	failure triage
"Which DEMO tests are flaky, and which should we quarantine?"	flakiness
"Are we ready to ship milestone 2.3?" or "what's blocking the release?"	release readiness
"Give me a quality pulse for DEMO for the last two weeks."	quality pulse
"What should QA test for the changes on this branch?"	change impact

Naming Qase in the question helps. A bare "suites" or "release" could belong to any tool you have connected.

Naming a skill makes it certain

Which skill fires is a judgement the model makes, and it is not deterministic. In the maintainers' testing over 180 runs, the right skill fired for a natural question about 91% of the time. It did not fire the wrong one; it sometimes answered without firing any. On smaller models the rate drops, though the analysis holds up once a skill does run.

A command or a named skill is an instruction rather than a routing decision, so it always runs:

/quality-supervisor:quality-report DEMO
/quality-supervisor:finding-coverage-gaps
/quality-supervisor:triaging-test-failures
/quality-supervisor:analyzing-test-flakiness
/quality-supervisor:assessing-release-readiness
/quality-supervisor:reporting-quality-pulse
/quality-supervisor:analyzing-change-impact

The quality-supervisor: prefix is required. A bare /quality-report comes back as an unknown command.

Reading the output

Three things to know before you act on a report.

Data confidence. Each report ends with what could not be measured: a truncated list, an empty time window, a field nobody fills in. If it says defect tracking is not in use, then "no blocking defects" means nobody is recording them, not that there are none.
Every number has a denominator."40 cases are not automated" reads differently against 50 cases than against 18,000, so both numbers are always shown.
Flaky and failing are separate findings. A case that both passes and fails is flaky, so fix or quarantine the test. A case that only ever fails is a regression, so fix the product.

Writing back to Qase

The sweep writes nothing. Ask directly when you want a change, and the skill proposes it first:

"Draft the missing cases for the billing suite."
"File defects for the real bugs in run 512."
"Flag those flaky tests in Qase."

Bulk writes show you a sample before the rest go through. Deletion is blocked in the plugin's code and cannot be approved, including a DELETE sent through the MCP server's raw-API tool.

Permissions and data

Every call runs as you, under your Qase roles and permissions. Nothing authenticates as the plugin.

The skills read project metadata, suites, milestones, cases, runs, results, and defects. reporting-quality-pulse also reads the project's user list to name who was active in the period. analyzing-change-impact reads your local git diff, which stays on your machine.

The repository's SECURITY.md documents the full data flow, and marks which claims are covered by automated tests.

Troubleshooting

Symptom	Cause
`Unknown command: /quality-report`	The prefix is required: `/quality-supervisor:quality-report`.
A skill says it has no access to Qase data	The MCP server is not connected or not authorised yet.
"QQL is only available in Teams and Enterprise"	A plan limit. There is no workaround in the plugin.
A question gets a general answer and no analysis	Routing missed. Name the skill, or use the command.
Suite or milestone lists look short	`qase_project_context` caps at 100 items per collection and says so. Treat them as a sample.
Numbers disagree with the Qase UI	Expected on older projects: the search index holds no historical backfill. The report flags this under data confidence.

Support

Wrong answer or a skill that did not fire: open a misfire report. The form asks for your prompt word for word, because routing depends on exact wording.
Anything else: GitHub Issues, our live chat, or [email protected].

Qase MCP Server