Skip to content
HamstikHamstik Docs

Quality & Testing

Quality & Testing

Quality & Testing is an opt-in Project area for planned verification. A test plan organizes reusable case definitions into suites, and runs record immutable execution evidence against those definitions. Projects that never create a plan see no change anywhere in Hamstik: no required artifacts, no new Work Item statuses, and no extra fields. Test-case coverage is never presented as source-code coverage.

Plans, suites, and cases

A test plan carries a name, description, and optional sprint or release association. Plans move through Draft → Active → Completed → Archived; reopening a completed plan is allowed and archiving is permanent. Only active plans accept new runs. A Project can hold up to 100 plans in all states.

A suite groups cases inside a plan with a fixed kind: functional, smoke, regression, integration, security, accessibility, acceptance, or custom. A plan can hold up to 50 suites.

A test case is the reusable definition: title, purpose, preconditions, ordered steps each with an expected result, an overall expected outcome, an owner, and an active | retired state (up to 100 steps, 2000 characters per step or expected result, 500 for the title). A case can link to Work Items of the same Project, and a case can belong to any number of suites. Editing a case bumps its revision; retirement is terminal but keeps every identifier resolvable so historical evidence keeps working.

Definitions are managed in the Project Quality & Testing area. Members can view everything; creating and editing definitions is limited to administrators and owners.

Runs

A run executes the cases of one active plan, optionally narrowed to a single suite. Runs carry a name, an optional target environment, and an optional build/version identifier. Run states are Not Started → In Progress → Completed or Aborted; Completed and Aborted are terminal and recorded results are never changed by completing or aborting. Recording the first result starts the run automatically.

The run header shows progress: how many cases are targeted, how many are executed, and how many of the latest results passed, failed, blocked, or were not run.

Results and immutable evidence

Every recorded result captures: tester identity, execution timestamp, outcome (passed, failed, blocked, or not_run), actual-result notes, the environment, and build/version identifiers, plus the execution sequence within its run and case.

Results are append-only. A rerun appends a new result; a correction supersedes the previous result and both stay readable as history. Recorded results can never be edited or deleted — not through the UI and not by later case edits.

The first execution of a case inside a run also captures a snapshot of the case definition exactly as executed. Later edits to the case never invalidate that snapshot: the run shows the executed definition revision alongside the current one, and every result in the run resolves to the same executed definition.

Files such as screenshots or logs attach to a result as evidence. Evidence rides Hamstik’s normal attachment storage and quota accounting, is append-only, and never modifies the result it documents. Historical runs and results stay queryable, so past evidence remains available even after definitions change.

Verification and signoff

Work Items whose required test cases are executed in runs carry a derived verification state, shown in a dedicated panel on the Work Item detail page. The state is never stored — it is recalculated from live execution evidence — and it is entirely separate from the Work Item’s workflow status: verification activity never changes status, priority, or any other field.

The states, in evaluation order:

  • Not required — the Work Item has no test cases linked as required for verification. Projects that do not use test management simply never see the panel.
  • Verification failed — the latest execution of any required case failed.
  • Verified — an explicit signoff exists and no required case has executed after it, or every required case’s latest execution passed.
  • In verification — at least one required case executed, but some are still unexecuted or blocked / not run.
  • Awaiting verification — required cases exist but none executed yet.

Failed reruns flip a verified item back to Verification failed; a passing rerun verifies it again without needing a new signoff.

A signoff is an explicit human approval on the Work Item: it records who approved, when, optional notes, and a pinned snapshot of the verification evidence (the derived state and latest results) plus its digest at approval time. Signoffs are immutable — a revocation or a re-approval is recorded as a new entry that supersedes the previous one, and the full history stays readable. Members with test-execution permission can sign off; automation never does.

Defects and regression

Defects are ordinary Bugs — Work Items of the existing bug type. No new type, status, or field is introduced. From a failed result you can create a Bug (its title and description are prefilled with the execution context: the case, run, environment, build/version, tester, timestamp, actual-result notes, and the reproduction steps exactly as executed) or link an existing Bug by key. Links store ids only and render through the normal Work Item services, so nothing is duplicated. Unlinking removes just the link row; the failed result is immutable, so the case → run → result → defect chain stays navigable even after the Bug is resolved or closed. Creating or linking follows the same project membership and Work Item rules as creating any Bug — it never bypasses project access.

The Bug’s detail page shows the linked failed results and the associated regression tests. A fixed Bug can be associated with one or more regression test cases; the regression status is derived from each associated case’s latest result — still reproduces (failed), fix verified (passed), or not executed yet — together with the run, environment, and build that established it. Reruns append new results and earlier executions stay readable, so a defect that reappears later is visible with the exact build and environment that established it.

The Regressions tab lists failed results joined through defect links and regression associations, filterable by plan, suite, environment, build/version, defect, and affected Work Item, and groupable by defect, case, plan, environment, or build. This is a structured quality-domain query — Work Item queries (SqueakQL) remain Work-Item-only.

Reporting

The Reports tab produces quality reports for a test plan, a sprint, or a Release Version. Counting uses the latest execution per planned test within the scope, so the numbers always add up: planned = executed + unexecuted, and executed = passed + failed + blocked + not run.

Two percentages are calculated and presented independently and are never combined into a single “test coverage” figure:

  • Execution — how much of the planned testing ran (“90% of planned tests executed”).
  • Pass rate — how much of the executed testing passed (“96% of executed tests passed”).

Scopes with zero planned tests report zero counts and no percentages — never a misleading 100%.

Coverage views stay link-based and separate from execution: Work Items with linked test cases versus Work Items without any, and Work Items whose derived verification state is Awaiting verification. Test-case coverage counts linked test cases; it is not source-code coverage, and Hamstik never presents it as such — every report carries that disclaimer.

Reports also surface defects originating from test failures (Bugs linked to failed results, with the latest failure’s run, environment, and build) and unresolved regression failures (associated regression cases whose latest execution still fails). The same calculation service feeds the session API and later quality gates, so every surface agrees on the numbers.

Release quality gates

Project administrators can opt into a Release quality gate from the Quality & Testing area. Projects without a configured gate keep the normal release workflow. A gate can require any combination of required test execution, no latest required-test failures, no latest blocked required tests, verified release Work Items, and an approved current release signoff.

When a gate is unmet, releasing returns every unmet condition and its evidence without changing the Release Version state. A project administrator may make an exceptional release by entering an override reason during the transition. The override snapshots the unmet conditions, reason, actor, and configuration revision in immutable Quality history; it documents the exception and never changes the underlying test or verification evidence.

Public API

Automation integrates through Public API v1 with the quality:read and quality:write scopes: plans, suites, cases, coverage links, runs, results, evidence, defect links, verification states and signoffs, gate configuration, and gate evaluation. Automated systems publish execution results with explicit automation provenance (testerKind: "automation" plus a label and bounded metadata) and never impersonate a human tester or record signoffs. Results stay append-only immutable evidence no matter which surface records them.

Bounds

  • Results per run per case: 50
  • Run names: 1–200 characters
  • Environment identifiers: 0–100 characters
  • Build/version identifiers: 0–200 characters
  • Notes: 0–10000 characters
  • Signoff notes: 0–2000 characters