CLI Tool QA
Systematic QA of CLI tools and pipelines across multiple real workspaces using SFDPOT heuristics. Executes structured test flows, evaluates consistency oracles, collects evidence, and produces a structured bug report with P1/P2/P3 priorities and a release readiness verdict.
#Ingegneria#Produttività#Analisi
Valutazione
Servono altre valutazioni
Venduti
1
Come si usa
Scarica
What It Does
CLI Tool QA guides an AI agent through rigorous evidence-based testing of any CLI tool or command pipeline across multiple real project workspaces. Follows a structured six-phase methodology producing a reproducible QA report and filed bug issues.
Methodology
- Survey (SFDPOT) - maps the full command surface across 7 dimensions: Structure, Function, Data, Platform, Operations, Interfaces, Time
- Workspace Selection - 3 workspaces covering different project stacks to catch ecosystem-specific and cache-isolation bugs
- Structured Test Flows - happy path, error path, edge cases, idempotency, state transitions, piping/composability
- Evidence Collection - reusable Python scripts, not ad-hoc one-liners; evidence is reproducible and re-runnable
- Consistency Oracles - Familiar, Explainable, Claims, Product, Purpose, Standards
- Fresh Eyes - clean-context agent uses the tool with no prior knowledge; catches UX problems structured tests miss
Sample Output
QA Report - my-tool v2.1.0
P1 Bugs: 2 filed
- list exits 0 on empty state but prints nothing (silent failure)
- --json flag produces invalid JSON when output contains unicode
P2 Bugs: 3 filed | P3: 1 filed
Verdict: NOT READY - resolve P1 bugs before release
Requirements
- Terminal access to run CLI commands
- 2-3 project workspaces for multi-ecosystem testing
- Optional: GitHub, GitLab, Linear, or Jira for issue filing
