QA Skills with LLM Agents — RAG + Playwright
Jan 2026 – Present
Problem
Manually writing test cases from user stories in Jira and Confluence consumed ~50 hours per sprint, creating QA bottlenecks. An isolated pipeline was not enough: the savings had to hold up in production and reach every team.
Architectural solution
RAG pipeline that ingests Jira and Confluence documentation (pgvector + ChromaDB, OpenAI/Anthropic API) and generates contextualized test cases. Refactored into modular skills inside the internal LLM-agent toolkit: test-case generation, automatic Jira upload, endpoint-to-Postman collection mapping (happy and unhappy paths), and Playwright E2E test generation that runs each case and captures evidence (screenshots and video).
Impact & Value
- 70.9% reduction in test-case creation time (50h → 14.55h per sprint)
- Skills in production, used daily by QA teams
- Automated traceability: user story → test case → E2E test with evidence
Technologies
Technologies