Satellite Application QA & Agentic Automation

Background

The challenge

• Zero existing testing footprint: The satellite service was a greenfield application with no manual test documentation, no automated framework, and no CI/CD integration.
• Complex SFTP and async domain logic: Verification meant validating multi-step, asynchronous file transfers and payload transformations, including edge cases where external provider responses could be delayed, malformed, or incomplete.
• Strict handover requirement: The team was not just building a test suite to run internally. Every strategy document, resource file, and test script had to be structured for an eventual ownership transfer to the consumer app's QA team.
• Compressed delivery timeline: Development resources were tightly constrained from the start, forcing QA to establish baseline test coverage while functional specifications were still actively shifting.
• Architectural untestability: Core features, such as the 24-hour recurring validation loop and guaranteed session-matching logic, were buried inside asynchronous background workers. This made true end-to-end testing impossible without dedicated dev-side hooks.

Our approach

• Framework selection & Playwright stack: Evaluated framework choices against the project's async and API-heavy requirements. Recommended a Playwright (TypeScript) architecture for its native parallel execution, fast runner performance, built-in network interception, and first-class API testing capabilities. A major factor was also its clean integration model for LLM-assisted workflows, ensuring the codebase remained maintainable for the client's internal team post-handover.
• The agentic test harness: To stay aligned with spec-driven development, we integrated an internal agentic test harness powered by dedicated rule, skill, and routing configurations.

  • Input ingestion: The harness ingested functional requirements directly from developer PR specifications and Jira ticket descriptions.
  • Agent pipeline: A Test Designer Agent processed the spec to automatically generate Gherkin feature files, mock data resources, and structured test strategy documentation.
  • Management integration: Using a custom Model Context Protocol (MCP) server, generated test cases and sprint-specific manual campaign metadata were pushed directly into the client's test management system.
• The end-to-end delivery pipeline

  • Manual verification: QA executed the initial manual campaign pushed by the MCP server against newly delivered features.
  • Automated implementation: Once manual passes stabilized, a Test Automation Agent drafted the Playwright automation scripts covering functional, API, security, and performance tests.
  • PR execution & LLM analysis: Opening a QA pull request triggered a pipeline that executed the full test suite.
  • Automated reporting: Upon run completion, a Test Analyzer Agent parsed execution logs, evaluated edge-case failures, and generated a shippable result report with clear metrics.
  • Gate enforcement: If the report showed a clean run, the developer PR was approved for merge. If defects were detected, the Jira ticket automatically reopened and returned to the dev team with logs attached.
• Comprehensive test coverage: Built and executed targeted test suites across all critical risk vectors:

  • E2E / API suite: 200 automated scenarios covering SFTP message formatting, payload parsing, entity lookup parameters, and asynchronous response handling.
  • Security suite: 50 automated tests focused on payload injection, unauthorized SFTP pathing, credential exposure, and role-based access boundaries.
  • Performance suite: 20 load scenarios validating high-volume entity batching, concurrent SFTP connection limits, and 24-hour validation queue throughput under stress.

Working against the grain

• The hardest constraint was time, compounded by initial untestability.
• Because the satellite application relied heavily on time-based triggers, such as the 24-hour recurring validation sweep, testing end-to-end flows naturally required waiting full calendar days. Similarly, verifying that entity matches were correctly guaranteed across multi-request sessions could not be done reliably through the exposed interface alone.
• We had to push back on the development schedule and request specific architectural changes. The dev team spent limited engineering cycles building dedicated test endpoints: one to manually trigger the 24-hour validation loop on demand, and another to expose session state flags for payload matching verification.
• Midway through the engagement, client-side priorities shifted, forcing the team to split focus between new feature delivery and maintaining previously delivered modules that required retrofitting. Managing this back-and-forth under a rigid November deadline meant keeping the agentic spec-generation pipeline running continuously to prevent documentation drift.

Results

• 270+ total automated scenarios established from zero: 200 E2E/API, 50 Security, and 20 Performance test cases integrated directly into the CI pipeline.
• End-to-end agentic workflow deployed: Fully functional pipeline running spec parsing, Gherkin case generation, custom MCP test management sync, and automated execution analysis.
• Complete QA system handover: Delivered comprehensive test strategy documentation, resource maps, and clean Playwright code bases to the consumer application's QA team, completing the transfer of ownership without blocking active release cycles.
• Testability improvements merged upstream: Successfully advocated for and verified dedicated testability endpoints within the satellite application core, enabling deterministic E2E validation of background jobs.
• Multi-domain coverage delivered on time: Ran the full engagement from January to November, taking the satellite application from an unvalidated prototype to a fully tested, documented, and transferable microservice.

Key technologies and deliverables

Automation & testing stack

Playwright (TypeScript), REST/API testing framework, k6 / performance suites, security scan integrations

Agentic & spec-driven tooling

Agentic test harness (Test Designer, Test Automation, and Test Analyzer agents), Custom MCP server for test management tool integration, LLM execution log analyzer

CI/CD & orchestration

Automated execution pipelines, artifact extraction, pass/fail gating, automated PR status reporting

Documentation & handover

Full testing strategy documents, Gherkin feature specifications, resource maps, complete ownership transfer to consumer application QA team