QA Lead / Lead SDET

About

We are looking for a QA Lead with strong automation experience. Your team will consist of three people: two manual QA engineers and one SDET. You will be responsible for overall platform quality: testing strategy, automation, QA processes, and team growth. The QA team is currently being re-assembled, so how it operates will largely be up to you.

Main goal: make quality measurable in a product where LLMs answer the same query differently every time.

Platform core: APIs, event processing via message brokers, and LLMs. Working alongside are classifiers, embeddings, and models that route requests.

You will spend a significant part of your time writing automated tests and reviewing code written by others. AI agents are a standard development tool for us, and in QA we need someone whose automation workflow with them is already proven in practice.

Your superpowers

• 5+ years in QA testing, including 3+ years in automation: built and maintained test automation on a live product
• Extensive practical experience applying AI in automation: writing and supporting automated tests daily with AI agents (Claude Code, Codex, Cursor, or custom solutions), generating test cases and test data with them, and troubleshooting test failures. You know where an agent saves hours versus where it writes a test that verifies nothing, and you know how to teach this to the team
• Led a QA team containing both manual testers and automation engineers: planned work, reviewed test design, test cases, bug reports, and automated tests, conducted onboarding, and participated in hiring
• Strong manual testing skills: test design, exploratory testing, feature acceptance. Can teach manual testers and knows when checking manually is more cost-effective than automating
• Understand the full software development lifecycle from requirements and refinement to release, and know QA's role at each step. Ability to build QA processes from scratch: test cases, regression, acceptance, definition of done for features and releases. Know which processes hinder a small team
• Strong TypeScript and Playwright expertise: API and UI tests. Sufficient Python reading and editing skills: services, unit, and contract tests are written in Python using pytest
• Practical understanding of the testing pyramid: what to cover with unit tests in services versus e2e. Avoid verifying via UI what can be tested at the API level
• Write clean, maintainable code for a framework built to scale long-term. REST, SSE streaming, and API contracts: skilled in testing response streaming, connection drops, request cancellation, and timeouts. Knowledge of OpenAPI, schema-based type generation, and breaking change identification
• Proficient working with LLM APIs and agentic systems, knowing common failure points: tool calling, structured outputs, context window management, retries, and provider rate limits. Understand how to validate agent decision chains leading to an answer
• Evals: built them or ready to learn. Understand why binary pass/fail validation is insufficient for features with generative output
• Knowledge of CI/CD, SQL (PostgreSQL), Redis, and message brokers (Kafka, RabbitMQ, or similar). Docker is required; Kubernetes is a strong plus. Root cause investigation skills using logs, OpenTelemetry traces, and metrics
• Take ownership and product-minded mindset: proactively identify risks, follow bugs through to fix and regression test coverage. Recognize that testing financial transaction logic takes priority over visual styling checks
• Clearly explain issues to Developers, PMs, and BAs: articulate specific risks instead of vague statements, and justify release blocks constructively and professionally

What you'll do

• Lead the QA team: two manual testers and one automation engineer. Plan work together with the EM, review test design, test cases, and automated tests, conduct onboarding for newcomers, and help them grow
• Establish the QA process: at what step QA gets involved in a task, how test cases and regression are maintained, who accepts a feature and how
• Integrate AI agents into QA workflow: how the team writes and maintains automated tests with them, and how it validates what they have produced
• Define the platform testing strategy: what unit tests cover in services, what e2e covers, what is tested manually, and what checks are mandatory before merge and release
• Be responsible for automation: framework built on TypeScript and Playwright, API, streaming, event, and UI tests—and write them yourself
• Cover the most critical areas with tests: financial transactions (double charge, frozen balance reserve, duplicate event handling), integration with model providers (fallbacks between providers, limits, token accounting), agent workflows, and MCP servers
• Build evals for AI features so that degradation after prompt or model changes is visible in metrics
• Intentionally break the product: prompt injection (including indirect via files or web pages), system prompt leakage, safety filter bypasses, unauthorized access to user data
• Make quality measurable through metrics: production bugs, flaky test rates, run duration. Fix flaky tests rather than hiding them behind retries
• Sign off before releases: state what was tested and what risks remain
• Participate in requirements and API contract reviews before development: we follow a contract-first approach where API changes are agreed upon via a separate PR first
• Analyze incidents and add a regression test for each one
• Participate in QA hiring

Nice to have

• Experience with evals and LLM observability platforms: Arize Phoenix, Langfuse, promptfoo, DeepEval, Ragas, Braintrust, LangSmith
• Red teaming, guardrails, and AI safety testing
• Performance and load testing: k6 or Locust
• Resilience testing: chaos engineering, fault injection
• API Gateways (APISIX, Kong, Traefik) and OIDC providers (Zitadel, Keycloak, Auth0)
• RAG, search systems, and traditional ML: evaluating quality for search, classification, or ranking
• Developer tooling: utilities, plugins, CLI tools, dashboards
• React code-reading proficiency for frontend codebases
• Open-source contributions, public testing frameworks, or custom eval benchmarks
• Relevant degree: Computer Science or a related field

Terms and benefits

• A unique opportunity to lead and establish the launch of a new brand
• Work in a dynamic and innovative AI Tech environment
• Competitive salary package
• 25 vacation days, additional days off, and sick leave
• Flexible working arrangements: remote or hybrid in one of our offices
• Clear pathways for professional and career growth