QA Automation Engineer / SDET

About

We're looking for a QA engineer with a strong engineering foundation. You'll own platform automation end to end: framework, contracts, CI, load, and evals. The core challenge is making quality measurable where the same input produces a different answer every time. The bulk of the work is testing APIs, event-driven pipelines, and LLMs. Alongside that are classifiers, embeddings, and models that decide where to route a request. Automation takes up most of your time, but we also test manually on a regular basis: exploratory testing, test design, accepting new features before they've stabilized, and incident analysis.

Your superpowers

• 3+ years building automation and living with it on a production product afterward
• Strong TypeScript; Python is fine at a read-and-edit level — our services are written in it, and that's where unit and contract tests live, using pytest
• Playwright, both API and browser. You understand the cost of a UI test and don't push up what can be checked at the API level
• You write clean, testable, maintainable code: a framework that gets extended six months from now, not thrown away
• REST and SSE streaming. You know how to test a stream of chunks, a dropped connection, request cancellation, a partial response, and a timeout
• Contracts: OpenAPI, generating types from a schema, backward compatibility, what counts as a breaking change
• Confident working with LLM APIs and know how they break: Chat Completions and Messages / Responses API, tokens and the context window, tool calling, structured outputs, prompt caching, retries, provider limits and errors
• Worked with agentic systems on a project or explored them on your own: agent loop, context and memory, tool calls. You understand that you need to check the path to the answer, not just the answer itself
• Evals: you've built them yourself or are ready to figure it out. You understand why binary pass/fail lies on a generative feature
• CI/CD: GitHub Actions or similar. Tests live in the pipeline, a run doesn't balloon to half an hour, and the report makes it immediately clear what failed
• You know SQL and PostgreSQL, have worked with Redis. When a response doesn't make it clear what happened, you go check the state in the database
• Docker is a must, Kubernetes is a big plus: find a pod, pull logs, get to the root cause
• You understand event-driven systems (Kafka, RabbitMQ, or similar): what happens if the same message arrives twice, and how to test for it
• Observability as a testing tool: you can find the cause of a failure in OpenTelemetry traces and in metrics
• SWE agents are part of your daily work (Claude Code, Codex, Cursor, your own harness): you write tests with them, generate data, offload routine work, and always verify what they produce
• You're curious about new things and quickly turn that curiosity into practice. You know what's changed with model providers and in test tooling over the past few months
• You're about ownership: you find risks yourself, clear blockers, and carry a bug from report to fix to a regression test. And you keep a product focus: a test for double-charging is worth more than a test for a button's color
• You communicate clearly: a bug report that reproduces on the first try, and a risk assessment instead of "something seems broken." Calm, to the point, and respectful of people

What you'll do

• Own platform test automation: API, streaming, event-driven pipelines, UI. Tests in TypeScript and Playwright, with API types generated from OpenAPI
• Test work with model providers: switching and fallbacks, retries, limits and quotas, token and cost accounting, behavior under provider degradation
• Test agentic scenarios: agent loop, tool calls, MCP servers, multi-step conversations, reproducibility of runs
• Keep money under test: double-charging, a stuck balance reservation, an event processed twice
• Build evals for AI features: a set of reference examples, automated checks, a CI run after every prompt or model change. So that "it got worse" shows up as a number
• Break things on purpose: prompt injections, including indirect ones via a file or web page. System prompt leaks, bypassing safety filters, malformed tool calls, access to other users' data
• Fix flaky tests: find the cause instead of wrapping them in a retry
• Run load tests before releases and work with the backend team to find bottlenecks
• Review requirements and contracts before code: we're contract-first, an API change goes into a PR before the implementation, and your voice matters there
• Analyze incidents and turn each one into a test
• Work hands-on where that's faster than an automated test: exploratory testing, test design, accepting new features
• Influence the process: which checks block a merge, what counts as done, where agents speed up the work

Nice to have

• Experience with eval and LLM observability platforms: Arize Phoenix, Langfuse, promptfoo, DeepEval, Ragas, Braintrust, LangSmith
• Depth in agentic systems: MCP, analyzing agent trajectories, debugging multi-step runs
• Red teaming, guardrails, AI security testing
• Classical ML: how to measure the quality of a classifier or a ranking
• RAG and search: retrieval metrics, hybrid search
• API gateways (APISIX, Kong, Traefik) and OIDC providers (Zitadel, Keycloak, Auth0)
• Load and stress testing: k6 or Locust
• Chaos and fault injection, resilience testing
• Experience building dev tools: utilities, plugins, CLIs, internal dashboards that make engineers' work easier
• React and enough frontend experience to read client code
• Personal projects, open-source contributions, public eval suites or test frameworks

Terms and benefits

• Work in a dynamic and innovative AI Tech environment
• Competitive salary package
• 25 days of vacation, additional days off, and sick leave
• Flexible work arrangements: remote or hybrid from our office in Prague
• Transparent paths for professional and career growth