Product Quality Analyst - 20-25M/month - Fast Growing Startup

We are building a next-generation AI onboarding agent for software companies.

We are building a complex AI-first product with many interaction paths, integrations, and real-world usage patterns. As the product evolves quickly, we need dedicated, continuous manual testing to catch bugs, regressions, and edge cases before users do.

In addition, many important product issues are not strict bugs — they are moments where the experience feels confusing, unnatural, or misaligned with how real users behave in their tools and workflows.

This role exists to systematically test the product end-to-end, try to break it, surface experiential issues, and identify capability gaps where the product cannot adequately support real-world user behaviour.

1. Manual & exploratory QA testing

Continuously and creatively test the product by actively imagining and simulating scenarios that could break it or degrade the user experience.

This includes testing across:

• Core user flows
• Edge cases and failure scenario
• Long-running or unusual usage patterns (e.g. extended sessions, repeated interactions)
• Different configurations, tools, environments, and devices (e.g. desktop vs mobile)

You are expected to:

• Think adversarially about how real users might use the product in unexpected ways
• Ask “what happens if…” questions and test those scenarios in practice
• Push the product beyond happy paths to uncover hidden weaknesses
• Actively try to break assumptions, not just verify expected behaviour
• Test both before and after releases to identify unintended side effects and behavioural drift

Examples include (but are not limited to):

• Staying in a live session for extended periods
• Asking the AI to handle complex, ambiguous, or high-stakes tasks (e.g. data migration, table or cell-level reasoning)
• Using the product in noisy, degraded, or imperfect conditions
• Switching devices or contexts mid-flow

2. Regression & release testing

• Validate new features and agent capabilities before they go live
• Re-test existing functionality after changes to ensure nothing degrades
• Verify that previously fixed issues do not reappear in new releases

3. Clear issue reporting

Document issues in a way that makes them easy to understand and act on, including:

• Clear reproduction steps
• Expected vs actual behaviour
• Supporting evidence such as screenshots, videos, logs, or audio recordings where relevant

Categorise issues clearly, for example:

• Bug
• Regression
• UX / product issue
• AI behaviour issue

Work closely with engineering and product to ensure issues are well-understood, properly scoped, and actionable.

Additional responsibility: product & AI experience quality

Alongside traditional QA work, this role helps maintain a high bar for product clarity and human-like AI behaviour.

This includes:

• Flagging UX issues that are confusing, unclear, or unintuitive (even if technically working)
• Identifying error messages or states that are overly technical or unhelpful
• Testing AI behaviour for:
• Unnatural phrasing or tone
• Awkward timing or interruptions
• Over- or under-explanation
• Calling out moments where “this technically works, but a real human wouldn’t do this”
• This role does not decide final wording or behaviour, but provides high-signal input to product and engineering.

Additional responsibility: capability gap discovery

While testing the product across different tools, integrations, and real-world setups, you will:

• Observe how users in different tools typically:
• Onboard
• Configure products
• Learn features (e.g. guided tours, staged onboarding, progressive setup)
• Identify cases where our product cannot adequately support these real-world workflows
• Document capability gaps, focusing on:
• The user’s real behaviour and expectations
• Where the product falls short
• The impact on onboarding success or usability

Capability gaps are:

• Logged separately from bugs
• Summarised periodically (e.g. weekly)
• Reviewed with product leadership

What this role does not do

• Write production code
• Design UX or UI
• Tune prompts or AI models
• Own release gates or quality standards (owned by the Product Platform Engineer)
• Decide roadmap priorities
• This role does not propose solutions, design features, or set priorities — it surfaces high-quality input to inform product decisions.

Must-have

• Proactive, highly motivated working style
• Ownership mindset - think of yourself as owning product quality
• Strong attention to detail and patience
• Strong product and UX taste
• Curiosity and a desire to break things in realistic ways
• Ability to test complex products independently
• Clear written and verbal communication
• Ability to explain why something feels broken, confusing, or insufficient

Nice-to-have

• QA or product testing experience
• Experience with SaaS or AI-powered products
• Technical literacy (but not required to code)

We offer competitive salary, high autonomy and try to accommodate everything you need to succeed in your role. In return we expect dedication, effort, long hours and passion. We’re on the mission of our lifetimes and every person on the team is a crucial part of our success.

If you’re looking for an easy 9-5 job, we are not for you. If you want to build a generation-defining company and a product that will be used by millions of people every day then please apply.

Our Operating Principles:

• Always Be Hustlin’ – Move fast, outwork the competition, stay scrappy.
• Relentlessly Curious - Reason from first principles, question assumptions and explore better ways.
• Super Pumped – Show relentless enthusiasm and drive.
• Make Magic – Create experiences that delight customers.
• Obsess Over the Details – Perfect the product experience, no matter how small.
• Big Bold Bets – Think big, take risks and go after huge, transformative opportunities.
• Ownership is Key - Own the outcome, not just the task.