
Moderated user interview
Interview
Task
- Help a neighboring product team conduct interviews for "Personal Account AI Gateway", a web app for internal AI agent developers
- The service was meant to track agent implementation status, provide training materials, and enable work with DevOps
- The product's key goal was to reduce time to market and the number of support tickets
What was done
- Defined the research goal — to identify the current pain points teams face during the AI agent rollout process
- Prepared a research guide: expected outcome, decisions the team could make based on the results, research questions, hypotheses to test, research method, respondent requirements, and the list of interview questions
- Aligned the guide with the team and got contacts for suitable respondents
- Conducted the research and, based on the insights gathered, formulated conclusions and proposals for improving problem areas
Methodology and respondents
- Method: in-depth interviews, 40–60 minutes, remote, with video and audio recording
- Respondent criteria: priority given to those who had already gone through the rollout process, or those preparing to do so and familiar with the documentation
- Conducted 8 interviews with POs and analysts responsible for the AI agent rollout process
Hypotheses
- Developers don't fully understand the structure and requirements described in the existing documentation
- When facing difficulties, developers turn to colleagues for help more often than to the documentation
- Developers don't understand what stage their agent is at (submitted, reviewed, approved, etc.)
- The lack of a unified interface makes it harder to track progress and fragments the rollout process
Result
- Conducted 8 interviews with team members (POs, analysts) responsible for the rollout process
- Confirmed 3 out of 4 hypotheses
- Identified key pain points in the rollout process and interface wishes
- Proposed solutions to the identified problems
- Presented the research results to the team