User Evaluation is a transformative AI-powered tool that is revolutionizing the field of customer experience research. By integrating AI into the user interaction and data analysis process, UE streamlines the research writing, generates precise interview summaries, generates charts, and presentations, and conducts trend analysis. The platform empowers users ranging from researchers, PMs, and designers to founders, unlocking faster, more insightful customer research. As a Senior Product Designer, I’m at the forefront of these exciting changes.
Role
Senior Product Designer
UI Design
UX Research
Product Strategy
Year
2022 – Present
Tools & Methods
GPT-4, LLMs,
User Research
Figma, Miro
Wireframes, Journey Maps
Prototypes
Interfaces
Usability Testing
Statistics
- 500+ Top Tier Companies
- 120k+ Insights Generated
- Top Product on Product Hunt
Problem
Traditional customer and UX research was labour-intensive and resource-demanding. Before modern solutions, businesses and researchers grappled with:
- Resource Drain: Significant amounts of time and money were required to meticulously go through the gathered data, seeking meaningful insights.
- Manual Efforts: A large portion of a researcher’s time was consumed in manually reviewing interview recordings, which often slowed down the research process considerably.
- Risk of Oversights: The manual design of charts and visual representations, combined with the vastness of data, made the process prone to human error and potential oversights.
The Solution
User Evaluation emerged as a revolutionary solution that utilizes the power of AI to transform the cumbersome process of customer and UX research. Before diving into the specifics, it’s worth noting the essence of our solution:
- AI-Powered Efficiency: The integration of AI not only expedited research writing but also automated tasks such as generating interview summaries, charts, and presentations.
- Deep Trend Analysis: Our tool went beyond traditional means to conduct an in-depth trend analysis, offering a more comprehensive view of user behaviour and insights.
- Unmatched Productivity: By significantly cutting down manual efforts, researchers, designers, PMs, and founders could delve into their data and achieve profound insights with efficiency that was previously unthinkable.
My Role
I lead the design as Senior Product Designer of our Large Language Model (LLM) interfaces, creating a harmonious blend of technology and user-centric design. My work also involves extensive qualitative and quantitative research and collaboration with data scientists and prompt engineers to ensure the optimal use of AI in delivering insightful user experiences. My tasks range from designing end-to-end intuitive interfaces for AI Chat to making complex data analysis visually accessible, ensuring users can efficiently unlock valuable insights from their research data.
My Process
Utilizing the Double Diamond model as the backbone of my design methodology ensures a holistic and user-centric approach to problem-solving.
01. Discover
Interviews, Research & Competitor Analysis
02. Define
Problem Statement, User Stories, Feature Mapping
03.
Ideate
LOFI Wireframe, User Testing & Present to stakeholders
04. Implement
Hi-Fi Design & Development
Decoding the Complexity of AI Design
Challenges
Learning to work with OpenAI’s GPT API was a significant part of my journey at User Evaluation. As these AI tools were still fresh, I had to learn quickly and adapt, experimenting as I went along. It was important to keep the user at the center of the design process, even as we explored new technology.
Our users were also new to these AI-powered apps. Designing for an audience who were still finding their way around the technology was another challenge. At the same time, working with Prompt Engineers called for a deeper understanding of Generative LLM models. This meant striking a balance between advanced tech and intuitive design.
Approach
In the face of these challenges, I took several steps to adapt and grow. I delved into OpenAI’s API and experimented with LLM Models, including GPT-4, and Dall-E, to understand their capabilities. By exploring various AI apps in the market like MidJourney, I expanded my design thinking and knowledge about the potential of AI.
I also conducted interviews with users, gathering their wishes, pain points, and comfort levels. These insights, along with data analysis from app usage and heatmaps, guided our design improvements. I boosted my technical knowledge by undertaking online courses on LLM and GPT technology. My coding background also came to my aid, enabling me to experiment directly with the API and understand prompts better.
User Research
During the user research phase of User Evaluation, I discovered a critical pattern: users were struggling to master the art of prompting. As with any new technology, the open-ended nature of AI can be overwhelming, often leaving users uncertain about how to engage effectively. I noticed that users would either over-prompt or under-prompt, inhibiting the AI’s capability to generate meaningful insights.
I facilitated interviews and group discussions, engaging with over 50+ customers and UX researchers via platforms like Zoom. Participants actively tested the app, sharing invaluable feedback. This hands-on experience was followed by a structured Q&A, ensuring a comprehensive understanding of their insights.
Response
In response to this finding, I decided to implement AI-assisted prompting. Users were delighted with this addition; the AI was able to provide them with a curated list of options, making their interactions feel more guided and less chaotic. This intervention not only simplified their user experience but also allowed them to better use the power of AI. Additionally, through my user interviews, I found that users highly valued simplicity.
Distribution of Feature Preferences
Some questions I asked
- How often do you interact with AI-driven applications or platforms?
- Can you walk me through how you would typically prompt the AI in this interface?
- Did you feel like you knew what to ask or prompt the AI with when you first started using the system?
- Were there instances where you felt the AI did not understand your prompt? Can you provide specific examples?
- On a scale of 1-10, how intuitive did you find the prompting process with the AI?
- How would you feel if the AI suggested or guided your prompts? Would that be helpful?
- Were there moments when you felt overwhelmed or confused by the possibilities of what to ask the AI?
- What other features would you like to see?
Design Transformation
Move the slider left or right
User Journey Map
Trace the routes users take, understanding their preferences, pain points, and moments of delight. A comprehensive look into their experience.
Hand Drawn Concepts
Before pixels, there are sketches. Dive into my preliminary sketches that lay the foundation for the UI designs.
Wireframes
Experience the starting point – wireframes that guarantee a clear, user-centric pathway to the final product.
Key Workflow
The Design Pattern section provides a glimpse into the journey I took to create an intuitive user experience for User Evaluation. It showcases how I designed logical paths from initial file upload to an insightful AI-based data examination. The emphasis here is on simplicity and user-centric design, an attempt to make complex interactions feel straightforward.
AI Chat Pattern
Here you’ll see my approach to simplifying complex AI interactions. I’ve designed a conversational interface that facilitates user interaction with advanced AI models, focusing on a clear, engaging, and easily navigable chat experience. The patterns here demonstrate how user comfort and comprehension are prioritized, making AI approachable for everyone.
Figma Prototypes
These underwent testing with researchers, stakeholders, and potential customers. From this testing, we identified specific user needs, such as the demand for prompt suggestions in the chat area.
Accessible UX
Creating Universally Accessible Experiences
Accessibility isn’t an afterthought in my designs, but a prerequisite. I always consider how to make my work accessible and enjoyable for everyone.
