Project 1 —
DevEco Vision @ .conf22
Identifying barriers and gaps for extending Splunk
Key Takeaway:
Conducting research around limited resources and limitations
Conducting research around limited resources and limitations
Project Intro
As part of the larger DevEco study, we established that there exist barriers for our customers to truly realize the value of Splunk and apply it to their unique business use cases. With the annual Splunk conference right around the corner, we thought it was a great opportunity to find out what these barriers were to our customer base, admins that used Splunk, who also happen to attend the event.
What is the current appetite admins have for building?
What is the admins’ builder journey?
Where are the biggest gaps and opportunities in this journey?
My Approach
1. Interactive “Builder” Journey Map Exercise
— What?
+ Created both digital and physical visualizations of the “builder” journey and devised questions to probe participants to talk about their experience at each stage.
+ Encouraged the participants to provide their feedback and responses to the questions with sticky notes.
— Why?
+ Encourage and facilitate eager participants at .conf22 willing to give feedback
+ Validate the “builder” journey we had created based on past research.
+ Discover any gaps or barriers within that journey
— What?
+ Created both digital and physical visualizations of the “builder” journey and devised questions to probe participants to talk about their experience at each stage.
+ Encouraged the participants to provide their feedback and responses to the questions with sticky notes.
— Why?
+ Encourage and facilitate eager participants at .conf22 willing to give feedback
+ Validate the “builder” journey we had created based on past research.
+ Discover any gaps or barriers within that journey
Note: Due to NDA, I cannot share the original picture of the Journey Map Exercise. Here is a similar picture of what it looked like at the conference.
2. Future of “Building” in Splunk Exercise
— What?
+ Created visualizations for the current vs future experience of building in Splunk using a simplified analogy by showing a comparison between the electronic part store and the Lego store.
+ These visualizations helped participants conceptualize what the future. experience could be and share their expectations of what they would want to see in the future.
— Why?
+ Explore ideas and concepts for the future experience of the “Builder” journey.
+ Test out the “Lego” concept as an inspiration for a better builder experience.
+ Gauge interest in some ideas and solutions to the “Builder” experience.
— What?
+ Created visualizations for the current vs future experience of building in Splunk using a simplified analogy by showing a comparison between the electronic part store and the Lego store.
+ These visualizations helped participants conceptualize what the future. experience could be and share their expectations of what they would want to see in the future.
— Why?
+ Explore ideas and concepts for the future experience of the “Builder” journey.
+ Test out the “Lego” concept as an inspiration for a better builder experience.
+ Gauge interest in some ideas and solutions to the “Builder” experience.
3. Exit Survey
— What?
+ Created a short survey that participants could take by scanning a QR code at the exercise at the conference booth.
— Why?
+ Gather some basic demographic data as well as user background in regard to building Splunk applications.
+ Could invite some of these participants based on their experience for more
in-depth interviews in the future.
— What?
+ Created a short survey that participants could take by scanning a QR code at the exercise at the conference booth.
— Why?
+ Gather some basic demographic data as well as user background in regard to building Splunk applications.
+ Could invite some of these participants based on their experience for more
in-depth interviews in the future.
.png)
What I learned
- It is important to create multiple ways to perform and gather data for research exercises as backups.
- Having open-ended questions is sometimes useful in eliciting feedback.
- Visuals are extremely helpful when conducting interviews or probing participants to share their thoughts as they act as a frame of reference that the participants build off of.
- When conducting research in a physical environment, it is important to take into consideration all the dependent variables that aren't just physical in nature but also the mindset that the environment brings about.
Project 2 —
Project Thanos
Consolidating Splunkbase admin tasks into one unified tool
Key Takeaway:
Learning how to manage and build and maintain stakeholder relationships as well as align interdisciplinary team goals between PM, Engineer, Designer, and Researcher.
Learning how to manage and build and maintain stakeholder relationships as well as align interdisciplinary team goals between PM, Engineer, Designer, and Researcher.
Project Intro
Currently, the Splunkbase admin experience is scattered across three different. This results in a workflow that is time-consuming and confusing for Splunkbase administrators to remember which activity is located where and what is visible only to admins and what is visible to developers. As a team of all interns (PM, Engineer, Designer, and Researcher), we took on this challenge to unify the Splunkbase admin workflow while also enhancing the developer experience.
What are the different tools admins use to perform their tasks?
What are the admins’ responsibilities and tasks?
What are the dependencies between current admin experience and dev experience?
My Approach
1. In-Depth Interviews
— What?
+ Reached out to various internal employees who interact with Splunkbase admin tools, such as admins, engineers, cloud vetting team, etc.
+ Conducted multiple in-depth interviews for each kind of user to get better understanding of their workflow and requirements of a unified admin tool.
+ Collaborated with the PM, Engineer, and Designer Interns to incorporate their queries and goals into the interviews.
— Why?
+ Extract admin duties, tasks, workflows, and pain points.
+ Understand the different admin tools and their unique use cases.
+ Gather the different needs and use cases of stakeholders.
— What?
+ Reached out to various internal employees who interact with Splunkbase admin tools, such as admins, engineers, cloud vetting team, etc.
+ Conducted multiple in-depth interviews for each kind of user to get better understanding of their workflow and requirements of a unified admin tool.
+ Collaborated with the PM, Engineer, and Designer Interns to incorporate their queries and goals into the interviews.
— Why?
+ Extract admin duties, tasks, workflows, and pain points.
+ Understand the different admin tools and their unique use cases.
+ Gather the different needs and use cases of stakeholders.
2. User Journey Workshop
— What?
+ Designed a user journey exercise that the admins could go through without moderation.
— Why?
+ Validate the user flow for the primary task of “Approving an app”.
+ Discover pain points and opportunities to improve user experience.
+ Gain deeper insights and findings by providing an interactive visual exercise that the admins can walk us through.
— What?
+ Designed a user journey exercise that the admins could go through without moderation.
— Why?
+ Validate the user flow for the primary task of “Approving an app”.
+ Discover pain points and opportunities to improve user experience.
+ Gain deeper insights and findings by providing an interactive visual exercise that the admins can walk us through.
.png)
3. Affinity Mapping with Design Intern
— What?
+ After conducting all the interviews and the user journey workshop, the designer and I had a jam session where we synthesized all the notes we had to create an affinity diagram.
— Why?
+ Determine priority of key elements and workflows of an admin and their tools.
+ Identify dependencies between different departments and users.
— What?
+ After conducting all the interviews and the user journey workshop, the designer and I had a jam session where we synthesized all the notes we had to create an affinity diagram.
— Why?
+ Determine priority of key elements and workflows of an admin and their tools.
+ Identify dependencies between different departments and users.
What I learned
- Forming stakeholder relationships and maintaining them throughout the project and even after is essential to ensure accurate requirements gathering and aligning goals.
- Working closely with designers, engineers, and PMs plays a vital role in aligning team goals and objectives while increasing awareness and visibility of every role's unique viewpoint and challenges.
- Being able to take a step back from the assumed requirements, to gauge the project scope allows for efficient work and meaningful solutions.
Project 3 —
DevEco Customer Mindsets
Redefining our users through the perspective of mindsets and lenses
Key Takeaway:
Conducting foundational research to direct and assist large-scale projects in the future.
Conducting foundational research to direct and assist large-scale projects in the future.
Project Intro
From the previous research study conducted internally as well as the study conducted at .conf22, we were able to identify the core segments of customer types. This research study was in hopes to gain deeper and qualitative insights into these customer segments.
What are the different types of users?
What are their key barriers to solving problems?
What are the different types of customer mindsets?
My Approach
1. Understanding the concept of mindsets
— What?
+ Explored the idea of defining our personas through the concept of mindsets and lenses.
+ Conducted lit review of past work internally, studies being done in other companies, and other external articles and journals.
— Why?
+ We believed that the current way of using personas to define our customers is shallow.
+ To develop a foundation-level resource for any future products within the DevEco system.
— What?
+ Explored the idea of defining our personas through the concept of mindsets and lenses.
+ Conducted lit review of past work internally, studies being done in other companies, and other external articles and journals.
— Why?
+ We believed that the current way of using personas to define our customers is shallow.
+ To develop a foundation-level resource for any future products within the DevEco system.
2. In-Depth interviews
— What?
+ Introduced the Jobs-To-Be-Done Framework as a better way to structure and inform the direction of the interviews.
+ Assisted my mentor who lead the interviews of various different Splunk customers who each had different backgrounds and experiences.
+ Took notes during the session and synthesized this data into valuable insights.
— Why?
+ Discover patterns between customer roles within companies and their mindsets towards Splunk.
+ Identify what the different customer mindsets care/don’t care about and if there are any gaps in what we offer to satisfy those needs.
— What?
+ Introduced the Jobs-To-Be-Done Framework as a better way to structure and inform the direction of the interviews.
+ Assisted my mentor who lead the interviews of various different Splunk customers who each had different backgrounds and experiences.
+ Took notes during the session and synthesized this data into valuable insights.
— Why?
+ Discover patterns between customer roles within companies and their mindsets towards Splunk.
+ Identify what the different customer mindsets care/don’t care about and if there are any gaps in what we offer to satisfy those needs.
What I learned
- How to approach foundational research, especially for something that will be used as a reference for any future product.
- Exploring and combining different frameworks and methodologies.
- Introducing visual and interactive exercises can help structure interviews more effectively. Especially when talking about broad topics.
Project 4 —
SCDE Usability Study
End-to-end usability study before the public launch of product
Key Takeaway:
Conducting a usability study in 2 weeks from recruitment to research readout of a highly technical product while utilizing and building on top of past studies.
Conducting a usability study in 2 weeks from recruitment to research readout of a highly technical product while utilizing and building on top of past studies.
Project Intro
As a final during my internship, I wanted to lead an end-to-end usability study to balance out all the foundational and generative research I had conducted during the internship. Luckily for me, the Splunk Cloud developer Edition was about to release to the public. After convincing the PMs of the product, I took on the challenge of conducting a final usability study, before the launch of the product in just 2 weeks.
Have we addressed usability issues uncovered in previous studies?
What are key opportunities for future development?
Is the product facilitating functionality that users are expecting?
My Approach
1. Moderated Usability Testing
— What?
+ Crafted Screener and Recruitment emails and utilized userinterviews.com to handle recruitment, scheduling, and incentivizing participants.
+ Worked closely with the principal designer, PM, and engineers on the product to uncover their requirements and points of concern to design the tasks and questions for the session.
— Why?
+ Uncover current usability issues when participants are walking through the prototype to complete tasks.
+ Probe participants to talk out loud and express their opinions on the functionality and experience being presented.
— What?
+ Crafted Screener and Recruitment emails and utilized userinterviews.com to handle recruitment, scheduling, and incentivizing participants.
+ Worked closely with the principal designer, PM, and engineers on the product to uncover their requirements and points of concern to design the tasks and questions for the session.
— Why?
+ Uncover current usability issues when participants are walking through the prototype to complete tasks.
+ Probe participants to talk out loud and express their opinions on the functionality and experience being presented.
2. Affinity Mapping
— What?
+ Gathered all the notes from the suability sessions and synthesized the data into an affinity map by grouping issues by type.
— Why?
+ An efficient way to hand off the data I had gathered to the designer, who could look through the affinity map along with the research readout, to take action on these insights
— What?
+ Gathered all the notes from the suability sessions and synthesized the data into an affinity map by grouping issues by type.
— Why?
+ An efficient way to hand off the data I had gathered to the designer, who could look through the affinity map along with the research readout, to take action on these insights
What I learned
- Conducting multiple usability sessions within the same day is challenging and having sufficient downtime between sessions is crucial.
- Involving PMs and designers when crafting the research plan, interview scripts, usability sessions, etc throughout the process bolsters stakeholder relationships and increases buy-in.
- It is important to take the initiative as a researcher, to sit in developer meetings, weekly standups, design update meetings, etc.
Takeaways
Learned what it was to be a researcher in the industry
This was my first time being exposed to the practices of research in the real world. It is vastly different from the structured and forgiving research processes in class projects in school. It is more important to be able to jump between projects, manage relationships with stakeholders, consider the limitations of developers and align with business goals
This was my first time being exposed to the practices of research in the real world. It is vastly different from the structured and forgiving research processes in class projects in school. It is more important to be able to jump between projects, manage relationships with stakeholders, consider the limitations of developers and align with business goals
Learn from everyone around you
During this internship, every single person I interacted with left me with invaluable advice and knowledge. Every person is an expert at something and has a vast amount of experience in various different fields and backgrounds. I hope to learn from everyone I meet.
During this internship, every single person I interacted with left me with invaluable advice and knowledge. Every person is an expert at something and has a vast amount of experience in various different fields and backgrounds. I hope to learn from everyone I meet.
Communication is fundamental in a cross-functional team
When working in teams of PMs, designers, engineers, etc, it is really to have constant communication and discussions for everyone to be aware of each other's progress, challenges, goals, and expectations. This helps align these different aspects to the overall goals and objectives of the product.
When working in teams of PMs, designers, engineers, etc, it is really to have constant communication and discussions for everyone to be aware of each other's progress, challenges, goals, and expectations. This helps align these different aspects to the overall goals and objectives of the product.
Defining project scope and expectations is crucial
Sometimes I tend to get carried away when coming up with solutions and recommendations to problems. It is really important to recognize what the scope of the project is and what amount of resources and effort is reasonable. By doing so, we can find that happy balance of innovation to the least resource-dependent solution.
Sometimes I tend to get carried away when coming up with solutions and recommendations to problems. It is really important to recognize what the scope of the project is and what amount of resources and effort is reasonable. By doing so, we can find that happy balance of innovation to the least resource-dependent solution.
Learning how to jump from one project to another as a researcher managing time and priority between projects
Most of my experience comes from working on a single project end to end, from conceptualization to final design. However, as a researcher, I would need to be able to juggle multiple projects at the same time. It is a luxury to be included in a project early as a researcher, I need to be able to assimilate and adapt to whatever lifetime of a project I am brought into.
Most of my experience comes from working on a single project end to end, from conceptualization to final design. However, as a researcher, I would need to be able to juggle multiple projects at the same time. It is a luxury to be included in a project early as a researcher, I need to be able to assimilate and adapt to whatever lifetime of a project I am brought into.
Tons of practice writing research plans and other documentation
It is extremely crucial to maintain proper research plans and documentation for anyone referring back to research studies in the future. Fortunately for me, I had the opportunity to create and maintain documentation for all the projects I worked on. So I was able to experience firsthand what the process is like and practice how to produce effective and efficient documentation.
It is extremely crucial to maintain proper research plans and documentation for anyone referring back to research studies in the future. Fortunately for me, I had the opportunity to create and maintain documentation for all the projects I worked on. So I was able to experience firsthand what the process is like and practice how to produce effective and efficient documentation.
