Mirror: A Learning Reflection Platform
Empowering education nonprofits to center student perspectives through holistic feedback
Mirror is the culmination of my capstone team’s six-month community-engaged research & co-design partnership with local youth art education nonprofit Urban ArtWorks.
As a desktop & mobile app that streamlines the process of collecting recurring, relevant feedback from students and teachers, Mirror ensures that education nonprofits have data-backed insights to center student perspectives in program improvements & grant reporting.
my role: Service Designer
- Leveraged co-design workshop findings to propose Mirror’s core functionality
- Created service blueprint and storyboard to communicate our idea
- Kimmie Aralar, designer
- Claire Yi, PM
- 2 member checks
- 1 co-design workshop
- 2 field observation sessions
- Service blueprinting
- Service blueprint
- Video prototype
- High-fidelity prototype
- Design documentation
timeline: summer 2022 (10 weeks)
This case study covers our co-design process; read more about our research with Urban ArtWorks for details on how we arrived at our design opportunity.
This project focused on supporting education nonprofits in collecting, organizing, and analyzing the holistic feedback required for student-centered program improvements and grant reporting.
Our collaboration with Urban ArtWorks supplied us with a nuanced understanding of education nonprofits’ needs & goals for student reflection data.
Urban ArtWorks is a Seattle-based youth arts education nonprofit that focuses on public murals. From our research patnership, we learned that staff face difficulties getting meaningful evaluation data due to decentralized systems and lack of youth engagement.
For this co-design phase, we conducted a series of workshops and observations with Urban ArtWorks, which allowed us a narrow but deeply situated understanding of their organization’s approach to and needs for using reflection data.
From here, I leveraged our secondary research to identify generalizable challenges and opportunities.
Three mural apprentices standing in front of their work, courtesy of Urban ArtWorks
Meaningful reflection data is crucial for student-centered program improvements and grant funding, which enable education nonprofits to thrive.
Meaningful feedback from students & teachers constitutes key data for education nonprofits to evaluate their programming, and also for staffers to create a compelling narrative around their programs’ impact for grant applications and reports. This grant funding is what has kept many education nonprofits afloat over the past few years, despite major decreases in revenue from program fees due to COVID19.
Current feedback systems which rely on a single summative evaluation show only a pale reflection of programs’ impact on students.
The standard model for most organizations is to give out a summative, end-of-course evaluation survey. In our research, we heard that this model of a single checkpoint failed to engage students and generated shallow responses that didn’t truly reflect the growth that teachers had witnessed.
“The data that we get now is not... It's not bad, but it's not the best [laughs] [...] students don't really take a lot of time to think about the questions that we're asking.”
- Eboni, Program Manager at Urban ArtWorks
Mirror ensures education nonprofits have access to a wealth of holistic feedback data by shifting the model from one summative evaluation to recurring, relevant reflections.
With Mirror, education nonprofits can transform their feedback model from a single end-of-course evaluation, to integrating recurring, content-relevant reflections into their curriculum. This also includes reflections from teachers, who give staffers insight into how a class is doing as a whole. Through central data management and automated report generation, Mirror allows staffers to easily craft rich insights about their organization’s impact.
When it comes time to craft the narrative of an organization’s impact, Mirror ensures that staffers don’t just have data - they have a new way to see their students’ perspectives.
Students stay engaged through reflection questions that are relevant to the material they’re learning.
Mirror allows education nonprofit staff to integrate recurring reflection surveys directly into their curriculum, ensuring that they are relevant to what students are learning. This way, they can assessing whether students’ experiences align with program goals without having to rely on a single post-program evaluation.
Teachers’ freeform reflections provide context to individual student and class growth.
Through Mirror, teachers are also prompted to regularly create text or video entries reflecting on how recent classes & activities went. These reflections contextualize how a class is doing as a whole and highlight important moments of which staff should take note.
Staff can easily pull together the data they need to create compelling grant report narratives & drive program improvements.
With an abundance of meaningful reflection data at staffers’ fingertips, Mirror’s organization system ensures that nobody’s story gets lost in the pages. Mirror empowers staff to find relevant quotes, videos, and images across their organization’s programs by 1) tagging student responses based on a survey’s designated focus area and 2) automatically transcribing teachers’ video reflections.
Our co-design process invovled identifying a promising design direction, aligning with Urban ArtWorks, facilitating collaborative ideation, then generalizing our learnings to construct Mirror’s conceptual underpinnings.
My capstone team’s process followed a route different from our peers’ by virtue of pursuing a co-design partnership. I believe co-design is an appropriate approach in community-oriented situations like this one, where it ensures a sense of mutual accountability & investment and helps to avoid exploitative research practices. Co-design also creates a more balanced power dynamic between researchers & collaborators, whose involvement makes them more than just participants and whose benefit far exceeds a gratuity payment.
Guided by the insights I had shaped during our research phase, we brainstormed ideas to determine that the focus area of evaluation data would allow us to contribute the most.
In order to generate a jumping-off point for our co-design process with Urban ArtWorks, my teammates and I each came up with 20 ideas for improved technologically-backed systems. We worked within potential opportunity areas idenfitied by our research: evaluations, tooling, streamlining, and formalization.
My ideas prioritized simplicity over boundary-pushing technology to ensure feasibility for our partner organization in the short- to medium-term.
After grouping similar concepts, we noted that our ideas for evaluation systems often also impacted the other axes, leading us to conclude that our work could have the most impact there.
We grouped ideas with similar goals, then extracted themes to describe each category
initial design direction
Our ideation highlighted a key opportunity of shifting the feedback model from a single evaluation to recurring reflections.
These ideas revealed the possibility of a fundamental shift in perspective on student feedback data: from a single evaluation that exists outside of a programs’ main activities, to recurring reflections which are deeply integrated and relevant to what students are learning at a given time.
By conducting two feedback sessions with Urban ArtWorks staff, we ensured that our proposal to focus on recurring reflections was indeed situated with their experiences and goals.
Before moving forward with our design process, my team engaged in a member checking process to investigate our interpretive validity; in other words, in order to confirm how closely our research insights reflected staff members’ experiences and our proposal aligned with their goals, we gave a presentation to staffers which non-judgementally laid out our understanding of challenges and possibilities.
For this feedback discussion with Urban ArtWorks staff, I created a simplified service blueprint intended to reflect our understanding of current systems and facilitate conversations about potential changes.
Staff members were excited about this update’s potential to increase investment from students & teaching artists in the feedback process.
The service blueprint I created depicted existing, updated, and new reflection activities across a standard 8-week program, where weeks 3-7 follow the pattern established in week 2
After hearing staff enthusiasm about shifting to recurring reflections, we conducted observations to experience current feedback systems firsthand.
Once staff members were onboard, my team set out to deepen our understanding of the status quo by conducting observations. We sat in on the first day of their Mural Apprentice Program in order to see how staff and teaching artists conducted their existing pre-program evaluation, and again a couple weeks into the program to observe how they ran weekly check-ins. This also allowed us the opportunity to speak with their lead teaching artist, who emphasized how crucial it was that reflection activities do not break the flow of class time.
An Urban ArtWorks teaching artist leads students in a beginning of class check-in activity
Grounded in our new understanding of the status quo, we facilitated a co-design workshop to ensure staffers all had a space for their ideas to be heard-- by us and each other.
After deepening our understanding of existing systems through firsthand observations, we led a co-design workshop with four staff members to surface their ideas, priorities, and constraints in the realm of recurring student reflections. We worked together to ideate upon potential reflection activities, as well as the specifics of how staffers envisioned them actually being conducted and integrated into their programming. We also structured our activity to make space for everyone’s perspective, from the Executive Director to a new employee who had started only a few weeks beforehand.
Each participant brainstormed a set of ideas for reflection activities
Our co-design workshop taught us that staffers saw teachers as playing a crucial role in the feedback process, beyond just facilitating reflection activities.
From both our observations and our co-design workshop, we learned that by focusing only on staff and students, we had underestimated the crucial role of teaching artists in generating holistic, meaningful feedback. Teachers didn’t just need the support to facilitate recurring reflection activities, they also needed agency to shape those activities and structure to provide their own insights that contextualize student responses.
Going forward, we ensured that our designs treated staff, teachers, and students as equally important stakeholders.
urban artworks outcomes
We concluded our partnership by ensuring staffers were equipped with a shared understanding of the opportunity for holistic feedback, a longer-term direction for recurring reflections, and an appropriate plan for short-term changes.
We ended our collaboration with Urban ArtWorks by handing over a one-page executive summary of our research, a simplified service blueprint, and a set of recommendations for streamlining their tooling systems that they are working to implement.
One of my goals for our collaboration from the very beginning was for the impact of our collaboration to extend beyond my team’s direct contributions in scope and timescale. I was thrilled that in the final weeks, Urban ArtWorks staffers began planning for their own brainstorming sessions which would bring together teachers and former students. By talking with us and each other about the challenges they had been running into and the ideas they had, staffers created a strong sense of internal alignment and motivation to make ongoing changes to their feedback systems.
After realizing that what we had learned from our co-design partnership could extend to fill a need far beyond Urban ArtWorks’ immediate goals and priorities, I generalized our insights to propose Mirror’s key features.
From our secondary research, we knew that the importance of getting holistic feedback data extended to education nonprofits broadly: most rely heavily on grant funding, for which data about student experiences is crucial. We knew also, both from our research and from our own lived experience as students, that the mechanisms to collect this data were inadequate to get students invested enough in the process to consistently provide meaningful feedback. Thinking about this common difficulty that education nonprofits face is what inspired us to create Mirror.
As my team’s lead researcher and service designer, I transformed our insights on education nonprofits’ goals and needs for reflection data into a key set of affordances that would become central to Mirror:
- Integrate recurring reflections into a program’s curriculum by connecting with common existing calendar applications. In this way, staff and teachers can identify appropriate student reflection checkpoints.
- Create student reflection surveys tied to program objectives, which teachers then have the agency to weave into their classes. These student responses comprise the bulk of the data that staffers needs for grant reporting & program improvements.
- Teachers can generate freeform video or text entries, reflecting upon how specific activities went as well as providing context to individual student and class growth.
- Staffers can make sense of reflection data via Mirror’s automated reporting capabilities, including tagging student responses based on selected survey objectives and automatically transcribing teachers’ video entries.
By streamlining the process of collecting & parsing meaningful, recurring feedback, Mirror ensures that staff and teachers have the insights they need to make data-driven program improvements and represent their organization’s impact in grant reporting.
telling Mirror's story
I led my team’s storyboarding & video prototyping to show how Mirror can transform education nonprofit’s feedback systems, giving them the necessary data for student-centered grant reporting & program improvements.
While my teammates iterated towards high-fidelity wireframes, I took charge of how we could convey the value of our designs to people who were not so deeply embedded into an education nonprofit’s needs and operations.
I developed multiple iterations of a storyboard to convey how Mirror could be used by a staff member to craft a narrative of their programming's impact on students. I overlaid sketches on top of photographs taken in Urban ArtWorks' studio and around Seattle in order to help viewers situate with an education nonprofit's studio environment.
My storyboard started with a few simple frames, and grew into the detailed version shown above. I then converted this refined storyboard into a shotlist, which served as the basis for our video prototype.
With more time, I would conduct a broader round of research with students and other education nonprofits.
By conducting concept evaluations with students, I would ensure that all stakeholders’ voices are heard.
Because of IRB restrictions, our team wasn’t allowed to interact directly with minors in a research capacity. This left us mainly relying on staff and teachers’ close relationships with students to understand their needs and goals for reflections. Although we did conduct observations to see how students interacted with each other and teachers, I acknowledge that Mirror was designed primarily around staff’s perspective with some input from teachers.
With more time, I would prioritize surfacing students’ perspectives on questions like what kind of reflection exercises could serve them in conceptualizing their growth throughout a program, and how to ensure that they understood how valuable their feedback was for the direction of the organization.
I would also validate my assumptions about generalizing by doing concept evaluations with education nonprofits that have a range of workflows and organizational structures.
While we were able to run one concept evaluation of Mirror’s essential service with a representative of a different education nonprofit, we have yet to conduct broader UX research to ensure Mirror has the appropriate customization options to suit a variety of organizations. This middle stage (preceding usability testing) might include additional concept evaluations as well as surveys to gain a broader perspective on different organizations’ needs and systems.
This project taught me that although co-design is not appropriate for every situation and can be tricky to do well, the depth of understanding and mutual accountability it generates is incomparable.
Doing a co-design partnership with such a vibrant organization was an incredible growth opportunity for me as a researcher and service designer.
On the one hand, my team faced challenges with recruiting, syncing timelines, and aligning on goals that we would not have encountered using other research & design methods. Conversely, this partnership also allowed me to develop real connections with our collaborators, and it continually challenged my preconceptions about my role as a designer and researcher.
The amount of additional care and labor required for maintaining a healthy partnership made it apparent that co-design would not be appropriate for many time- and resource-constrained projects, or for efforts that are not community-focused. However, I also learned that co-design in practice exists along a spectrum of involvement, and I firmly believe that it has a place in industry practices that seek to center community perspectives. My increased familiarity with the basic processes would also give me a stronger knowledge base to work from in future co-design partnerships.
Mirror team with nametags we created during Urban ArtWorks’ program kickoff day