TextAid Intervention Study Results and Synthesis

Pau Anzaldo, Oscar Dumlao, Jennifer He, Medha Verma, Mary Zhu

From our baseline study and post-study interviews, we had a couple of key takeaways:

• The list view of unread texts often overwhelmed the users and prevented them from responding. We saw from many of our users that there were multiple periods through the day where they were incentivized to respond to their text messages. However, when they actually went on to their messaging app, they were overwhelmed by the long list of blue circles and were too daunted to actually begin the task.
• The “heaviness” of the text previews often dictated whether they would respond or not. Some of the participants who identified as “texting-procrastinators” mentioned their biggest motive for not responding to texts was if they were sent “loaded” messages. If it required careful reading and a thoughtful response, it was almost never the right time to respond to the message.
• Despite wanting to respond to unread messages, most people had no routine to ensure it happened. Participants mentioned that they recognized that the reason they didn’t respond to their text messages was because they didn’t organize time in their day to respond. However, many argued that this was because their days were very variable. This often led them to forget to respond to the text message.

To ensure we had an effective intervention, we decided to start by system map our participants’ behaviors around responding to messages.

System Map on Text Responses

Our system maps one glaring issue that we wanted to address in our intervention. Our participants had erratic reminders — each with a varying level of motivation — to actually respond to their texts.

For instance, we saw users sometimes were reminded to respond to their texts because they got one urgent message which motivated them to respond to the rest, or it was boredom, or it was a feeling of wanting to “clean up”.

Our system map and takeaways allowed us to have a creative session brainstorming some of our interventions (seen here). Much of our brainstorm focused on two of the main pain points: i.) reducing stress around responding to messages and ii.) reminding users to respond to messages. After much discussion, we decided to focus our attention on the second option because we believed it would result in solving one.

The intervention we ended up landing on was the Sticky Note Challenge.

Participants were given these instructions before the study

The key assumption that we were trying to test was whether more intentional display and organization of unread messages remind and motivate users to respond to their unread messages.

When it came to our actual study, we ended up having 5 participants involved in our study. Unfortunately, two of our original participants dropped our study. Despite having unread messages, they did not feel incentivized enough to change their behavior or they were comfortable with how they were responding to their messages.

After the results of our study came in, we were surprised by the overwhelmingly negative response to the study. Most participants believed that added an additional friction to responding to messages, and it created an even more negative experience. Furthermore, we can see that in the system map, there’s a whole lot of red in this and not a whole lot of blue. The current solution space is just to add, add, add, and we unintentionally did some of that with our intervention too (essentially we said: let’s add more reminders!)

When we did a deeper dive into the results with post-interviews, it became evident why.

1. The act of organizing the data was not inherently stress-reducing. In our post-study interviews, many mentioned that they would rather just respond to their messages than having to go through and summarize each one. In fact, seeing all the messages on their walls sometimes made them more stressed.
2. People were as erratic with noting down when they were most and least stressed. We noticed a really similar pattern to how they responded to texts when it came to this intervention. What reminded them to write down their unread texts on sticky notes and take them down was not their moods rather the same random factors which reminded them to respond to text messages. However, there was a slight positive difference because the intervention moderators (aka us) reminded them also.
3. There was not a significant difference in their response rates. We noticed that despite having a visual display of their messages near them, there wasn’t a significant drop in stress or unread text messages. We, however, believe that this might have been due to limitations of the study also (e.g. half of the study happened over the weekend where participants were not near their desks and working spaces). An interesting insight we noticed from one of our participants was that the sticky notes almost became justifications for why they hadn’t responded.

One of our participants’ data. Moderator nudged participants 2x a day to ensure they completed the activity.

Overall, we saw that because it was more work and not really effective in either reducing stress or responding to messages, our participants didn’t want to do it. We also noticed in our system map that there’s a whole lot of red in this and not a whole lot of blue lines. The current solution space is just to add, add, add, and partially we did that with our intervention too (essentially we said: let’s add more reminders!) Unsurprisingly, this is overwhelming and the addition of more pieces just makes replies more complicated.

However, despite the failed results of our intervention, we were very satisfied with the larger insights that we got from it. We learned many useful insights such as:

i.) Making participants do more work with no reward is a huge nono

ii.) Although passive reminders(sticky notes) don’t change behavior, active reminders (us reminding them to do the study) does

iii.) Having sticky notes in front of them, in cases, increased stress and didn’t help with messaging responses and iv.) you can’t expect users to do a study like this without consistent anchors.

It also led us to a new question: does reducing stress around messaging actually cause users to be more motivated and intentional about responding to messages? Based on our learnings, we decided to the pivot to Omwriter-type solutions, which instead seek to remove/decrease many of these factors (total # reminders, stress, unjustified urgency, guilt) bringing more balance to the system. We hope to explore this in our next study and transition into making a more stress-related platform.