AI ASSISTANT

Litmus Assistant - powered by ChatGPT | Litmus | 04-2023

Product Design, Animation, IxD, UI, UX, Product Strategy

Lead designer

RESEARCH AND DISCOVERY

We conducted research on the Builder code editor during Litmus Live. Some takeaways are the need for spellcheck and language translation. However, this AI project was also a competitive play to maintain our status as an email leader in the industry. I conducted a competitive analysis and found other email tools are already launching AI powered subject line generators.

THE GOAL

The goal was to test Phase 1 of incorporating copywriting into the email product workflow but also launch and learn about how receptive customers are to AI technology.

As apart of the Engage & Retain squad, we were focused on the Adopt stage of the customer lifecycle, turning Beginners into Regulars. The feature is available on all plans, so new user acquisition was another goal.

The PROBLEM

Currently, copywriting happens outside of the Litmus product. Once copy and design are approved, users start coding their email in Litmus.

How Might We…leverage ChatGPT to help email marketers write better and more effective email copy and spark inspiration?

THE CHALLENGES

  • Should the AI Assistant have a name and mascot? What is their voice and tone?

  • ChatGPT’s API is a generative AI model, so there are limitations to copywriting

  • Crafting the right prompt for ChatGPT is tricky since the output needs to be formatted

  • 1 month quick turnaround timeframe and small engineering team of 3

THE PROCESS

At first, my team wanted to launch 4 copywriting capabilities: Proofreading, Subject line generation, Tone of voice rewriting and Language translation. After showing my initial designs to Engineering for a feasibility gut check, I learned that given the timeframe it would be tough to launch all 4. I advocated strongly for Subject line generation over Proofreading since ChatGPT’s strength is copy generation. I mocked up designs for Proofreading, but in order to do it well, there was a lot of complexity around type of error, skip or ignore parts, etc. If we pursued Proofreading, we would leverage another AI model’s API like Grammarly. We settled on the 3 capabilities, since it was the most straightforward with an input and an output.

 
 

DESIGN DECISIONS

  • Litmus Assistant name - for Phase 1 we decided to go with ‘Litmus Assistant’ since the assistant might offer general email help in the future, not limited to only AI functionality.

  • Mascot - internally the Litmoose is our mascot, so we thought it would add a touch of personality and delight to our microanimations even if the assistant has no name.

  • Email centric layout - mirrors other parts of the app but also is one step towards a future vision of a sidebar instead of tabs with the email at the center. Visually it gives context for the input into the assistant, the email content helps generate the output.

  • Tone of voice and Language options - we looked at common tones in UX writing from NNG and selected 5 that made the most sense for email. For language, we pulled a list of top languages in the world using a Latin alphabet for left-to-right readability then selected the top 6 languages.

  • CTAs - Generate again is the primary CTA, but for Phase 1 we offer Copy as a secondary CTA. This is because copy generations will change each time, especially for subject line. Also, copying the output is needed when copy creation lives outside the Litmus app.

 

USER STORY MAPPING

Collaborating with the engineers, we did a story mapping exercise to facilitate agile development. We carved out the tasks for each capability and was able to estimate the projected level of effort and time. This is where we also weighed the 3 final capabilities, descoping Proofreading.

Chat gpt prompt design

In order to generate the output in the proper formatting to feed into the user interface, we had to get creative with ChatGPT’s prompt design. We have best practices in Builder to keep subject lines under 40 characters. For the prompt, we asked the AI to return 5 subject lines in a numbered list with each one under 40 characters, and using the email’s current subject line and email copy as input. We also played around with the Proofreading prompt, but the output was so hard to make it actually useful. Tone of voice and language were more straightforward prompts.

THE SOLUTION

The first screen is an easy to get started menu, introducing the 3 capabilities: Subject line generator, Tone of voice and Language translation. There’s a Litmoose microanimation as the loading state for delight. Labor perception theory says that users expect complex actions to take more time, so adding a fun loading state helps create more value during the wait. The second screen offers the output copy in an editable field for easy copying of select parts, as well as a dropdown navigation at the top instead of going back to the first screen. Below every output, there is a feedback banner with thumbs up/thumbs down - so if we ever wanted to collect the data ourselves, we can train the model. Friendly copy is used throughout the experience to give Litmus Assistant personality and convey a conversational tone that is helpful.

 

ReSULTS and IMPACT

  • Within the first week of launch, there were 1,878 unique views on the Litmus Assistant

  • Subject line generation is the most popular capability with 501 requests made to the API in the first week

  • Lesson learned was some customers are scared of AI and have asked to be opted out of the Beta

Next iteration: In Phase 2 we want to create a copy of your current email with the generated version for tone of voice and language, and the ability to replace a suggested subject line.