Problem

• For example users with the roles such as 'Director', 'CEO', 'Founder', 'Executive Director', 'Creative Director', 'Manager' are picking other over C-level / Executive.  
• Some roles are also not available when selecting certain organizations, for example Consultant is only shown when selecting the Self Employed as your organization.
• Some roles do not fit in with any of our current roles, example includes any healthcare roles, or Product/PM work. If we want to get rid of other as an option, we should consider where these roles will fit.

Onboarding flow

Goal

Capture our core users accurately through onboarding and increase the sum of core roles by 7%.

Assumption

• Many users that select other as their role could fall under our core roles (execs/ hr/ marketers/ analysts/ consultants).
• The OTHER Organization should have a similar distribution of roles as the rest of the organizations, and can be the first organization we experiment with since there is a huge gap between those that select other as their role (8.56%) and those that select an actual role (14.4%).

Hypothesis

We believe that [ by providing more encompassing role titles for execs, HR, analysts and by providing an additional consultant role title ]
For [ new users who select OTHER as their organization ]
Will [ increase the % of users choosing the 5 core roles from 8.03% by a minimum of 7% ]
Because [ these users aren’t currently finding role titles that they identify with due to the specificity of the language used, and choose other instead ]

Design

I explored ideas of asking for role information in the second or third session to make the onboarding experience more seamless - with the assumption that many users just want to get to know the tool first and see whether it will work for them without having to answer so many questions. However, we heavily relied on role information for our marketing campaigns and personalization, therefore we decided to come back to these ideas later on.

For the first iteration, We changed the role grouping names to be more inclusive, and added a few new roles that did not have an equivalent such as Learning & development and Product development. We kept the role other and made sure the position of roles would randomize.

For the second iteration, we removed the role other from the mix.

Results

We released this experiment to 20% of users going through onboarding, of those users if they selected OTHER as their organization they saw the new role options. Each week between 1300-1600 users went through the new flow.

We saw an increase in the sum of core roles by ~17%, therefore this experiment was considered a success. Onboarding completion was not affected.

Opportunity

We identified that 75.5% of our retained and active user cohorts signed up with a company email (email that did not contain yahoo, gmail, outlook, aol, iCloud, hotmail).

If people who provide their work emails are more likely to be retained, how can we encourage them? 

Many opportunities can be explored once a user signs up with their work email : We can auto generate brand assets (experiment 3), encourage them to add team members, show them the value of Venngage teams, and ultimately upsell them to the business plan.

Goal

The goal was to determine if a customer was going to use Venngage for Business (vs personal use) and if so, ask for their work email upfront.

Increase the % of core users registering with a company email - specifically increase the baseline from 17.8% to 22.8% (↑ 5%) in one week.

Hypothesis

We believe that [ targeting users that are coming to Venngage with a business JTBD and suggesting they input their work email in the email input field on the sign up page] 
‍For [ 50% of new users who enter the registration page through the top sign up flows ]
‍Will [increase the % of core users that register with their work email from 17.8% to 22.8% ]
‍‍Because [ presumably the majority of users coming to Venngage with a business JTBD will have a company email and have the potential to add their work email. ] 

Design

Using Evan Miller's sample size calculator, at a 5% absolute MDE: 946 users are needed in the sample (37.14% of our core users that go through this flow in 1 week).

We will release the experiment to 50% of users who sign up through the top sign up flows. After 1 week if 946 core users have seen the experimental condition and there has been an increase of 5% in those users registering with a work email, the experiment can be considered a success.

Adding a step before registration could impact registration rates, therefore we will also be monitoring this.

Results

Next Steps

The experiment was a success. We decided to incorporate the question as part of onboarding permanently so that we could further personalize it for users with different JTBDs. We also changed the sign up email copy to encourage work email input for everyone.

Rebranded onboarding with JTBD question

Problem

In 2018, we worked on a feature called autobrand where users could input a website URL and in response get the logo and colours used on the website automatically added to their brand kit instead of having to do so manually.

The feature was not very discoverable, as it was tucked all the way inside the brand kit. In 2018 it was surfaced to all users during onboarding, however it did not lead to an increase in brand usage and negatively impacted onboarding metrics.

I revisited the data : In 2019, the volume of users having confirmed autobrand and retaining after 8 weeks was about 2x higher than users retaining from the confirmation of autobrand in 2020 during the same timeline (when autobrand was removed from onboarding and only visible in the brand kit). Engagement in the brand kit had dropped significantly overall for the same cohort and time period.

I also did a deep dive on the autobrand generations. This led me to 2 hypotheses for why the experiment had failed in 2018.

• The quality of generations was not good.
  I did an audit of what was being generated against the companies of our users and they were not always accurate. After the launch of the feature in 2018, we had not invested any more resources into improving the generations.
  
  I did some research on using a third party API instead of our built in autobrand feature - so that we could solve our problems and not worry about consistent maintenance. I came across the Brandfetch API and the team decided to test it out in the Venngage environment. The generations were much better than what we previously had, especially when tested against the small businesses of our customers. We decided to make the switch permanent! 
• There was no personalization on who was seeing the autobrand question in onboarding.
  We decided to test this hypothesis out by only surfacing it for users coming in with a business JTBD.

Goal

Increase engagement with the brand for users coming in with a business JTBD.

Assumptions

• Bringing back autobrand into onboarding for users that are coming in with a business JTBD will increase awareness of the feature
• It will not impact onboarding completion, as we are being intentional with who is seeing this new question
• A longer onboarding will filter out low quality users early on and will correlate better with retention and conversions

Hypothesis

We believe that [ adding autobrand back in to onboarding ]
‍For [ users that select 'Work' as their JTBD ]
‍Will [maintain onboarding completion rates and increase engagement with the brand ]
‍‍Because [ we are being intentional with who we are showing autobrand to] 

Design

Onboarding Overview

Experiment Flow

Results

Confirmed autobrand

• 54.62% confirmed autobrand during onboarding
• Confirmed autobrand → Creation of a template (68%)
• Confirmed autobrand → Use brand kit in the editor (14.12%)
• Confirmed autobrand → Upgrade to Business (1.82%)

Skipped Autobrand

• 47.08% skipped autobrand during onboarding
• Skipped autobrand → Creation of a template (65%)
• Skipped autobrand → Use brand kit in the editor (1.52%)
• Skipped autobrand → Upgrade to Business (0.32%)

Next Steps

After digging into the metrics, I noticed that majority of the users skipping autobrand during onboarding have not signed up with work emails. They have used emails such as @gmail, @hotmail etc, therefore we are not able to pre-generate their brand assets by using the company domain. They have to manually enter their website and this extra step can easily be skipped.

One of the first steps will be looking at ways to engage these users with the brand kit once they are inside the tool. The simplest, low effort way is to run targeted product tours using Intercom and measure engagement with the brand kit afterwards.