Wikimedia Apps/iOS Suggested edits project/Activity Tab Experiment

Background

In a recent study on Editing and Reading on the iOS App, we saw that notifications about edit counts and activity related to their contributions were motivating for some new editors. One experienced editor also mentioned feeling encouraged by seeing their edit count on the desktop version. But, unstructured editing on the iOS app requires using Wikitext, which is intimidating to most newcomers.

We know from our recent experiences with Personalized Year in Review that viewing insights is motivating for both readers and editors, and that folks are willing to create an account to have access to them.

Our drafted is part of the 2024-2025 Annual Plan


Key Result	iOS team hypothesis	Timeline
Wiki Experiences, Key Result 1.2 Constructive Activation: Widespread deployment of interventions shown to collectively cause a 10% relative increase (y-o-y) on mobile web and a 25% relative increase (y-o-y) on iOS of newcomers who publish ≥1 constructive edit in the main namespace on a mobile device, as measured by controlled experiments.	If we temporarily present a population of new account holders with an invitation to “add an image via Suggested Edits within an Activity tab, we'll learn if this explicit call to action, and corresponding structured workflow, is ≥25% more effective at increasing constructive activation than a generic invitation to "Edit" Wikipedia presented in the same place.	In progress April 2025

How will we know we were successful

Validation

KR 1.1 Users in the experiment group with access to Add an image in their activity tab have a 25% higher constructive activation rate compared to those who did have access (or control group)
KR 1.2 15% of users who engage with the Activity tab click on the prompt for unlocking insights

Guardrails

Revert rate for edits from users who engaged with Activity tab does not exceed 18%

Curiosities

What other features are users looking for in the activity tab? What did they expect to see there? (Survey)

Experiment Requirements

The activity tab should be a temporary experiment
Reading **history** should still be accessible for users who opt not to log-in or engage with the Activity tab
All users in target audiences see the new Activity tab
Logged-out users
- See prompt to log-in/create an account for insights
Logged-in users
- Control: Prompt them to edit to make their editing stats increase
- Variant: Prompt them to add an image to make their editing stats increase. Show entry point to Add an image
- Both: Display at least one reading insight
“Add an image” is available to users under 50 edits for the target audiences in the experiment group
Editing stats increase after someone makes an edit
After engaging with the Activity tab, users are asked for their feedback through an in-app survey or Google Form survey

Audience

Our audience is newcomers, and new account holders. We plan to run a short, 15-day experiment in the following languages. We chose these languages because they already have the Growth Image Recommendations feature available to newcomers on Web (https://www.mediawiki.org/wiki/Growth/Deployment_table)

Chinese
French
Turkish
Spanish

How you can follow along

We have created Activity Tab Experiment as our Phabricator epic. We encourage your collaboration there or on our Talk Page. There will also be periodic updates to this page as we make progress on the experiment.

Translations

If you are interested to help add translations, this feature uses the Wikipedia iOS Mobile Group. Search "activity-tab" to find the relevant translations. Translations are especially welcome for our experiment languages of Chinese, French, Turkish, and Spanish.

Designs

The activity tab will have 2 variations, as shown below. Group B will see a generic invitation to edit that leads to the iOS FAQ Page for editing. Group C will see an invitation to add images using "Add an image". For both groups, after they edit, their edit count will go up, and they'll have access to their contributions.

Updates

June 2025

The experiment ended, and we have 30-day results. From these results, we think that features like the activity tab with a clear call to action for creating an account, and an invitation to edit have the potential to impact activation, and increase reader engagement. Due to the Mobile Apps’ upcoming focus on readers, will focus on exploring the reader engagement potential of the activity tab in further experiments.
- Our hypothesis was: If we temporarily present a population of new account holders with an activity tab containing a generic call to action, or a suggested edit, we’ll see a ≥25% increase in constructive activation compared to control, and learn which editing call to action is more impactful.
- We found our hypothesis overall inconclusive
  - Those who had access to the activity tab activated at a higher rate than those who did not have access, and we saw a slightly higher activation rate in the group that had access to a suggested edit, but results were not statistically significant because of the small sample size.
  - The following factors limited our ability to detect an effect of 25% during a 30 day experiment:
    - In measuring the metric of constructive activation, we’re looking at a point deep in the funnel. This limited our audience in each experiment group to the users who created new accounts.
    - We limited which languages we ran the experiment in, to align with community configuration: some languages do not allow newcomers to access “Add an image” on Web (via Community Configuration), so we limited our languages to those who are opted-in on Web.
- Key learnings for reader engagement
  - Adding the Activity tab space in the app has potential to increase the number of new accounts who constructively activate. We cannot point to which part of the activity tab most impacted activation, but investing in a more holistic reading and editing activity space, and placing it in a prominent location within the app caused more users to activate.
  - Unlocking personalized insights is a strong motivator for users to create an account. 24.1% of logged-out users who looked at the activity tab clicked on the prompt to log-in or create an account to unlock insights. This resulted in a statistically significant 21% increase in account creation rate for the variants compared to control.
  - iOS App users expressed interest in the activity tab experience. 92.9% of users felt neutral, satisfied, or very satisfied with the activity tab. In survey feedback, users requested more insights, including time spent on pages and number of views on articles they’ve edited recently. One beginner editor also asked for additional support in understanding the interface for edits and discussions.
- Validation
  - KR 1.0 Users in the experiment groups with access to activity tab have a 25% higher constructive activation rate compared to those who did have access (or control group)
    - Inconclusive, we saw an increase of more than 28% and 37% for both variant groups compared to control, but this was not statistically significant
  - KR 1.1 Users in the experiment group with access to Add an image in their activity tab have a 25% higher constructive activation rate compared to those who did have access (or control group)
    - Inconclusive, we saw a 6% increase, but it was not statistically significant
  - KR 1.2 15% of users who engage with the Activity tab click on the prompt for unlocking insights
    - Met: 24% of logged-out users who looked at the activity tab clicked on the prompt to log-in or create an account to unlock insights
- Guardrails
  - Revert rate for edits from users who engaged with Activity tab does not exceed 18%
    - Met: The revert rate in the variant groups did not exceed 18%. Group B had a revert rate of 15%, and group C had a revert rate of 12%.

May 2025

The experiment is running and we’ll evaluate its success after 30 days (T391997)

April 2025

We released the experiment to external testers on Chinese, French, Spanish, and Turkish Wikipedias, and we’re monitoring for issues!