IN THIS ARTICLE
Whether your Team has only started to reply to reviews or already uses a well-established strategy of processing the user feedback, it is important to see what effect this work has on the app rating.
A user may update the review rating several times, changing the opinion and actively discussing an issue with the app developer. However, stores don't provide features for tracking updates made by users. So, it is hard to estimate the developer impact on the user opinion.
At AppFollow we have created an algorithm that helps evaluating the real effect of your replies. It counts only rating updates that happened as a result of the developer reply. The system takes into account the chain of review replies and rating updates and the time between the rating update and developer reply.
Reply effect is a vital KPI to create and adjust your reply-to-reviews strategy. AppFollow Team has built an algorithm that measures what effect developer reply has on rating change of the app.
The algorithm is based on 2 major factors: whether review was updated after developer reply and how much time has passed between reply and update.
To calculate the Reply Effect we count:
1. Review ratings update that took place after the developer reply.
Example of the review update WITHOUT the Reply Effect:
Example of the review update WITH the Reply Effect:
2. Review rating update made within 3 months since the developer reply.
Since to adjust your reply strategy it is important to be sure that the review was updated BECAUSE of developer reply, a time threshold has been set. If there's more than 3 months passed between reply and update, such updates are considered to have no reply effect.
Example of the review update WITHOUT the Reply Effect (user updated the review more than 3 months later than the developer reply):
Example of the review update WITH the Reply Effect (the user updated the review rating the next day since the developer reply):
Where can I track the Reply Effect?
There are a number of pages you can track the Reply Effect on.
Reviews Analysis provides the most detailed statistics on updated reviews and the effect of the developer replies on the review rates.
This section shows the effect of reviews update on total rating of the app. This way only the latest update of the review rating within the selected time range.
For example, a user changed the rating for the review from 4 to 3 stars and then back from 3 to 4 within the period of time. In the Reply Effect table, we will only count the final review rating update (plus 1).
Replies Effect table provides the following data:
1. Total updated reviews - all reviews that have the latest update within the selected time range.
The list of all updated reviews is available on the Reply to Reviews page by applying the filter by update (‘Updated’).
2. No Reply Effect - all updated reviews that don’t have the developer reply or the developer reply was left earlier than 3 months before the update.
The list of all updated reviews without reply effect is available on the Reply to Reviews page by applying the filter by update (‘No Reply Effect’).
3. Reply Effect - all updated reviews that have the developer reply within 3 months before the update. Such a review update is counted as a result of the developer’s reply.The list of all updated reviews with reply effect is available on the Reply to Reviews page by applying the filter by update (‘Reply Effect’).
Table columns cover:
- Average rating change - the change of the average ratings of the reviews with the changes to the previous period;
- Updates to ratings - the share of reviews with the ratings changed to more or less stars of without changes in ratings;
- Matches - number of reviews with the latest update within the selected time range;
- Share of all reviews - the share of the matches in the total number of reviews received within the selected time range.
Ratings & Reviews Dashboard
Ratings & Reviews Dashboard - Reply Stats section helps to get a quick overview of how the Average Ratings Changed by reviews with the Reply Effect and without. You can quickly see if the efforts on replying to reviews are worth it.
Here you’ll find the Average Rating Change for the selected app with and without replies.
Reply to Reviews
Apply the filters by update and by rating change to analyze how exactly the review rating changed. Click on the ‘Show history’ to see the details on the review changes over time.
Replies chart shows how many replies were submitted and how the replies affected the review rating changes.
On this chart, the total number of replies, replies time, and the report a concern statistics for a period of time is displayed.
On the Replies tab, use the filter by review changes to view how many replies affected the review rating change:
- All reviews - total number of reviews for the period;
- Became better - review rating increased after the developer reply;
- Became worse - review rating decreased after the developer reply;
- Without changes - reviews without rating updates.
Apply the filter by users to view the impact of each team member in replying to reviews.
Under the chart, there is the total number of replies sent to the store, reviews received and the average ratings change within the time range.
The Reply Effect will help to understand how the replies are affecting your app rating and get insight on the optimisation of the Team efforts. We’ll be adding more valuable reports on the Reply Effect in the recent future.
In the meantime, if you have any questions or feedback, hit the chat button to reach us or email to firstname.lastname@example.org.