Entre julho e dezembro de 2024, a equipe de Web conduziu experimentos voltados à análise de como facilitar o processo de descoberta de informa??es pelos leitores das wikis. Isso fazia parte do nosso plano anual focado na descoberta de conteúdo para leitores.
|
Experimentos em descoberta de conteúdo
|
Quatro experimentos foram executados pela equipe. Esta página descreve os detalhes de cada experimento, bem como nossas descobertas e cronograma.
Os resultados dos experimentos ser?o utilizados para identificarmos quais recursos desenvolveremos no futuro, utilizando-se como base o qu?o eficazes esses provaram ser na descoberta de informa??es úteis e interessantes por parte dos usuários nas wikis.
Atualiza??es
Contexto
à medida que novas gera??es de leitores visitam nossos sites, queremos tornar o aprendizado mais fácil para eles. Isso inclui facilitar a busca pelas informa??es de que precisam em artigos e páginas, bem como tornar mais fácil a descoberta de novas informa??es com base em suas necessidades ou interesses. Para isso, esperamos aproveitar os avan?os tecnológicos e desenvolver novas capacidades em nossa plataforma, o que nos permitirá tornar a descoberta e a navega??o mais fáceis do que no passado. Este objetivo levou a equipe a se concentrar inicialmente em trabalhos mais experimentais, como uma forma de testar mais recursos rapidamente. Come?amos com a seguinte hipótese:
Projetamos e avaliamos qualitativamente três provas de conceito focadas em criar experiências de navega??o e aprendizado mais organizadas, personalizadas e colaborativas. Isso nos permitiu estimar o potencial de aumento na reten??o de leitores (experimento 1: oferecer conteúdo recomendado em contextos de busca e artigos; experimento 2: resumindo e simplificando o conteúdo do artigo; experimento 3: facilitando a multitarefa em wikis).
Essa hipótese busca identificar ideias para recursos ou projetos que facilitem a navega??o e o aprendizado nas wikis, tanto em websites para desktop quanto para dispositivos móveis. O trabalho envolveu a identifica??o e a cria??o de provas de conceito para cada ideia, além da apresenta??o dos resultados dos testes iniciais.
O objetivo é encontrar ideias que tenham potencial para funcionar em grande escala, desenvolvê-las e disponibilizá-las nas wikis.
Esse trabalho inicia a descoberta em áreas nas quais ainda n?o havíamos trabalhado no passado, como resumir ou reorganizar conteúdo. Trabalhar em conjunto com as comunidades para garantir fluxos de trabalho adequados de edi??o e modera??o para esses novos tipos de conteúdo será crucial.
Experimento 1: Exibir recomenda??es de artigos em locais mais destacados e sugest?es de pesquisa
The search suggestions experiments proposed the development of a feature which provides suggestions to readers when they open the search bar of a page. This feature was proposed as a tool that could make browsing easer for readers by providing a quicker way to explore topics similar to the ones the reader is currently on. This set of experiments explored reader interest in the search feature, and the feature's effects on clickthrough and session length. After all experiment hypotheses were supported, the feature was selected for a full rollout across wikis.
Experiment 2: Improve article recommendations APIs
We wanted to compare various existing and proposed APIs recommending content independent of the user interface. This work allowed us to test different recommendation methods against one another to determine which ones are the most useful for readers.
Experiment hypothesis
If we compare readers preferences for suggested article APIs, we will learn which API readers find most useful.
Experimental setup
A quicksurvey was displayed to a small percentage of readers on English and Spanish Wikipedias, inviting them to take a survey. The survey presented recommendations from three different APIs. Users were be asked to submit which recommendations were most useful and interesting to them.
Experiment 3: Display article recommendations in more prominent locations, article pages
We wanted to explore displaying recommendations in other parts of the page. This experiment focused on displaying recommendations alongside article content on desktop.
Experimental setup
The experiment code was loaded in the Wikipedia Recommendations browser extension. Users of the browser extension saw the recommendations immediately.
Experiment hypothesis
If we display recommendations alongside article pages, people will be interested in reading the recommended pages, as shown through a clickthrough rate similar to, or higher than, existing recommendation entry points (Related pages, Experiment 1).
Results
- On English wikipedia (enwiki) desktop, the overall CTR for article recommendations is 0.54%
- On Spanish wikipedia (eswiki) desktop, the overall CTR for article recommendations is 0.28%
We observed that the CTR for sidebar article recommendations is higher than that for related pages on desktop (listed at the bottom of articles) for both English Wikipedia and Spanish Wikipedia
- English Wikipedia (enwiki): 0.41% ->0.54% , increased 0.13 pp
- Spanish Wikipedia (eswiki): 0.13% -> 0.28%, increased 0.15 pp
Hypothesis was successful:
? Readers will want to click on recommendations on the right of the article (measured by comparing the CTR of recommendations in search with other recommendation locations - specifically related pages on desktop).
Next steps
Since the hypothesis was successful, we recommend to proceed with an A/B test of the feature on the desktop website.
Experiment 4: Article Summaries
Research
- Wikimania 2024, "Written by AI" How do editors and machines collaborate to create content. At Wikimania 2024, Wikimedians discussed ways that AI/machine-generated remixing of our existing content can be used to make Wikipedia more accessible and easier to learn from. The goal for the new features is to help a new generation of readers discover and learn from reliable, encyclopedic information on Wikipedia. The session focused on ways to give control and oversight to editors and communities over the structure and content of these new concept and feature ideas.
- Usability study on Simple Summaries. We ran an unmoderated Userlytics study on a prototype of our Simple Summary feature with 8 participants. The prototype included a summary of the introduction of the English Wikipedia article on Dopamine. We analyzed videos and screen recordings of the sessions. Participants found simple summaries easy to use, useful, and had an appropriate level of trust in the machine-generated summary. Responses were much more positive than expected. The main issue to be addressed before release is ambiguity between the Simple Summary and the main article content while the simple summary accordion is closed.
Timeline
| Estimated time | Planned activity |
|---|---|
| |
| Building experiments | |
| |
| Running experiment 4 |