Objective Argument Summarization in Search

verfasst von

Timon Ziegenbein, Shahbaz Syed, Martin Potthast, Henning Wachsmuth

Abstract

Decision-making and opinion formation are influenced by arguments from various online sources, including social media, web publishers, and, not least, the search engines used to retrieve them. However, many, if not most, arguments on the web are informal, especially in online discussions or on personal pages. They can be long and unstructured, subjective and emotional, and contain inappropriate language. This makes it difficult to find relevant arguments efficiently. We hypothesize that, on search engine results pages,“objective snippets” of arguments are better suited than the commonly used extractive snippets and develop corresponding methods for two important tasks: snippet generation and neutralization. For each of these tasks, we investigate two approaches based on (1) prompt engineering for large language models (LLMs), and (2) supervised models trained on existing datasets. We find that a supervised summarization model outperforms zero-shot summarization with LLMs for snippet generation. For neutralization, using reinforcement learning to align an LLM with human preferences for suitable arguments leads to the best results. Both tasks are complementary, and their combination leads to the best snippets of arguments according to automatic and human evaluation.

Organisationseinheit(en)

Fachgebiet Maschinelle Sprachverarbeitung

Externe Organisation(en)

Universität Leipzig
Bauhaus-Universität Weimar
Zentrum für skalierbare Datenanalyse und Künstliche Intelligenz Dresden/Leipzig (ScaDS.AI)

Typ

Aufsatz in Konferenzband

Publikationsdatum

06.2024

Publikationsstatus

Veröffentlicht

Peer-reviewed

Ja