What should I wear to a party in a Greek taverna? Evaluation for Conversational Agents in the Fashion Domain
Large language models (LLMs) are poised to revolutionize the domain of online fashion retail, enhancing customer experience and discovery of fashion online. LLM-powered conversational agents introduce a new way of discovery by directly interacting with customers, enabling them to express in their own ways, refine their needs, obtain fashion and shopping advice that is relevant to their taste and intent. For many tasks in e-commerce, such as finding a specific product, conversational agents need to convert their interactions with a customer to a specific call to different backend systems, e.g., a search system to showcase a relevant set of products. Therefore, evaluating the capabilities of LLMs to perform those tasks related to calling other services is vital. However, those evaluations are generally complex, due to the lack of relevant and high quality datasets, and do not align seamlessly with business needs, amongst others. To this end, we created a multilingual evaluation dataset of 4k conversations between customers and a fashion assistant in a large e-commerce fashion platform to measure the capabilities of LLMs to serve as an assistant between customers and a backend engine. We evaluate a range of models, showcasing how our dataset scales to business needs and facilitates iterative development of tools.
Code (0)
등록된 구현이 없습니다.
Tasks
4kMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
SSN\_NLP\_MLRG at SemEval-2020 Task 12: Offensive Language Identification in English, Danish, Greek Using BERT and Machine Learning Approach
Offensive language identification is to detect the hurtful tweets, derogatory comments, swear words on social media. As an emerging growth of social media communication, offensive language detection has received more att…
Language IdentificationSecure Multi-party Computation for Cloud-based Control
In this chapter, we will explore the cloud-outsourced privacy-preserving computation of a controller on encrypted measurements from a (possibly distributed) system, taking into account the challenges introduced by the dy…
Privacy PreservingHELFI: a Hebrew-Greek-Finnish Parallel Bible Corpus with Cross-Lingual Morpheme Alignment
Twenty-five years ago, morphologically aligned Hebrew-Finnish and Greek-Finnish bitexts (texts accompanied by a translation) were constructed manually in order to create an analytical concordance (Luoto et al., 1997) for…
TranslationLogion: Machine Learning for Greek Philology
This paper presents machine-learning methods to address various problems in Greek philology. After training a BERT model on the largest premodern Greek dataset used for this purpose to date, we identify and correct previ…
Causal Hangover Effects
It's not unreasonable to think that in-game sporting performance can be affected partly by what takes place off the court. We can't observe what happens between games directly. Instead, we proxy for the possibility of at…