{"id":526,"date":"2023-05-20T11:11:03","date_gmt":"2023-05-20T09:11:03","guid":{"rendered":"https:\/\/thomaskosch.com\/?p=526"},"modified":"2023-10-24T10:39:58","modified_gmt":"2023-10-24T08:39:58","slug":"ai-placebos-undermine-the-validity-of-human-ai-studies","status":"publish","type":"post","link":"https:\/\/thomaskosch.com\/index.php\/2023\/05\/20\/ai-placebos-undermine-the-validity-of-human-ai-studies\/","title":{"rendered":"Placebo Effects Undermine the Validity of Human-AI Studies"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">Supposed enhancements through the presence of Artificial Intelligence\n(AI) can increase user expectations towards self-perceived interaction\nefficiency. User expectations are a prerequisite for placebo effects that can\nundermine the validity of human-AI evaluations and individual risk-taking\nbehavior. We evaluated this hypothesis in two user studies.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">To confirm changes in user expectations and, therefore, the presence of\nplacebo effects during the interaction with AI systems, we designed a study\nwhere participants solved word puzzles with different difficulty adaptations\n[1]. A narrative primed participants that they were interacting with an AI\nadapting the word puzzle difficulty based on performance or facial expressions.\nA third narrative explained that no adaptation occurred, and the word puzzles\nwere displayed in a randomized order. However, the system did not adjust the\nword puzzle difficulty, and all participants were confronted with randomized\nword puzzles.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">We probed the user expectations before and after the interaction, showing that manipulating user expectations worked before and posterior to the interaction. Participants were believed to perform significantly better using AI adaptations than not receiving support. This impacted the subjective and objective performance as well.<\/p>\n\n\n\n<figure class=\"wp-block-gallery columns-2 is-cropped wp-block-gallery-1 is-layout-flex wp-block-gallery-is-layout-flex\"><ul class=\"blocks-gallery-grid\"><li class=\"blocks-gallery-item\"><figure><img loading=\"lazy\" decoding=\"async\" width=\"942\" height=\"940\" src=\"https:\/\/thomaskosch.com\/wp-content\/uploads\/2023\/05\/prior_expectations.png\" alt=\"\" data-id=\"528\" data-full-url=\"https:\/\/thomaskosch.com\/wp-content\/uploads\/2023\/05\/prior_expectations.png\" data-link=\"https:\/\/thomaskosch.com\/prior_expectations\/\" class=\"wp-image-528\" srcset=\"https:\/\/thomaskosch.com\/wp-content\/uploads\/2023\/05\/prior_expectations.png 942w, https:\/\/thomaskosch.com\/wp-content\/uploads\/2023\/05\/prior_expectations-300x300.png 300w, https:\/\/thomaskosch.com\/wp-content\/uploads\/2023\/05\/prior_expectations-150x150.png 150w, https:\/\/thomaskosch.com\/wp-content\/uploads\/2023\/05\/prior_expectations-768x766.png 768w, https:\/\/thomaskosch.com\/wp-content\/uploads\/2023\/05\/prior_expectations-510x510.png 510w\" sizes=\"auto, (max-width: 942px) 100vw, 942px\" \/><\/figure><\/li><li class=\"blocks-gallery-item\"><figure><img loading=\"lazy\" decoding=\"async\" width=\"941\" height=\"933\" src=\"https:\/\/thomaskosch.com\/wp-content\/uploads\/2023\/05\/post_expectations.png\" alt=\"\" data-id=\"530\" data-full-url=\"https:\/\/thomaskosch.com\/wp-content\/uploads\/2023\/05\/post_expectations.png\" data-link=\"https:\/\/thomaskosch.com\/post_expectations\/\" class=\"wp-image-530\" srcset=\"https:\/\/thomaskosch.com\/wp-content\/uploads\/2023\/05\/post_expectations.png 941w, https:\/\/thomaskosch.com\/wp-content\/uploads\/2023\/05\/post_expectations-300x297.png 300w, https:\/\/thomaskosch.com\/wp-content\/uploads\/2023\/05\/post_expectations-150x150.png 150w, https:\/\/thomaskosch.com\/wp-content\/uploads\/2023\/05\/post_expectations-768x761.png 768w\" sizes=\"auto, (max-width: 941px) 100vw, 941px\" \/><\/figure><\/li><\/ul><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Our results show that the community must rethink how to evaluate\nhuman-AI interaction. The research community falls back on human-computer\ninteraction methods, which are not suitable for human-AI evaluations.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A follow-up study exemplarily shows the implications of the example of risk-taking behavior. In a user study, we show that participants who an AI augments are willing to take more risks, although the AI is a sham and not contributing to augmentation at all [2]. Measuring electroencephalography, an assessment modality for cortical activity, during the experiment, we find potential features that can be used to detect perceived conflicts during the interaction with human-AI systems, such as the P300 or N400.<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"490\" height=\"326\" src=\"https:\/\/thomaskosch.com\/wp-content\/uploads\/2023\/05\/erp.png\" alt=\"\" class=\"wp-image-529\" srcset=\"https:\/\/thomaskosch.com\/wp-content\/uploads\/2023\/05\/erp.png 490w, https:\/\/thomaskosch.com\/wp-content\/uploads\/2023\/05\/erp-300x200.png 300w\" sizes=\"auto, (max-width: 490px) 100vw, 490px\" \/><\/figure><\/div>\n\n\n\n<p class=\"wp-block-paragraph\">How can AI placebos be sensed and controlled? Together with LMU Munich\nand Aalto University collaborators, we are currently working on these exciting\nresearch challenges.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">[1]: Kosch, T., Welsch, R., Chuang, L.,\n&amp; Schmidt, A. (2023). The Placebo Effect of Artificial Intelligence in\nHuman\u2013Computer Interaction.&nbsp;ACM Transactions on Computer-Human Interaction,&nbsp;29(6),\n1-32.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">[2]: Villa, S., Kosch, T., Grelka, F.,\nSchmidt, A., &amp; Welsch, R. (2023). The placebo effect of human augmentation:\nAnticipating cognitive augmentation increases risk-taking behavior.&nbsp;Computers\nin Human Behavior, 107787.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Supposed enhancements through the presence of Artificial Intelligence (AI) can increase user expectations towards self-perceived interaction efficiency. User expectations are a prerequisite for placebo effects that can undermine the validity of human-AI evaluations and individual risk-taking behavior. We evaluated this hypothesis in two user studies. To confirm changes in user expectations and, therefore, the presence [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":531,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-526","post","type-post","status-publish","format-standard","has-post-thumbnail","category-uncategorized","czr-hentry"],"_links":{"self":[{"href":"https:\/\/thomaskosch.com\/index.php\/wp-json\/wp\/v2\/posts\/526","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/thomaskosch.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/thomaskosch.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/thomaskosch.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/thomaskosch.com\/index.php\/wp-json\/wp\/v2\/comments?post=526"}],"version-history":[{"count":3,"href":"https:\/\/thomaskosch.com\/index.php\/wp-json\/wp\/v2\/posts\/526\/revisions"}],"predecessor-version":[{"id":543,"href":"https:\/\/thomaskosch.com\/index.php\/wp-json\/wp\/v2\/posts\/526\/revisions\/543"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/thomaskosch.com\/index.php\/wp-json\/wp\/v2\/media\/531"}],"wp:attachment":[{"href":"https:\/\/thomaskosch.com\/index.php\/wp-json\/wp\/v2\/media?parent=526"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/thomaskosch.com\/index.php\/wp-json\/wp\/v2\/categories?post=526"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/thomaskosch.com\/index.php\/wp-json\/wp\/v2\/tags?post=526"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}