About me
I am a Ph.D. candidate at the faculty of Data and Decisions Science of the Technion – Israel Institute of Technology. My advisor is Professor Roi Reichart, and I am honored to be a Clore fellow. Before that, I graduated with my B.Sc. in Data Science and Engineering from the Technion and was awarded the Schulich Leaders scholarship.
My research is in the field of Natural Language Processing (NLP) and lies in the intersection of Domain Adaptation, Causal Inference, and Language Generation. Specifically, I develop causal-inspired methods to improve out-of-distribution generalization, interpretability, and reliability of NLP systems. I am also working on developing NLP technology for scientific modelling in brain, cognitive and mental health sciences.
I’m happy to talk about my research. If you have any questions about one of my papers, or my overall research, feel free to reach out!
Publications
On Behalf of the Stakeholders: Trends in NLP Model Interpretability in the Era of LLMs
Nitay Calderon and Roi Reichart
Published in arXiv (under review), 2024
Abstract
Recent advancements in NLP systems, particularly with the introduction of LLMs, have led to widespread adoption of these systems by a broad spectrum of users across various domains, impacting decision-making, the job market, society, and scientific research. This surge in usage has led to an explosion in NLP model interpretability and analysis research, accompanied by numerous technical surveys. Yet, these surveys often overlook the needs and perspectives of explanation stakeholders. In this paper, we address three fundamental questions: Why do we need interpretability, what are we interpreting, and how? By exploring these questions, we examine existing interpretability paradigms, their properties, and their relevance to different stakeholders. We further explore the practical implications of these paradigms by analyzing trends from the past decade across multiple research fields. To this end, we retrieved thousands of papers and employed an LLM to characterize them. Our analysis reveals significant disparities between NLP developers and non-developer users, as well as between research fields, underscoring the diverse needs of stakeholders. For example, explanations of internal model components are rarely used outside the NLP field. We hope this paper informs the future design, development, and application of methods that align with the objectives and requirements of various stakeholders.bibtex
@article{calderon2024trends,
title={On Behalf of the Stakeholders: Trends in NLP Model Interpretability in the Era of LLMs},
author={Calderon, Nitay and Reichart, Roi},
journal={arXiv preprint arXiv:2407.19200},
year={2024}
}
Measuring the Robustness of NLP Models to Domain Shifts
Nitay Calderon*, Naveh Porat*, Eyal Ben-David, Alexander Chapanin, Zorik Gekhman, Nadav Oved, Vitaly Shalumov and Roi Reichart
*Equal contribution
Published in arXiv (under review), 2024
Abstract
Existing research on Domain Robustness (DR) suffers from disparate setups, limited task variety, and scarce research on recent capabilities such as in-context learning. Furthermore, the common practice of measuring DR might not be fully accurate. Current research focuses on challenge sets and relies solely on the Source Drop (SD): Using the source in-domain performance as a reference point for degradation. However, we argue that the Target Drop (TD), which measures degradation from the target in-domain performance, should be used as a complementary point of view. To address these issues, we first curated a DR benchmark comprised of 7 diverse NLP tasks, which enabled us to measure both the SD and the TD. We then conducted a comprehensive large-scale DR study involving over 14,000 domain shifts across 21 fine-tuned models and few-shot LLMs. We found that both model types suffer from drops upon domain shifts. While fine-tuned models excel in-domain, few-shot LLMs often surpass them cross-domain, showing better robustness. In addition, we found that a large SD can often be explained by shifting to a harder domain rather than by a genuine DR challenge, and this highlights the importance of TD as a complementary metric. We hope our study will shed light on the current DR state of NLP models and promote improved evaluation practices toward more robust models. bibtex
@article{calderon2023measuring,
title={Measuring the Robustness of NLP Models to Domain Shifts},
author={Calderon, Nitay and Porat, Naveh and Ben-David, Eyal and Chapanin, Alexander and Gekhman, Zorik and Oved, Nadav and Shalumov, Vitaly and Reichart, Roi},
journal={arXiv preprint arXiv:2306.00168},
year={2024}
}
The Colorful Future of LLMs: Evaluating and Improving LLMs as Emotional Supporters for Queer Youth
Shir Lissak*, Nitay Calderon*, Geva Shenkman, Yaakov Ophir, Eyal Fruchter, Anat Brunstein Klomek and Roi Reichart
*Equal contribution
Published in NAACL, 2024
Abstract
Queer youth face increased mental health risks, such as depression, anxiety, and suicidal ideation. Hindered by negative stigma, they often avoid seeking help and rely on online resources, which may provide incompatible information. Although access to a supportive environment and reliable information is invaluable, many queer youth worldwide have no access to such support. However, this could soon change due to the rapid adoption of Large Language Models (LLMs) such as ChatGPT. This paper aims to comprehensively explore the potential of LLMs to revolutionize emotional support for queers. To this end, we conduct a qualitative and quantitative analysis of LLM’s interactions with queer-related content. To evaluate response quality, we develop a novel ten-question scale that is inspired by psychological standards and expert input. We apply this scale to score several LLMs and human comments to posts where queer youth seek advice and share experiences. We find that LLM responses are supportive and inclusive, outscoring humans. However, they tend to be generic, not empathetic enough, and lack personalization, resulting in nonreliable and potentially harmful advice. We discuss these challenges, demonstrate that a dedicated prompt can improve the performance, and propose a blueprint of an LLM-supporter that actively (but sensitively) seeks user context to provide personalized, empathetic, and reliable responses. Our annotated dataset is available for further research. bibtex
@inproceedings{lissak-etal-2024-colorful,
title = "The Colorful Future of {LLM}s: Evaluating and Improving {LLM}s as Emotional Supporters for Queer Youth",
author = "Lissak, Shir and
Calderon, Nitay and
Shenkman, Geva and
Ophir, Yaakov and
Fruchter, Eyal and
Brunstein Klomek, Anat and
Reichart, Roi",
editor = "Duh, Kevin and
Gomez, Helena and
Bethard, Steven",
booktitle = "Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)",
month = jun,
year = "2024",
address = "Mexico City, Mexico",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2024.naacl-long.113",
pages = "2040--2079"
}
Faithful Explanations of Black-box NLP Models Using LLM-generated Counterfactuals
Yair Gat*, Nitay Calderon*, Amir Feder, Alexander Chapanin, Amit Sharma and Roi Reichart
*Equal contribution
Published in ICLR, 2024
Abstract
Causal explanations of the predictions of NLP systems are essential to ensure safety and establish trust. Yet, existing methods often fall short of explaining model predictions effectively or efficiently and are often model-specific. In this paper, we address model-agnostic explanations, proposing two approaches for counterfactual (CF) approximation. The first approach is CF generation, where a large language model (LLM) is prompted to change a specific text concept while keeping confounding concepts unchanged. While this approach is demonstrated to be very effective, applying LLM at inference-time is costly. We hence present a second approach based on matching, and propose a method that is guided by an LLM at training-time and learns a dedicated embedding space. This space is faithful to a given causal graph and effectively serves to identify matches that approximate CFs. After showing theoretically that approximating CFs is required in order to construct faithful explanations, we benchmark our approaches and explain several models, including LLMs with billions of parameters. Our empirical results demonstrate the excellent performance of CF generation models as model-agnostic explainers. Moreover, our matching approach, which requires far less test-time resources, also provides effective explanations, surpassing many baselines. We also find that Top-K techniques universally improve every tested method. Finally, we showcase the potential of LLMs in constructing new benchmarks for model explanation and subsequently validate our conclusions. Our work illuminates new pathways for efficient and accurate approaches to interpreting NLP systems.</pre> bibtex
@article{gat2023faithful,
author = {Yair Ori Gat and
Nitay Calderon and
Amir Feder and
Alexander Chapanin and
Amit Sharma and
Roi Reichart},
title = {Faithful Explanations of Black-box {NLP} Models Using LLM-generated
Counterfactuals},
journal = {CoRR},
volume = {abs/2310.00603},
year = {2023},
url = {https://doi.org/10.48550/arXiv.2310.00603},
doi = {10.48550/ARXIV.2310.00603},
eprinttype = {arXiv},
eprint = {2310.00603},
timestamp = {Wed, 18 Oct 2023 16:20:58 +0200},
biburl = {https://dblp.org/rec/journals/corr/abs-2310-00603.bib},
bibsource = {dblp computer science bibliography, https://dblp.org}
}
Social Media Images Can Predict Suicide Risk Using Interpretable Large Language-Vision Models
Yael Badian, Yaakov Ophir, Refael Tikochinski, Nitay Calderon, Anat Brunstein Klomek, Eyal Fruchter and Roi Reichart
Published in The Journal of Clinical Psychiatry, 2023
Abstract
Background: Suicide, a leading cause of death and a major public health concern, became an even more pressing matter since the emergence of social media two decades ago and, more recently, following the hardships that characterized the COVID-19 crisis. Contemporary studies therefore aim to predict signs of suicide risk from social media using highly advanced artificial intelligence (AI) methods. Indeed, these new AI-based studies managed to break a longstanding prediction ceiling in suicidology; however, they still have principal limitations that prevent their implementation in real-life settings. These include “black box” methodologies, inadequate outcome measures, and scarce research on non-verbal inputs, such as images (despite their popularity today).
Objective: This study aims to address these limitations and present an interpretable prediction model of clinically valid suicide risk from images.
Methods: The data were extracted from a larger dataset from May through June 2018 that was used to predict suicide risk from textual postings. Specifically, the extracted data included a total of 177,220 images that were uploaded by 841 Facebook users who completed a gold-standard suicide scale. The images were represented with CLIP (Contrastive Language-Image Pre-training), a state-of-the-art deep-learning algorithm, which was utilized, unconventionally, to extract predefined interpretable features (eg, “photo of sad people”) that served as inputs to a simple logistic regression model.
Results: The results of this hybrid model that integrated theory-driven features with bottom-up methods indicated high prediction performance that surpassed common deep learning algorithms (area under the receiver operating characteristic curve [AUC] = 0.720, Cohen d = 0.82). Further analyses supported a theory-driven hypothesis that at-risk users would have images with increased negative emotions and decreased belongingness.
Conclusions: This study provides a first proof that publicly available images can be leveraged to predict validated suicide risk. It also provides simple and flexible strategies that could enhance the development of real-life monitoring tools for suicide. bibtex
@article{badian2023social,
title={Social media images can predict suicide risk using interpretable large language-vision models},
author={Badian, Yael and Ophir, Yaakov and Tikochinski, Refael and Calderon, Nitay and Klomek, Anat Brunstein and Fruchter, Eyal and Reichart, Roi},
journal={The Journal of Clinical Psychiatry},
volume={85},
number={1},
pages={50516},
year={2023},
publisher={Physicians Postgraduate Press, Inc.}
}
A Systematic Study of Knowledge Distillation for Natural Language Generation with Pseudo-Target Training
Nitay Calderon, Subhabrata Mukherjee, Roi Reichart and Amir Kantor
Published in ACL, 2023
Abstract
Modern Natural Language Generation (NLG) models come with massive computational and storage requirements. In this work, we study the potential of compressing them, which is crucial for real-world applications serving millions of users. We focus on Knowledge Distillation (KD) techniques, in which a small student model learns to imitate a large teacher model, allowing to transfer knowledge from the teacher to the student. In contrast to much of the previous work, our goal is to optimize the model for a specific NLG task and a specific dataset. Typically, in real-world applications, in addition to labeled data there is abundant unlabeled task-specific data, which is crucial for attaining high compression rates via KD. In this work, we conduct a systematic study of task-specific KD techniques for various NLG tasks under realistic assumptions. We discuss the special characteristics of NLG distillation and particularly the exposure bias problem. Following, we derive a family of Pseudo-Target (PT) augmentation methods, substantially extending prior work on sequence-level KD. We propose the Joint-Teaching method for NLG distillation, which applies word-level KD to multiple PTs generated by both the teacher and the student. Our study provides practical model design observations and demonstrates the effectiveness of PT training for task-specific KD in NLG. bibtex
@inproceedings{calderon2023systematic,
author = {Nitay Calderon and
Subhabrata Mukherjee and
Roi Reichart and
Amir Kantor},
editor = {Anna Rogers and
Jordan L. Boyd{-}Graber and
Naoaki Okazaki},
title = {A Systematic Study of Knowledge Distillation for Natural Language
Generation with Pseudo-Target Training},
booktitle = {Proceedings of the 61st Annual Meeting of the Association for Computational
Linguistics (Volume 1: Long Papers), {ACL} 2023, Toronto, Canada,
July 9-14, 2023},
pages = {14632--14659},
publisher = {Association for Computational Linguistics},
year = {2023},
url = {https://doi.org/10.18653/v1/2023.acl-long.818},
doi = {10.18653/v1/2023.acl-long.818},
timestamp = {Thu, 10 Aug 2023 12:36:02 +0200},
biburl = {https://dblp.org/rec/conf/acl/CalderonMRK23.bib},
bibsource = {dblp computer science bibliography, https://dblp.org}
}
A Picture May Be Worth a Thousand Lives: An Interpretable Artificial Intelligence Strategy for Predictions of Suicide Risk from Social Media Images
Yael Badian, Yaakov Ophir, Refael Tikochinski, Nitay Calderon, Anat Brunstein Klomek and Roi Reichart
Published in arXiv, 2023
Abstract
The promising research on Artificial Intelligence usages in suicide prevention has principal gaps, including black box methodologies, inadequate outcome measures, and scarce research on non-verbal inputs, such as social media images (despite their popularity today, in our digital era). This study addresses these gaps and combines theory-driven and bottom-up strategies to construct a hybrid and interpretable prediction model of valid suicide risk from images. The lead hypothesis was that images contain valuable information about emotions and interpersonal relationships, two central concepts in suicide-related treatments and theories. The dataset included 177,220 images by 841 Facebook users who completed a gold-standard suicide scale. The images were represented with CLIP, a state-of-the-art algorithm, which was utilized, unconventionally, to extract predefined features that served as inputs to a simple logistic-regression prediction model (in contrast to complex neural networks). The features addressed basic and theory-driven visual elements using everyday language (e.g., bright photo, photo of sad people). The results of the hybrid model (that integrated theory-driven and bottom-up methods) indicated high prediction performance that surpassed common bottom-up algorithms, thus providing a first proof that images (alone) can be leveraged to predict validated suicide risk. Corresponding with the lead hypothesis, at-risk users had images with increased negative emotions and decreased belonginess. The results are discussed in the context of non-verbal warning signs of suicide. Notably, the study illustrates the advantages of hybrid models in such complicated tasks and provides simple and flexible prediction strategies that could be utilized to develop real-life monitoring tools of suicide. bibtex
@article{badian2023picture,
author = {Yael Badian and
Yaakov Ophir and
Refael Tikochinski and
Nitay Calderon and
Anat Brunstein Klomek and
Roi Reichart},
title = {A Picture May Be Worth a Thousand Lives: An Interpretable Artificial
Intelligence Strategy for Predictions of Suicide Risk from Social
Media Images},
journal = {CoRR},
volume = {abs/2302.09488},
year = {2023},
url = {https://doi.org/10.48550/arXiv.2302.09488},
doi = {10.48550/arXiv.2302.09488},
eprinttype = {arXiv},
eprint = {2302.09488},
timestamp = {Thu, 23 Feb 2023 16:02:44 +0100},
biburl = {https://dblp.org/rec/journals/corr/abs-2302-09488.bib},
bibsource = {dblp computer science bibliography, https://dblp.org}
}
A Functional Information Perspective on Model Interpretation
Itai Gat, Nitay Calderon, Roi Reichart and Tamir Hazan
Published in ICML, 2022
Abstract
Contemporary predictive models are hard to interpret as their deep nets exploit numerous complex relations between input elements. This work suggests a theoretical framework for model interpretability by measuring the contribution of relevant features to the functional entropy of the network with respect to the input. We rely on the log-Sobolev inequality that bounds the functional entropy by the functional Fisher information with respect to the covariance of the data. This provides a principled way to measure the amount of information contribution of a subset of features to the decision function. Through extensive experiments, we show that our method surpasses existing interpretability sampling-based methods on various data signals such as image, text, and audio. bibtex
@inproceedings{gat2022functional,
author = {Itai Gat and
Nitay Calderon and
Roi Reichart and
Tamir Hazan},
editor = {Kamalika Chaudhuri and
Stefanie Jegelka and
Le Song and
Csaba Szepesv{\'{a}}ri and
Gang Niu and
Sivan Sabato},
title = {A Functional Information Perspective on Model Interpretation},
booktitle = {International Conference on Machine Learning, {ICML} 2022, 17-23 July
2022, Baltimore, Maryland, {USA}},
series = {Proceedings of Machine Learning Research},
volume = {162},
pages = {7266--7278},
publisher = ,
year = {2022},
url = {https://proceedings.mlr.press/v162/gat22a.html},
timestamp = {Tue, 12 Jul 2022 17:36:52 +0200},
biburl = {https://dblp.org/rec/conf/icml/GatCRH22.bib},
bibsource = {dblp computer science bibliography, https://dblp.org}
}
DoCoGen: Domain Counterfactual Generation for Low Resource Domain Adaptation
Nitay Calderon*, Eyal Ben-David*, Amir Feder and Roi Reichart
*Equal contribution
Published in ACL, 2022
Abstract
Natural language processing (NLP) algorithms have become very successful, but they still struggle when applied to out-of-distribution examples. In this paper we propose a controllable generation approach in order to deal with this domain adaptation (DA) challenge. Given an input text example, our DoCoGen algorithm generates a domain-counterfactual textual example (D-con) - that is similar to the original in all aspects, including the task label, but its domain is changed to a desired one. Importantly, DoCoGen is trained using only unlabeled examples from multiple domains - no NLP task labels or parallel pairs of textual examples and their domain-counterfactuals are required. We show that DoCoGen can generate coherent counterfactuals consisting of multiple sentences. We use the D-cons generated by DoCoGen to augment a sentiment classifier and a multi-label intent classifier in 20 and 78 DA setups, respectively, where source-domain labeled data is scarce. Our model outperforms strong baselines and improves the accuracy of a state-of-the-art unsupervised DA algorithm. bibtex
@inproceedings{calderon2022docogen,
author = {Nitay Calderon and
Eyal Ben{-}David and
Amir Feder and
Roi Reichart},
editor = {Smaranda Muresan and
Preslav Nakov and
Aline Villavicencio},
title = {DoCoGen: Domain Counterfactual Generation for Low Resource Domain
Adaptation},
booktitle = {Proceedings of the 60th Annual Meeting of the Association for Computational
Linguistics (Volume 1: Long Papers), {ACL} 2022, Dublin, Ireland,
May 22-27, 2022},
pages = {7727--7746},
publisher = {Association for Computational Linguistics},
year = {2022},
url = {https://doi.org/10.18653/v1/2022.acl-long.533},
doi = {10.18653/v1/2022.acl-long.533},
timestamp = {Mon, 01 Aug 2022 16:27:51 +0200},
biburl = {https://dblp.org/rec/conf/acl/CalderonBFR22.bib},
bibsource = {dblp computer science bibliography, https://dblp.org}
}
From Limited Annotated Raw Material Data to Quality Production Data: A Case Study in the Milk Industry
Roee Shraga, Gil Katz, Yael Badian, Nitay Calderon and Avigdor Gal
Published in CIKM, 2021
Abstract
Industry 4.0 offers opportunities to combine multiple sensor data sources using IoT technologies for better utilization of raw material in production lines. A common belief that data is readily available (the big data phenomenon), is oftentimes challenged by the need to effectively acquire quality data under severe constraints. In this paper we propose a design methodology, using active learning to enhance learning capabilities, for building a model of production outcome using a constrained amount of raw material training data. The proposed methodology extends existing active learning methods to effectively solve regression-based learning problems and may serve settings where data acquisition requires excessive resources in the physical world. We further suggest a set of qualitative measures to analyze learners performance. The proposed methodology is demonstrated using an actual application in the milk industry, where milk is gathered from multiple small milk farms and brought to a dairy production plant to be processed into cottage cheese. bibtex
@inproceedings{shraga2021limited,
author = {Roee Shraga and
Gil Katz and
Yael Badian and
Nitay Calderon and
Avigdor Gal},
editor = {Gianluca Demartini and
Guido Zuccon and
J. Shane Culpepper and
Zi Huang and
Hanghang Tong},
title = {From Limited Annotated Raw Material Data to Quality Production Data:
{A} Case Study in the Milk Industry},
booktitle = {CIKM '21: The 30th {ACM} International Conference on Information
and Knowledge Management, Virtual Event, Queensland, Australia, November
1 - 5, 2021},
pages = {4114--4124},
publisher = ,
year = {2021},
url = {https://doi.org/10.1145/3459637.3481921},
doi = {10.1145/3459637.3481921},
timestamp = {Tue, 16 Aug 2022 23:04:38 +0200},
biburl = {https://dblp.org/rec/conf/cikm/ShragaKBCG21.bib},
bibsource = {dblp computer science bibliography, https://dblp.org}
}