Identifying single-item faked responses in personality tests: A new TF-IDF-based method

Abstract

Faking in a psychological test is often observed whenever an examinee may gain an advantage from it. Although techniques are available to identify a faker, they cannot identify the specific questions distorted by faking. This work evaluates the effectiveness of term frequency- inverse document frequency (TF-IDF)-an information retrieval mathematical tool used in search engines and language representations-in identifying single-item faked responses. We validated the technique on three datasets containing responses to the 10- item Big Five questionnaire (total of 694 participants, respectively 221, 243, and 230) in three faking situations. Each participant responded twice, once faking to achieve an objective in one of three contexts (one to obtain child custody and two to land a job) and once honestly. The proposed TF-IDF model has proven very effective in separating honest from dishonest responses-with the honest ones having low TF-IDF values and the dishonest ones having higher values-and in identifying which of the 10 responses to the questionnaire were distorted in the dishonest condition. We also provide examples of the technique in a single-case evaluation.