Can LLMs Recommend More Responsible Prompts?

Human-Computer Interaction practitioners have been proposing best practices in user interface design for decades. However, generative Artificial Intelligence (GenAI) brings additional design considerations and currently lacks sufficient user guidance regarding affordances, inputs, and outputs. In this context, we developed a recommender system to promote responsible AI (RAI) practices while people prompt GenAI systems, by recommending addition of sentences based on social values and removal of harmful sentences. We detail a lightweight recommender system designed to be used in prompting-time and compare its recommendations to the ones provided by three base large language models (LLMs) and two LLMs fine-tuned for the task, i.e., recommending inclusion of sentences based on social values and removal of harmful sentences from a given prompt. Results indicate that our approach has the best F1-score balance in terms of recommendations for additions and removal of sentences (0.591 and 0.500, respectively), while a fine-tuned merlinite-7b-lab-Q4_K_M model obtained the best F1-score for additions (0.933) and our approach obtained the best F1-score for removals of harmful sentences (0.500). In addition, fine-tuned models improved the objectiveness of responses by reducing the verbosity of generated content in 93% when compared to the content generated by base models. While our approach provided responses of 28.83 words on average, a fine-tuned merlinite-7b-lab-Q4_K_M model delivered responses of 36.08 words on average, and the more verbose responses were from llama-3-8b-instruct, which provided recommendations of 1138.95 words on average. Presented findings contribute to RAI by showing the limits and bias of existing LLMs in terms of recommendations on how to create more responsible prompts and how open-source technologies can fill this gap in prompting-time.