Few-Shot Sentiment Classification via Prompt Engineering and Contrastive Learning in Large Language Models
Keywords:
few-shot learning, sentiment analysis, large language models, prompt engineering, contrastive learning, system design, fairness, sustainability, governanceAbstract
The emergence of large language models has reshaped sentiment classification by enabling competitive performance with only a handful of labeled examples. In this regime, prompt engineering and contrastive learning stand out as complementary strategies that can jointly improve accuracy, robustness, and interpretability. This paper offers a system-level analysis of few-shot sentiment classification that integrates prompt-based interfaces with contrastive representation learning. We examine design choices across prompt formulation, contrastive objective selection, and model calibration, while situating these technical decisions within broader concerns of infrastructure cost, fairness, sustainability, and governance. The discussion reveals fundamental trade-offs between adaptation flexibility and inference latency, between representation invariance and domain specificity, and between the benefits of pre-trained knowledge and the risks of encoded societal biases. By framing few-shot sentiment classification as a socio-technical pipeline, we highlight how architecture selection, data curation, and deployment strategy collectively shape outcomes in high-stakes applications such as content moderation, public health surveillance, and financial news analysis. The paper further addresses policy implications related to transparency, auditability, and the carbon footprint of iterative prompt search. Rather than proposing a single optimal recipe, we argue for a layered design methodology that harmonizes prompt templates, contrastive fine-tuning objectives, and post-hoc calibration mechanisms while accounting for downstream governance requirements. We conclude with a roadmap for future research that aligns technical innovation with responsible deployment.
References
1. Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 4171–4186.
2. Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., & Sutskever, I. (2019). Language models are unsupervised multitask learners. OpenAI Blog, 1(8), 9.
3. Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., ... & Amodei, D. (2020). Language models are few-shot learners. Advances in Neural Information Processing Systems, 33, 1877–1901.
4. Schick, T., & Schütze, H. (2021). Exploiting cloze questions for few-shot text classification and natural language inference. Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, 255–269.
5. Gao, T., Yao, X., & Chen, D. (2021). SimCSE: Simple contrastive learning of sentence embeddings. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 6894–6910.
6. Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2023). Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys, 55(9), 1–35.
7. Khosla, P., Teterwak, P., Wang, C., Sarna, A., Tian, Y., Isola, P., ... & Krishnan, D. (2020). Supervised contrastive learning. Advances in Neural Information Processing Systems, 33, 18661–18673.
8. Wei, J., Bosma, M., Zhao, V. Y., Guu, K., Yu, A. W., Lester, B., ... & Berant, J. (2022). Finetuned language models are zero-shot learners. International Conference on Learning Representations.
9. Wang, X., Yu, Z., & He, X. (2023). Sentiment analysis in the era of large language models: A survey. ACM Computing Surveys, 56(6), 1–38.
10. Min, S., Lewis, M., Zettlemoyer, L., & Hajishirzi, H. (2021). MetaICL: Learning to learn in context. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2791–2805.
11. Li, Q. (2026). Dynamic Adaptive Attention and Supervised Contrastive Learning: A Novel Hybrid Framework for Text Sentiment Classification. arXiv preprint arXiv:2604.10459.
12. Gunel, B., Du, J., Conneau, A., & Stoyanov, V. (2021). Supervised contrastive learning for pre-trained language model fine-tuning. International Conference on Learning Representations.
13. Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., ... & Lowe, R. (2022). Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35, 27730–27744.
14. Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021). On the dangers of stochastic parrots: Can language models be too big? Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 610–623.
15. Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., ... & Liang, P. (2021). On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258.
16. Zhao, T., Yu, H., & Weng, R. (2022). Calibrate before use: Improving few-shot performance of language models. International Conference on Machine Learning, 12697–12706.
17. Yin, W., Hay, J., & Roth, D. (2019). Benchmarking zero-shot text classification: Datasets, evaluation and entailment approach. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, 3914–3923.
18. Desai, S., & Durrett, G. (2020). Calibration of pre-trained transformers. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, 295–302.
19. Gururangan, S., Marasović, A., Swayamdipta, S., Lo, K., Beltagy, I., Downey, D., & Smith, N. A. (2020). Don't stop pretraining: Adapt language models to domains and tasks. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 8342–8360.
20. Zhang, Z., & Sabuncu, M. R. (2018). Generalized cross entropy loss for training deep neural networks with noisy labels. Advances in Neural Information Processing Systems, 31.
21. Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017). Grad-CAM: Visual explanations from deep networks via gradient-based localization. Proceedings of the IEEE International Conference on Computer Vision, 618–626.
22. Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). "Why should I trust you?" Explaining the predictions of any classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 1135–1144.
23. Rosenthal, S., Farra, N., & Nakov, P. (2017). SemEval-2017 task 4: Sentiment analysis in Twitter. Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), 502–518.
24. Dodge, J., Ilharco, G., Schwartz, R., Farhadi, A., Hajishirzi, H., & Smith, N. A. (2022). Fine-tuning pretrained language models: Weight initializations, data orders, and early stopping. arXiv preprint arXiv:2002.06305.
25. Strubell, E., Ganesh, A., & McCallum, A. (2019). Energy and policy considerations for deep learning in NLP. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 3645–3650.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Journal of Advanced Artificial Intelligence Research
This work is licensed under a Creative Commons Attribution 4.0 International License.
This article is published under the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
