Large Language Model Evaluation for Domain-Specific Scientific Information Retrieval

Amélie De Smet

Large Language Model Evaluation for Domain-Specific Scientific Information Retrieval

Author(s)	Amélie De Smet
Country	Belgium
Abstract	Scientific information retrieval plays an essential role in supporting academic research, evidence discovery, knowledge synthesis, and the identification of relevant scholarly publications. Conventional search systems frequently employ lexical matching and statistical ranking methods to identify documents associated with user queries. Although these approaches provide established mechanisms for locating scientific information, they may encounter difficulties when researchers use specialized terminology, interdisciplinary expressions, abbreviations, or conceptual questions that differ from the wording of indexed publications. Advances in natural language processing and large language models have introduced additional opportunities for improving query interpretation, semantic document representation, and relevance ranking. The present paper develops a hypothetical methodological framework for evaluating large language model-assisted scientific information retrieval across multiple research domains. Four retrieval configurations are considered: BM25 lexical retrieval, scientific-domain dense retrieval, large language model-based query expansion with BM25, and hybrid retrieval with large language model reranking. A fictional corpus containing 3,200 scientific document abstracts and 240 research queries is introduced to demonstrate the proposed comparative design. The represented disciplines include biomedical sciences, materials science, environmental sciences, and computer science. The methodology incorporates scientific corpus preparation, query construction, relevance assessment, model configuration, ranking evaluation, and evidence-grounding analysis. Normalized Discounted Cumulative Gain, Recall, and Mean Reciprocal Rank are selected as principal retrieval measures. All numerical values are manually constructed and do not represent executed retrieval experiments or verified model outputs. The methodological discussion examines domain-specific terminology, semantic relevance, cross-disciplinary retrieval, computational costs, model generalization, and the reliability of generated scientific explanations. Particular attention is given to distinguishing document retrieval effectiveness from the factual accuracy of language-model-generated responses. The paper concludes that large language models offer a relevant methodological direction for scientific information retrieval, although their practical effectiveness requires reproducible experiments, independent relevance judgments, transparent evaluation, and careful verification of cited scientific evidence.
Keywords	Large Language Models, Scientific Information Retrieval, Domain-Specific Search, Semantic Retrieval, Natural Language Processing, Scientific Document Ranking, Retrieval-Augmented Generation, Query Expansion, Relevance Evaluation, Scholarly Information Systems.
Field	Engineering
Published In	Volume 2, Issue 3, May-June 2026
Published On	2026-05-18

View / Download PDF File

E-ISSN: 3067-5073

doi

CrossRef DOI is assigned to each research paper published in our journal.

IJMLAIDSE DOI prefix is
10.00000/IJMLAIDSE

Downloads

Research Paper Format Copyright Permission and Undertaking Form

All research papers published on this website are licensed under Creative Commons Attribution-ShareAlike 4.0 International License, and all rights belong to their respective authors/researchers.

CC-BY-SA

About IJMLAIDSE Fees & Payment Current Issue Publication Archive	Submit Research Paper Track Submission Status Publication Guidelines Publication Ethics Peer Review & Plagiarism	Join as a Reviewer Editors & Reviewers Get Reviewer Membership Certi.	Website/Journal Policies Usage Policy Content Policies Privacy Policy

Contact Us	+1-872-302-3851	editor@uscse.com

International Journal of Machine Learning, AI & Data Science Evolution

A Widely Indexed Open Access Peer Reviewed Multidisciplinary Monthly Scholarly International Journal

Large Language Model Evaluation for Domain-Specific Scientific Information Retrieval

Share this