Artificial intelligence in information retrieval

AI has been used as a tool in databases and search engines, as well as in the algorithms that control their operations, for a long time. The development hasn’t been linear; sometimes it has progressed quickly, sometimes there is quieter.

Language models use the law of averages

At present, AI tools are taken notice again, especially the Large Language Models (LLM), which are utilized e.g. by ChatGPT. The language models are based on the probability of consecutive occurrence of words. Thus, a language model itself is not a search engine.

Applications utilizing language models can be used to help you search for information: it can be used to get to know the topic you are looking for and it can be used to for asking suitable keywords and even queries for databases. Some AI applications, like Copilot, are suitable for information retrieval, too. The answer is not always the same, even if you question is exactly the same. It is always up to you to evaluate the validity of answers.

Be careful with data privacy and security! Generative AI applications should not be given sensitive or confidential information.

Generative AI can suggest search terms

You can ask AI for suitable search terms for your topic. If the suggested search terms seem very general or obvious, ask again for more specific words.

Often, instead of individual words, you will receive longer phrases as an answer. When you edit these phrases for purposes of search queries, break down the phrases into separate terms and add operators between them.

Formulating search queries is partially successful

Some applications can also formulate search queries suitable for databases when asked. The operators and phrases usually are marked correctly (not always!), but word truncation is missing. Additionally, individual search terms are often enclosed in quotation marks, which generally weakens the functionality of automatically find different forms or spellings of words.

AI in databases and information retrieval

Databases use various algorithms and machine learning methods for collecting, presenting, and recommending new results.

Traditional databases now also include generative AI tools based on language models. In the Web of Science database, the AI tool is called Research Assistant. A similar tool will be available in UEF-Primo in November 2024. In Scopus database the tool is named Scopus AI. The database’s AI tool uses real articles as its sources, which it retrieves from the database itself. Based on the articles, the program creates a summary of the topic. However, the search and the resulting summary are only a quick overview of the topic, not a thorough or exhaustive answer. The search can easily be continued with more specific questions though.

Many separate AI applications can already search for real sources on which they base their answers. The search can target the open web or specifically only openly published scientific articles, like the Elicit application.

Read more about the topic on the library’s homepage: AI in information retrieval. The page provides more details on which applications and how to use them in information retrieval.

Next page: Reliability of results