Artificial intelligence in information retrieval

AI has been used as a tool in databases and search engines, as well as in the algorithms that control their operations, for a long time. The development hasn’t been linear; sometimes it has progressed quickly, sometimes there is quieter. Even failures happen from time to time; some product just doesn’t gain the trust of users.

Language models use the law of averages

At present, AI tools are taken notice again, especially the Large Language Models (LLM), which are utilized e.g. by ChatGPT. The language models themselves do not search for information online or in databases, but are based on the probability of consecutive occurrence of words. Therefore, as such, they are not suitable for information retrieval. All the answers the system should always be checked somewhere else. Real and correct references to scientific publications ChatGPT cannot provide at all.

However, language models can be used to help you search for information: it can be used to get to know the topic you are looking for and it can even be used to ask for help as suitable keywords. Sometimes the program does this better, sometimes it does it worse. It is up to you to evaluate the validity of suggestions.

AI in databases

The reference databases use various algorithms and means of machine learning to collect, present and recommend new results, among other things. New functions based on language models are now also coming to traditional databases. These will be seen already in late 2023 and more in 2024.

Next page: Reliability of results