Analysis of 19.9 million publications from the PubMed/MEDLINE database using artificial intelligence methods: approaches to the generalizations of accumulated data and the phenomenon of “fake news

Introduction. The English-language databases PubMed/MEDLINE and Embase are valuable information resources for finding original publications in basic and clinical medicine. Currently, there are no artificial intelligence systems to evaluate the quality of these publications.

Aim. Development and testing of a system for sentiment analysis (i.e. analysis of emotional modality) of biomedical publications.

Materials and methods. The technique of analysis of the “Big data” of biomedical publications was formulated on the basis of the topological theory of sentiment analysis. Algorithms have been developed that allow for the classification of texts from 16 sentiment classes with 90% accuracy (manipulative speech, research without positive results, propaganda, falsification of results, negative personal attitude, aggressive text, negative emotional background, etc.). Based on the algorithms, a scale for assessing the sentiment quality of research (β-score) is proposed.

Results. Abstracts of 19.9 million publications registered in PubMed/MEDLINE over the past 50 years (1970–2019) were analyzed. It was shown that publications with low sentiment quality (the value of the β-score of the text is less than zero, which corresponds to the prevalence of manipulative and negative sentiments in the text) comprise only 18.5% (3.68 out of 19.9 million). The greatest values of the β-score were characterized by publications on sports medicine, systems biology, nutrition, on the use of applied mathematics and data mining in medicine. The rubrication of the entire array of publications by 27,840 headings (MESH-system of PubMed/MEDLINE) indicated an increase in the β-score by years (i.e., the positive dynamics of sentiment quality of the texts of publications) for 27,090 of the studied headings. The most intense positive dynamics was found for research in genetics, physiology, pharmacology, and gerontology. 249 headings with sharply negative dynamics of sentiment quality and with a pronounced increase in the manipulative sentiments characteristic of the tabloid press were highlighted. Separate assessments of international experts are presented that confirm the patterns identified.

Conclusion. The proposed artificial intelligence system allows a researcher to make an effective assessment of the sentiment quality of biomedical research papers, filtering out potentially inappropriate publications disguised as “evidence-based”.  

1. Canese K., Weis S. PubMed: The Bibliographic Database. 2002 Oct 9 [Updated 2013 Mar 20]. In: The NCBI Handbook. 2nd edition. Bethesda (MD): National Center for Biotechnology Information (US); [Electronic resource] URL: https://www.ncbi.nlm.nih.gov/books/NBK153385/. Accessed: 12.12.2019.

2. Li L., Smith H. E., Atun R., Tudor Car L. Search strategies to identify observational studies in MEDLINE and Embase. Cochrane Database Syst Rev. 2019; MR000041. DOI: https://dx.doi.org/10.1002/14651858.MR000041.pub2.

3. Gromova O. A., Torshin I. Yu. Vitamin D – a paradigm shift. Moscow. 2017; 750.

4. Gromova O. A., Torshin I. Yu. Micronutrients and reproductive health. Guide. Moscow. 2019; 672 c.

5. Stewart Chaplin. The Stained Glass Political Platform. The Century Magazine. USA. 1900.

6. Summers E. Weasel Words: 200 Words You Shouldn’t Trust: 200 Words You Can’t Trust. Chambers (Ed.), Slang & Idiom Dictionaries. 2009; 208 p.

7. Watson D. Watson’s Dictionary of Weasel Words, Contemporary Cliches, Cant and Management Jargon. Knopf, 1st Ed. 2004; 357 p.

8. Torshin I. Y., Rudakov K. V. Combinatorial analysis of the solvability properties of the problems of recognition and completeness of algorithmic models. Part 1: factorization approach. Pattern Recognition and Image Analysis (Advances in Mathematical Theory and Applications). 2017; 27 (1): 16–28.

9. Torshin I. Yu., Rudakov K. V. Combinatorial analysis of the solvability properties of the problems of recognition and completeness of algorithmic models. Part 2: metric approach within the framework of the theory of classification of feature values. Pattern Recognition and Image Analysis (Advances in Mathematical Theory and Applications). 2017; 27 (2): 184–199.

10. Torshin I. Y. Optimal dictionaries of the final information on the basis of the solvability criterion and their applications in bioinformatics. Pattern Recognition and Image Analysis (Advances in Mathematical Theory and Applications). 2013; 23 (2): 319–327.

11. Torshin I. Yu., Rudakov K.V. On the theoretical basis of the metric analysis of poorly formalized problems of recognition and classification. Pattern Recognition and Image Analysis (Advances in Mathematical Theory and Applications). 2015; 25 (4): 577–587.

12. Torshin I. Y., Rudakov K. V. On metric spaces arising during formalization of problems of recognition and classification. Part 1: properties of compactness. Pattern Recognition and Image Analysis (Advances in Mathematical Theory and Applications). 2016; 26 (2): 274.

13. Torshin I. Yu., Rudakov K. V. On metric spaces arising during formalization of problems of recognition and classification. Part 2: density properties. Pattern Recognition and Image Analysis (Advances in Mathematical Theory and Applications). 2016; 26 (3): 483–496.

14. Torshin I.Y., Rudakov K.V. On the application of the combinatorial theory of solvability to the analysis of chemographs. part 1: fundamentals of modern chemical bonding theory and the concept of the chemograph. Pattern Recognition and Image Analysis (Advances in Mathematical Theory and Applications). 2014; 24 (1): 11–23.

15. Torshin I. Y., Rudakov K. V. On the application of the combinatorial theory of solvability to the analysis of chemographs. Part 2: local completeness of invariants of chemographs in view of the combinatorial theory of solvability. Pattern Recognition and Image Analysis (Advances in Mathematical Theory and Applications). 2014; 24 (2): 196–208.

16. Torshin I. Yu., Rudakov K. V. On the Procedures of Generation of Numerical Features Over Partitions of Sets of Objects in the Problem of Predicting Numerical Target Variables. Pattern Recognition and Image Analysis. 2019; 29 (4): 654–667. DOI: https://dx.doi.org/10.1134/S1054661819040175.

17. Chernyshev V. M. Sword Double-edged. Abstract on Sectology. Moscow. 2011; 138 s. (in Russ)

18. Dvorkin A.L. Sectology: Totalitarian sects. Experience in systematic research. 3rd ed., Revised. and add. N. Novgorod. 2014; 816 s. (in Russ)

19. Okter A. Mastermind: The Truth of the British Deep State. Arashtirma Publishing. 2017; 698 pp.

20. Koterov A. N. Causal Criteria in Medical and Biological Disciplines: History, Essenceand Radiation Aspect. Report 1. Problem Statement, Conceptionof Causes and Causation, False Associations. Radiatsionnaya biologiya, radioekologiya (in Russ). 2019; 59 (1): 5–36. DOI: https://dx.doi.org/10.1134/S0869803119010065.

21. Popper K. R. Assumptions and rebuttals: the growth of scientific knowledge. Moscow. 2004.

22. Gromova O. A., Torshin I. Yu., Tetruashvili N. K., Tapil’skaya N.I. A systematic analysis of the effects of molybdenum: the health of the pregnant woman and the fetus. Voprosy ginekologii, akusherstva i perinatologii. 2019; 18 (4): 83–94 (in Russ). DOI: https://dx.doi.org/10.20953/1726-1678-2019-4-83-94.

23. Gromova O. A., Torshin I. Yu., Tetruashvili N. K., Galustyan A. N., Kuritsyna N. A. On prospects for using combinations of folic acid and active folates for the nutritional support of pregnancy. Akusherstvo i ginekologiya. 2019; 4: 87–94 (in Russ). DOI: https://dx.doi.org/10.18565/aig.2019.4.87-94

24. Torshin I.Y., Lila A.M., Gromova O.A., Naumov A.V., Gromov A.N. On the anticoagulant and antiaggregatory properties of a glucosamine sulfate molecule. Modern Rheumatology Journal. 2019; 13 (3): 135–141. (In Russ.) DOI: https://doi.org/10.14412/1996-7012-2019-3-135-141.

25. Gromova O. A., Torshin I. Y., Maximov V. A., Gromov A. N., Rudakov K. V. Systematic analysis of lactitol studies. Experimental and Clinical Gastroenterology. 2019;(2):131-1 42. (In Russ.) DOI: https://doi.org/10.31146/1682-8658-ecg-162-2-131-142.

26. Arnold V., Ilyashenko Yu., Anosov D. et al. Dynamical systems – 1. Results of science and technology. Ser. Lying. prob. mat. Fundam. Directions. Moscow. 260 s. (in Russ)

27. Levenshtein V. I. Binary codes with correction of loss, insertion and substitution of characters. Doklady Akademii Nauk SSSR. 1965; 163 (4): 845–848 (in Russ).

28. Ioannidis J. P.A. Hijacked evidence-based medicine: stay the course and throw the pirates overboard. J Clin Epidemiol. 2017 Apr; 84: 11–13. DOI: https://dx.doi.org/10.1016/j.jclinepi.2017.02.001.

29. Ioannidis J. P. Evidence-based medicine has been hijacked: a report to David Sackett. J Clin Epidemiol. 2016 May; 73: 82–6. DOI: https://dx.doi.org/10.1016/j.jclinepi.2016.02.012.

30. Møller M.H., Ioannidis J.P.A., Darmon M. Are systematic reviews and meta-analyses still useful research? We are not sure. Intensive Care Med. 2018 Apr; 44 (4): 518–520. DOI: https://dx.doi.org/10.1007/s00134-017-5039-y.

31. Cochrane is a registered trademark in Australia, Canada, the European Community and the USA. 2017-09-19. [Electronic resource] URL: trademarks.justia.com/791/85/cochrane-79185910.html. Accessed: 12.12.2019.

32. Torshin I. Y., Gromova O. A., Kobalava Z. D. Concerning the “repression” of ω -3 polyunsaturated fatty acids by adepts of evidencebased medicine. FARMAKOEKONOMIKA. Modern Pharmacoeconomic and Pharmacoepidemiology. 2019; 12 (2): 91–114 (In Russ.) DOI: https://doi.org/10.17749/2070-4909.2019.12.2.91-114.

33. Hannemann A., Wallaschofski H., Nauck M., Marschall P., Flessa S., Grabe H. J., Schmidt C. O., Baumeister S. E. Vitamin D and health care costs: Results from two independent population-based cohort studies. Clin Nutr. 2018 Dec; 37 (6 Pt A): 2149–2155. DOI: https://dx.doi.org/10.1016/j.clnu.2017.10.014.

34. Mekhanik A. G. Artificial intelligence on guard of health. The second conversation with O. A. Thundering and I. Yu. Torshin. Stimul: Zhurnal ob innovatsiyakh v Rossii. 30.10.2019. (in Russ) [Electronic resource] URL: https://stimul.online/articles/science-and-technology/iskusstvennyy-intellekt-na-strazhe-zdorovya-beseda-vtoraya/. Accessed: 12.12.2019.

35. Blinov D.V., Akarachkova E.S., Orlova A.S., Kryukov E.V., Korabelnikov D.I. New framework for the development of clinical guidelines in Russia. FARMAKOEKONOMIKA. Modern Pharmacoeconomic and Pharmacoepidemiology. 2019; 12 (2): 125–144 (In Russ.) DOI: https://doi.org/10.17749/2070-4909.2019.12.2.125-144.

36. Zhuravleva N.I., Shubina L.C., Sukhorukikh O.A. The use of the level of evidence and grade of recommendations scales in developing clinical guidelines in the Russian Federation. FARMAKOEKONOMIKA. Modern Pharmacoeconomic and Pharmacoepidemiology. 2019; 12 (1): 34–41 (In Russ.). DOI: https://doi.org/10.17749/2070-4909.2019.12.1.34-41.

37. Khachatryan G.R., Omelyanovskiy V.V., Melnikova L.S., Ratushnyak S.S. Organizational structure and funding of health technology assessment agencies around the world. FARMAKOEKONOMIKA. Modern Pharmacoeconomic and Pharmacoepidemiology. 2019; 12 (2): 146–154 (In Russ.) DOI: https://doi.org/10.17749/2070-4909.2019.12.2.146-154

38. Lazareva M. L., Tyurina I. V. Financial statistical reporting by medical organizations: shortcomings and areas of optimization. FARMAKOEKONOMIKA. Modern Pharmacoeconomic and Pharmacoepidemiology. 2018; 11 (4): 61–66. (In Russ.) DOI: https://doi.org/10.17749/2070-4909.2018.11.4.061-066.

39. Omelyanovsky V. V., Fedyaeva V. K., Musina N. Z. The concept of multi-criteria analysis of decision-making in the current system of health technology assessment in Russia. FARMAKOEKONOMIKA. Modern Pharmacoeconomic and Pharmacoepidemiology. 2018; 11 (3): 3–7 (In Russ.) DOI: https://doi.org/10.17749/2070-4909.2018.11.3-003-007

40. Khrustalev M.B., Maksimova A.A. Effective search for potentially innovative scientific results in medicine. FARMAKOEKONOMIKA. Modern Pharmacoeconomic and Pharmacoepidemiology. 2019; 12 (1): 27–33. (In Russ.) DOI: https://doi.org/10.17749/2070-4909.2019.12.1.27-33

41. Musina N.Z., Fedyaeva V.K., Omel’yanovskii V.V., Khachatryan G.R., Gerasimova K. V., Lemeshko V. A., Konchits K. P. Review of the current approaches to the assessment of the drug innovative potential worldwide. FARMAKOEKONOMIKA. Modern Pharmacoeconomic and Pharmacoepidemiology. 2017; 10 (3): 66–74. (In Russ.) DOI: https://doi.org/10.17749/2070-4909.2017.10.3.066-074.