In an era of smooth-talking chatbots, how do researchers find trustworthy answers?

How many stars are there in the universe? Before the internet, finding the answer might have meant a trip to the library. Now, a modern search engine can provide an answer instantly.

While search engines have long been essential for navigating the internet, AI is starting to change online search. A recent study found that 40% of U.S.-based Google search results now include an AI-generated overview — with sometimes strange results. Last year, one AI summary said there were “at least 1,024” stars in the universe, which, while technically correct, is not the type of answer that should be included in a research paper.

APS recently supported two surveys aimed at understanding how AI is changing how scholars find information online and how those changes might impact researchers, publishers, and science as a whole. Below are key takeaways, along with insights from APS editors and staff on the implications for the future of research.

While most researchers still turn to traditional search engines when they begin their research discovery process, AI is poised for growth — but carries some pitfalls.

APS co-sponsored a survey, administered by the science research firm Kudos, of more than 10,000 researchers on how they search for scientific literature online.

Nearly 70% of respondents said that they use a search engine every time or most of the time when looking for information online. When they begin their search, around 25% of respondents reported using a search engine where AI-generated summaries are often embedded, while 10% of respondents turn directly to AI. Early-career researchers and researchers in industry were twice as likely as other respondents to begin searching for answers using a chatbot.

On APS websites, a significant percentage of page views is dominated by bots, said Jakrey Myers, the Society’s director of information systems. He said this signals a major change in how researchers are retrieving information and highlights the need for metrics that can track these visits and help publishers determine a study’s impact beyond downloads and page views.

One issue with AI-based information retrieval is a lack of process transparency and repeatability. A recent study found that different AI tools tend to cite different sources, with Claude more likely to link to resources from PubMed or ScienceDirect and Gemini turning to Reddit and YouTube. A phenomenon known as “citation drift” can also occur, where up to half of the sources cited will change each time a question is searched for.

APS’ Head of Ethics and Research Integrity Daniel Ucko noted another risk: A user might perceive content generated by AI as neutral, when in reality “there is no such thing as a neutral technology,” he said. “These [AI] models are made by people, and people will inject their own biases and preferences and prejudices into the process, so [these tools aren’t] neutral by any stretch of the imagination,” he added.

Researchers are aware of the AI-generated ‘knowledge panels’ that appear at the top of search results but are not as well-versed in where they come from — and they don’t always verify a summary’s accuracy.

Nearly 85% of respondents said they have noticed summary panels at the top of search results, but only around 30% of respondents knew this content was entirely AI-generated. Around 20% thought they were direct extracts from websites, while 40% thought there was some level of human curation involved. Students were less likely to know that the panels were AI-generated and more likely to think that they were human-moderated.

This lack of AI fluency is concerning, said Myers. While these tools can create content that may seem accurate at first glance, what they generate is “susceptible to many of the same issues that we have as humans,” he said — and can, like plenty of internet content, be false. He emphasized that users “really have to be certain of the source and the provenance of that information.”

These results also highlight how even the most astute, highly trained scholars are not immune to AI-generated misinformation or AI-related hype, said Andrew Larkoski, an associate editor with Physical Review D. “We are told that AI can solve these big problems,” he said, referring to the Navier-Stokes problem that made recent headlines. “And for people who already solve big problems — like studying the Standard Model — they might want to recapture some of that, and they can see the AI as a tool that can help them do that.”

Around 75% of the respondents said that when they notice a source linked in an AI-generated summary, they regularly click through — more than half of the time to verify a statement’s accuracy or cite the original source. But if the goal is to get a general overview of a topic, 58% of respondents said they would just rely on the summary.

APS members are frequently using AI, including for research discovery, but are concerned about its potential impacts.

In a separate survey, APS asked more than 1,100 researchers how they use AI in their work. Literature discovery was the most popular use; 86% of respondents said they used AI to identify relevant papers, and 31% said they used AI to generate summaries or overviews of the current literature.

APS journals’ recently updated AI policy now specifies how AI used during the research, manuscript preparation, or peer review should be disclosed. The current guidance stipulates that AI can’t be an author on a paper, Ucko explained, because of AI’s lack of transparency and accountability. “It's important that [a researcher] can stand behind what’s being created,” he said.

Researchers also remain worried about the potential impacts of AI on science as a whole. In the survey of APS members, 30% of respondents said that they were concerned that an overreliance on these tools could lead to a “de-skilling” of researchers.

To this end, Ucko cautioned against anyone outsourcing their critical thinking or judgment while using these tools — to “keep the human not just in the loop, but in the driving seat,” he said.

Larkoski also voiced concerns about AI’s tendency toward uniformity of styles and ideas. This can make it hard for researchers to stand out, especially early in their careers. “The point of science is not to just be another cog in the wheel,” he said. “It is to be an individual who has unique ideas and thoughts and who sees the world in a different way than everyone else — and that enriches the whole ecosystem.”

Unlike finding out how many stars are in the universe (around one septillion, in case you’re curious), there’s no quick answer for how AI will impact research discovery moving forward.

But perhaps, Myers said, if the scientific community comes together to better understand AI’s benefits and pitfalls, it can collectively figure out how to make this tool more useful for everyone. “That's what science does,” he said. “We try to understand something, and then figure out how to best utilize it.”