Almost everywhere you go on the internet, it seems almost impossible to avoid AI-related articles. We even published a few at UT. They usually focus on how a particular research group uses technology to make sense of datasets. But this kind of pattern recognition isn’t the only thing AI is good at. In fact, he becomes quite skilled at abstract thinking. One of the places where abstract thinking can be useful is in the development of new scientific theories. With this in mind, a team of researchers from the European Space Agency, Columbia and the Australian National University (ANU) used artificial intelligence to generate scientific hypotheses in astronomy.

They did this particularly in the subfield of Galactic Astronomy, which specializes in research surrounding the formation and physics of galaxies. A recent article on arXiv mentions that they chose this subfield because of its “integrative nature”, which requires “knowledge from various subfields”.

This seems like something AI is already good at. However, a standard major language model (LLM) such as those that have become popular recently (ChatGPT, Bard, etc.) will not have sufficient subject knowledge to develop valid hypotheses in this area. It may even be the victim of “hallucinations,” which some researchers (and journalists) warn about being one of the downsides of interacting with models.

This version of Fraser’s Question Show is partly about artificial intelligence.

To avoid this problem, researchers led by Ioana Chuk? and Yuan-Seng Ting of ANU used a piece of code written in Python known as an application programming interface (API) known as Langchain. This API allows more advanced users to use LLMs such as GPT-4, the newest framework for ChatGPT. In the researchers’ case, they downloaded more than 1,000 scientific papers on galactic astronomy from NASA’s Astrophysics Data System and then uploaded them to GPT-4.

One of the researchers’ experiments was to test how the number of documents the model had access to affected the proposed hypotheses. They noticed a significant difference between accessing only ten documents and accessing a full thousand documents and the proposed hypotheses they developed.

But how did they judge the probability of the hypotheses themselves? They did what any self-respecting scientist would do, and they brought in experts. Or rather, two. And they asked them to come up with hypotheses based on the originality of thought, the feasibility of testing hypotheses, and the scientific correctness of their basis. The experts found that even with a limited dataset of only ten articles, the hypotheses they called their model “Astro-GPT” scored only slightly lower than a competent doctoral degree. student. Astro-GPT, which provides access to all 1,000 articles, has reached the “near expert level” level.

Here’s a NASA scientist’s brief account of how artificial intelligence will take astronomy to the next level. Credits: Science Museum, Boston YouTube Channel

A critical factor in determining the final hypotheses submitted to the experts was the refinement of the hypotheses through a “competitive claim”. While this may sound aggressive, it simply means that in addition to the program that developed the hypotheses, another program is trained on the same dataset and then gives feedback to the first program about its hypotheses, thereby forcing the original program to correct its logical errors. often produces much better ideas.

Even with conflicting assessments, there’s no reason to be an astronomy candidate. students’ refusal to present their own unique ideas in their field. However, this study points out that these LLMs are underutilized. As they become more common, scientists and lay people can increasingly use them to come up with new and better ideas to test. Source