Researchers in Singapore and China revealed on October 11 that OpenAI's developer tools struggle to replicate the exact reference behavior seen in the consumer chatbot version. The academic study evaluated whether standard search infrastructure and current developer endpoints can independently match the specific web citations generated by advanced language models.
The study specifically tested the GPT-5.5 Instant configuration to understand how underlying retrieval mechanisms operate. Investigators discovered that combining standard search engine results with the developer-facing Responses API only successfully uncovered a fraction of the original web references.
The joint research team from the National University of Singapore and Xi’an Jiaotong University analyzed the exact pathways artificial intelligence models use to source information. Their experiments showed that algorithmic pipelines do not always mirror the conversational interface users experience daily.
When researchers combined the output from web query engines with developer endpoints, the setup successfully located only 26.2 percent of the exact destination pages previously cited by the chatbot. On a broader domain level, the combined tools performed slightly better by identifying 48.7 percent of the parent websites behind those specific references.
The observed discrepancy highlights a significant gap between consumer-facing applications and raw developer infrastructure. Software developers relying on backend endpoints may receive different reference patterns compared to end-users interacting with the primary web interface. This variance complicates efforts to audit, verify, or duplicate artificial intelligence research that depends on transparent sourcing.
What did the recent academic paper investigate? Researchers tested whether standard search results and OpenAI developer tools could successfully replicate the exact web citations generated by the chatbot's GPT-5.5 Instant setting.
How accurate were the developer tools at finding cited pages? Combined search and API tools only located 26.2 percent of the exact pages cited and 48.7 percent of the parent websites behind those references.