Skip to content

Science: New arXiv tools: Search & Load PDF #72

Open

Description

@SommerEngineering

Idea

For scientific research, the AI should be able to search arXiv for suitable sources. Once it has found a source, it should be able to fetch and read the PDF.

Tools

  • Search (arxiv_search): Searches arXiv and returns for each hit the arXiv ID, title, authors, abstract, categories, dates, and, when available, the DOI and journal reference.
  • Load PDF (arxiv_load_pdf): Loads a paper by its arXiv ID and hands its text to the model.

API

arXiv offers a public API, which we tested on 2026-10-03:

  • Endpoint https://export.arxiv.org/api/query, no API key needed, response as Atom XML.
  • Search via search_query with field prefixes (ti, au, abs, cat, all) and Boolean operators, paging via start and max_results, sorting by relevance or date.
  • Every entry links to its PDF (https://arxiv.org/pdf/<id>v<version>). Many newer papers are also available as HTML (https://arxiv.org/html/<id>), which is easier to turn into text. Load PDF could prefer the HTML version and fall back to the PDF.

Requirements

  • Rate limit: The terms of use allow at most one request every three seconds and a single connection at a time. Both tools need a shared throttle, also when the model calls them in parallel.
  • Copyright: Metadata such as titles and abstracts is CC0, but the papers remain under the copyright of their authors. We may load them for the user, but we must not host or redistribute them.
  • No endorsement: We must not present AI Studio as endorsed or supported by arXiv.
  • Prompt injection: The loaded text is untrusted content and has to pass the prompt injection check like every other tool result.
  • Long papers: A paper can exceed the context window, so Load PDF needs a way to page through the text. The PDF is streamed and never held in memory as a whole.

Related

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    • Status
      Accepted

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions