Skip to content

perf: Speed up read_graph() and write_graph() for ~ paths, URLs and connections - #3011

Open
ptweezy wants to merge 1 commit into
igraph:mainfrom
ptweezy:perf-read-graph-toraw
Open

ptweezy wants to merge 1 commit into
igraph:mainfrom
ptweezy:perf-read-graph-toraw

Conversation

@ptweezy

@ptweezy ptweezy commented Nov 8, 2026

Copy link
Copy Markdown

When the path contains ~ or ://, or file is a connection, read_graph() and write_graph() copy the file with the internal read.graph.toraw(), as do max_cliques(file = ) and graph_from_graphdb(). That function grew its result with c() once per 20000-byte chunk. Each c() copies everything read so far, so the copy took quadratic time in the file size.

This PR collects the chunks in a list and calls unlist() once. The readBin() calls and the stop condition are unchanged, so the result is the same; an empty file still gives raw(0).

Timings for a 32.8 MB GraphML file (Windows 11, R 4.6.2, median of 5 runs, main and this PR alternating, fresh Rscript each):

Operation main this PR Speedup
read.graph.toraw() alone 25.35 s 0.078 s 322x
read_graph("<dir>/g~.graphml", "graphml") 26.51 s 0.71 s 37x
write_graph(g, "<dir>/out~.graphml", "graphml") 32.65 s 0.99 s 33x
read_graph("<dir>/g.graphml", "graphml"), control 0.47 s 0.47 s 1.0x

The copy speedup is 24x at 2 MB and 85x at 8 MB.

New tests in test-foreign.R read files of 0 to 123457 bytes through a path and through file and raw connections, and check that ~ paths and connections give the same graphs and files as a plain path. They also pass on main. The full test suite passes on both, and air format changes nothing.

Worked with Claude to write and test the patch for this issue.

  • By submitting this pull request, I assign the copyright of my contribution to The igraph development team.

… and connections

For these inputs both functions copy the file with the internal
`read.graph.toraw()`, which grew its result with `c()` once per
20000-byte chunk. Each `c()` copies everything read so far, so the copy
took quadratic time in the file size.

Collect the chunks in a list and call `unlist()` once. The `readBin()`
calls and the stop condition are unchanged, so the result is the same.

Add tests for chunk boundaries and for `~` paths and connections.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant