Skip to content

compiler: Schedule the readers of a waited-on Function after the wait - #3037

Open
mloubout wants to merge 1 commit into
mainfrom
fix-stream-wait-first-reader
Open

mloubout wants to merge 1 commit into
mainfrom
fix-stream-wait-first-reader

Conversation

@mloubout

Copy link
Copy Markdown
Contributor

No description provided.

@mloubout mloubout added GPU compiler bug-C bug in the generated code labels Sep 29, 2026
@codecov

codecov Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.29%. Comparing base (6d2ad82) to head (d63ddf3).
⚠️ Report is 15 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #3037   +/-   ##
=======================================
  Coverage   84.28%   84.29%           
=======================================
  Files         258      258           
  Lines       57469    57487   +18     
  Branches     4914     4912    -2     
=======================================
+ Hits        48439    48459   +20     
+ Misses       8206     8205    -1     
+ Partials      824      823    -1     
Flag Coverage Δ
pytest-gpu-aomp-amdgpuX 68.47% <56.25%> (-0.02%) ⬇️
pytest-gpu-gcc- 78.93% <100.00%> (+<0.01%) ⬆️
pytest-gpu-icx- 78.86% <100.00%> (+<0.01%) ⬆️
pytest-gpu-nvc-nvidiaX 69.17% <56.25%> (-0.03%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@mloubout mloubout changed the title compiler: Make every reader of a prefetched buffer wait for it compiler: Schedule the readers of a waited-on Function after the wait Sep 30, 2026
Comment thread devito/passes/clusters/fusion.py Outdated

@FabioLuporini FabioLuporini left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be handled by Scope, not with an ad-hoc solution in here

it's plausible we have to revisit the way we construct it: https://github.com/devitocodes/devito/blob/main/devito/ir/clusters/cluster.py#L159-L160
e.g. by explictly supply the syncs as a sequence of dummy equations or, probably simpler, by explicitly passing extra Scope(..., reads=...)

@mloubout
mloubout force-pushed the fix-stream-wait-first-reader branch from 751250e to ac60b1c Compare October 26, 2026 13:48
@mloubout

Copy link
Copy Markdown
Contributor Author

Moved to Scope

Comment thread devito/ir/support/basic.py Outdated
# Any access to a Function may depend on a wait on it
for i, e in enumerate(self.exprs):
for f in {j.function for j in e.reads_explicit}:
yield TimedAccess(mocksym_wait(f), 'R', i, e.ispace)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do you need a mock symbol and not just f, or (should standalone f be problematic) f.indexed ?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because this is an access on the function, but without dimensions (a plain "wait on f"). If you use f or f.iindexed then you introduce dimensions and it will break most cases

Comment thread devito/ir/support/basic.py Outdated
yield TimedAccess(mocksym_wait(f), 'W', 0, self.exprs[0].ispace)

@cached_property
def waited(self):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think Scope should see synchronization operations as it's a different level of abstraction. Instead, these should be pulled in Cluster/ClusterGroup, and then fed to Scope as reads/writes or , if necessary (I don't think so) mock expressions

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It can't work both way. It's either in the Scope and it needs to know the syncs or it needs to be back to what it was and filtered in the clusters.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it still in the Scope; but it's rather passed to Scope as further read/written Functions

in ir/clusters/cluster.py , look for def scope and change its implementation to also include such syncs , that's basically what I was suggesting

@mloubout
mloubout force-pushed the fix-stream-wait-first-reader branch 4 times, most recently from 58a6bcf to 2176a13 Compare October 26, 2026 16:28
Comment thread devito/ir/clusters/cluster.py Outdated
TimedAccesses, to be fed to a Scope.
"""
waited = {s.target for s in flatten(syncs.values()) if isinstance(s, WaitLock)}
writes = [TimedAccess(mocksym_wait(f.name), 'W', 0, exprs[0].ispace)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not just f.indexed?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably safer to pass in self instead of exprs/syncs, and rather use self.ispace (ie cluster.ispace) instead of exprs[0].ispace

we may also consider calling it extract_accesses instead of wait_accesses since in the future we might need generalisations

Comment thread devito/ir/clusters/cluster.py Outdated
TimedAccesses, to be fed to a Scope.
"""
waited = {s.target for s in flatten(syncs.values()) if isinstance(s, WaitLock)}
writes = [TimedAccess(mocksym_wait(f.name), 'W', 0, exprs[0].ispace)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably safer to pass in self instead of exprs/syncs, and rather use self.ispace (ie cluster.ispace) instead of exprs[0].ispace

we may also consider calling it extract_accesses instead of wait_accesses since in the future we might need generalisations

Comment thread devito/ir/clusters/cluster.py Outdated
waited = {s.target for s in flatten(syncs.values()) if isinstance(s, WaitLock)}
writes = [TimedAccess(mocksym_wait(f.name), 'W', 0, exprs[0].ispace)
for f in waited if exprs]
reads = [TimedAccess(mocksym_wait(f.name), 'R', i, e.ispace)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

again, f.indexed should suffice

@mloubout mloubout closed this Oct 26, 2026
@mloubout
mloubout force-pushed the fix-stream-wait-first-reader branch from 2176a13 to 6d2ad82 Compare October 26, 2026 17:14
Fusion's toposort orders ClusterGroups by the data dependences their Scopes
report, and a WaitLock, e.g. for a prefetch to complete, induced none. A
reader of the prefetched buffer could then be scheduled ahead of the Cluster
holding the wait, e.g. to fuse with a Cluster with the same guards, and read
the previous snapshot: under the CUDA backend, the mass term of a
reflection-FWI adjoint ran before the wait while its halo readers ran after.

A Cluster(Group) now feeds its Scope the accesses of its syncs: a WaitLock is
an index-free write of its target, on which every access to the target
depends. The `reads` and `writes` of a Scope are thus accesses on top of
those of its expressions.
@mloubout mloubout reopened this Oct 26, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug-C bug in the generated code compiler GPU

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants