Skip to content

tickets/SP-3223: Add extinction and cloud stacker - #523

Draft
ehneilsen wants to merge 3 commits into
mainfrom
tickets/SP-3223
Draft

ehneilsen wants to merge 3 commits into
mainfrom
tickets/SP-3223

Conversation

@ehneilsen

Copy link
Copy Markdown
Collaborator

Two stackers are added to maf here, intended to be applied to visit sequences from consdb queries rather than opsim output.

The first uses sklearn robust linear fitting tools to fit for photometric extinction and zero point given the the zero_point_median, band, and airmass columns from consdb.
The next uses the results of this and zero_point_median to estimate the extinction from clouds, falling back on rubin_sim.phot_utils.predicted_zeropoint if there was no good fit. It supports an optional offset, so you can (and I probably will) feed in rubin_nights.reference_values.ZEROPOINT_OFFSETS_LSSTCAM, which if I understand what's going on will replicate what rubin_sim_addons does.

This was developed with a mixture of hand-coding and claude code assistance using the experimental support of claude code at the USDF.

@ehneilsen
ehneilsen requested a review from rhiannonlynne June 9, 2026 21:25
@ehneilsen ehneilsen changed the title tickets/SP-3223: Add extinction stacker tickets/SP-3223: Add extinction and cloud stacker Jun 9, 2026
@ehneilsen
ehneilsen marked this pull request as draft June 12, 2026 15:59

@rhiannonlynne rhiannonlynne left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ugh, realized I had some comments that have been stuck forever in pending.
I also realize this is still a draft.
Also, I suppose I don't remember the details here anymore (X-1) vs (X) vs (1-X)..

# The extinction coefficient is independent of exptime.
zp_at_X1 = predicted_zeropoint(band, airmass=1.0, exptime=1.0)
zp_at_X2 = predicted_zeropoint(band, airmass=2.0, exptime=1.0)
k_predicted = zp_at_X1 - zp_at_X2 # positive extinction coefficient

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of this, we should just make the extinction coefficients available from the same phot_utils module as predicted_zeropoint.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not the same k value as in predicted_zeropoint, although the absolute value should be the same.

k_predicted = zp_at_X1 - zp_at_X2 # positive extinction coefficient

# The fitted zeropoint is the intercept at airmass=0.
zp_at_X0 = zp_at_X1 + k_predicted

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the fit is to k * (1 - X) ... why do you want the zero point at X=0?

@ehneilsen ehneilsen Oct 7, 2026 •

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fit I am doing is to k*X, not k*(1-x): the independent variable is assigned to airmass, not 1-airmass (here). This is a more useful fit: the zero point set at airmass=0 is independent of anything the atmosphere does, and should be very stable, while if it is at airmass=1, it depends on the atmosphere.

So, for the purpose of understanding whether only the things that should be changing really are, separating the influence of the atmosphere from the influence of the instrument is what we want, so we want the zp to be top of the atmosphere.

Because the X=0 zp is expected to be very stable and probably traceable to events like cleaning the mirrors, there is a second variant of the fit that only fits k, and does not allow the zp to move. If we use zp1, though, it varies with k, and setting the instrumental portion to a constant is not possible without converting it back to zp0.

Note that the zp at airmass=0 form is more traditional as well. Aside from the LSST tech notes, the zp at X=1 form is only used by Pan-STARRS that I can find. Everything else I've been able to find (SDSS, DES, and reference books like Birney's Observational Astronomy, Hiltner's Astronomical Techniques, and Budding's An Introduction to Astronomical Photometry) uses the zp at X=0 form for Bouguer's Law (which is what the variation of m0 with airmass is called, a name I hadn't known before this literature search).

@ehneilsen
ehneilsen force-pushed the tickets/SP-3223 branch 2 times, most recently from 341435c to 16b6bcd Compare September 18, 2026 17:43
@rhiannonlynne

Copy link
Copy Markdown
Member

To do:

  • find some cloudy and non cloudy nights that overlap DP2 FGCM available visit zeropoint offsets
  • find 'cloud extinction' from this stacker
  • find 'cloud extinction' from simple rubin_nights assumptions

Plot them all together, look at 'cloud' values over each of the nights
Check time required for stacker vs. simple method -- as a reference

to consider:
Should we add this value to new consdb 'predictions' table?

Generated with AI
Co-Authored-By: SLAC AI
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants