[jhonabreul]

Description

Consolidator registration on quote-only feeds (forex, cfd) and with mismatched periods fails late or needlessly. The engine can adapt type mismatches, validate periods at registration, accept calendar periods in RegisterIndicator, and reject null symbols clearly.

The change:

• SubscriptionManager.AddConsolidator adapts trade bar consolidators to quote-only feeds through a new QuoteBarToTradeBarAdapter.
• Each QuoteBar is collapsed into an internal QuoteDerivedTradeBar: mid-point prices, the fill forward flag kept, and reading or writing Volume throws InvalidOperationException.
• Volume indicators on quote-only feeds therefore fail at the first bar with a clear message, instead of computing on zero volume.
• TradeBarConsolidator keeps its consolidated bar volume-less when fed collapsed bars.
• Every adapted registration logs a warning naming the consumer and stating that reading Volume throws, through RegisterIndicator, Consolidate and SubscriptionManager.AddConsolidator alike.
• Readers that use volume only when it is there (HeikinAshi, the default volume selector of the Renko/Range consolidators, SessionBar) check the new TradeBar.HasVolume and skip it on collapsed bars.
• QCAlgorithm.RegisterIndicator<T> does the same collapse at the indicator layer (e.g. self.mass(symbol, resolution=Resolution.HOUR) on a cfd), and registers the consolidator only after the output type check.
• Consolidate with a TradeBar handler and WarmUpIndicator (by period, resolution or automatic warm-up) are adapted the same way, so registration and warm-up agree.
• Adapters derive from ConsolidatorAdapterBase, which equals the consolidator it wraps and shares its hash code, so users remove the consolidator they hold.
• ConsolidatorIdentityComparer backs SubscriptionDataConfig.Consolidators and the scan dictionary, so lookups match from either side (ConcurrentSet now accepts a comparer).
• Fixed time span periods are validated at registration, through the adapter when there is one: a 5-minute consolidator on a daily subscription fails in Initialize.
• That check uses the bar span the runtime check uses: the regular session length for daily data with strict end times, so a 12-hour consolidator on daily SPY still works.
• RegisterIndicator/ResolveConsolidator accept calendar rules (Calendar.WEEKLY, Calendar.MONTHLY, custom), in C# and Python; a Python type passed by mistake (e.g. timedelta without arguments) is rejected rather than taken as a calendar.
• Clearer rejections: the type mismatch error suggests Consolidate(), and a null symbol (e.g. Future.Mapped before the first mapping) throws ArgumentNullException explaining the likely cause.
• BaseData.IsFillForward setter is now protected so the collapsed bar can override Clone(bool) without touching volume.

Important

Known limits

• adapter.Equals(wrapped) is true, but wrapped.Equals(adapter) is false.
• == / ReferenceEquals between the adapter and the wrapped consolidator stay false.
• The comparer covers the collections where identity matters (SubscriptionDataConfig.Consolidators and the scan dictionary).
• SubscriptionDataConfig.Consolidators yields the adapter for adapted registrations; IndicatorBase.Consolidators holds the consolidator the user registered.
• Volume-dependent indicators (OBV, VWAP, ...) on a quote-only feed fail on their first bar, not at registration; the registration warning tells the user so and what to use instead.

Related Issue

N/A

Motivation and Context

The engine has all the information to adapt or fail fast at registration, and the late period error can kill a backtest months of simulated time in.

Requires Documentation Change

register_indicator docs can mention calendar periods. The quote-to-trade adaptation is transparent, except that volume on the collapsed bars throws, which the registration warning states.

How Has This Been Tested?

• ConsolidatorAutoAdaptationRegressionAlgorithm (C# + Python): trade bar consolidator, MassIndex and HeikinAshi on EURUSD receive collapsed bars, and the bars throw on volume.
• RegisterIndicatorWithCalendarRegressionAlgorithm (C# + Python): weekly RSI registered with Calendar.WEEKLY updates when the week boundary is crossed.
• ConsolidatorPeriodValidationRegressionAlgorithm (C# + Python): 10-second consolidator on minute data is rejected in Initialize.
• QuoteDerivedTradeBarTests: mid-point collapse, volume get/set throw, HasVolume false, Renko tolerates the bars, fill forward flag kept, clones keep the type, consolidation keeps bars volume-less.
• SubscriptionManagerTests: adapter data flow and removal, Python adapted consolidator add/remove, adapter equality, comparer matching both ways, period validation cases (also through the adapter, and 7/12/23-hour periods accepted on daily data with precise end times), null-symbol message, mismatch suggestion.
• AlgorithmIndicatorsTests: calendar registration in C# and Python, MassIndex and HeikinAshi on a quote-only feed update, OnBalanceVolume on a quote-only feed throws on the first bar.
• AlgorithmIndicatorsTests: adapted registrations warn the user by resolution, trade consolidator, quote consolidator, Consolidate and direct AddConsolidator; trade feeds do not warn.
• AlgorithmIndicatorsTests: the indicator keeps the consolidator the user registered while the subscription holds the adapter that equals it.
• AlgorithmIndicatorsTests: a type mismatch leaves no consolidator registered, timedelta itself as third argument is rejected, Consolidate with a trade bar handler on EURUSD receives collapsed bars.
• AlgorithmIndicatorsTests: MassIndex on EURUSD warms up by period, by resolution and through automatic warm-up.
• PythonIndicatorTests: updated messages.
• Suites: affected fixtures 570 passed, 0 failed, including the six regression algorithm cases.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• Refactor (non-breaking change which improves implementation)
• Performance (non-breaking change which improves performance. Please add associated performance test and results)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Non-functional change (xml comments/documentation/etc)

Checklist:

• My code follows the code style of this project.
• I have read the CONTRIBUTING document.
• I have added tests to cover my changes.
• All new and existing tests passed.
• My branch follows the naming convention bug-<issue#>-<description> or feature-<issue#>-<description>

[quantconnect-kronos-reviewer]

Thanks, this is careful work. The null-symbol message, the Consolidate() hint and calendar periods in RegisterIndicator all look good. CI is green. I built the head commit and ran two small backtests on both master and this branch. Two behavior regressions came out of that, so I'm requesting changes.

1. The registration-time period check rejects consolidators that work today (blocking).
ValidateConsolidatorPeriod compares the period to subscription.Increment. The existing check in PeriodCountConsolidatorBase.Update compares it to the length of the bar that actually arrives (data.EndTime - data.Time), and those two differ for daily data. With the default DailyPreciseEndTime = true, a daily equity bar runs 09:30 to 16:00, so it is 6.5h long, not 1 day.

• Probe: SPY at daily resolution plus new TradeBarConsolidator(TimeSpan.FromHours(12)), from 2013-10-07 to 2013-10-11.
• master: runs and emits 4 consolidated bars.
• This branch: fails in Initialize with Unable to add consolidator for symbol SPY: the consolidator period (12:00:00) is smaller than the daily subscription period (1.00:00:00).

Any fixed period between the session length and 1 day is affected, on any daily subscription with precise end times. The new check has to be no stricter than the runtime one. One option is to validate only intraday resolutions, where the increment equals the bar length. Another is to derive the daily bar length from exchange hours. Either way, please add a test case for this.

2. Quote-to-trade adaptation moves some failures from Initialize to the first bar (blocking).
This contradicts the PR's own goal of failing at registration.

• Volume indicators. RegisterIndicator(eurusd, new OnBalanceVolume(), Resolution.Hour) used to throw a type mismatch in Initialize. Now it registers fine and throws on the first consolidated bar. VolumeIndicatorOnQuoteOnlyFeedFailsWhenReadingVolume asserts exactly that. In live trading, that bar can arrive hours after deployment.
• Code that reads volume only when it's there. HeikinAshi reads Volume from any TradeBar (Indicators/HeikinAshi.cs:111). BaseTimelessConsolidator's default volume selector does the same (BaseTimelessConsolidator.cs:75), as do IntradayVwap and SessionBar. All of them now throw on a QuoteDerivedTradeBar.
  • Probe: EURUSD at minute resolution plus RegisterIndicator(eurusd, new HeikinAshi(), new TradeBarConsolidator(TimeSpan.FromHours(1))).
  • master: type mismatch in Initialize.
  • This branch: runtime error The trade bar for EURUSD was collapsed from a quote bar and has no volume ... Use ... an indicator that does not depend on volume on the first bar. HeikinAshi doesn't depend on volume, so that message points users the wrong way.
  • ClassicRenkoConsolidator<TradeBar> / RenkoConsolidator<TradeBar> on forex hit the same throw through the default volume selector.

I'd keep every case that fails today failing in Initialize. Some possible ways:

• Adapt only when the consumer is known not to read volume.
• Give the collapsed bar zero volume plus a flag that volume-dependent indicators check at registration.
• Keep the type mismatch for registrations that pass an explicit TradeBarConsolidator.

Whichever you pick, please cover the chained cases above with tests.

3. Scope. There's no linked issue, and the PR bundles four independent changes. Calendar periods, the null-symbol message and the mismatch hint are low risk and could merge on their own. The adaptation and the period validation change behavior and carry the issues above. I'd suggest splitting those two into their own PR.

Minor

• ConsolidatorAdapterBase.Equals isn't symmetric (adapter.Equals(wrapped) is true, wrapped.Equals(adapter) is false). The description calls this out. It's only safe as long as every collection holding adapters uses ConsolidatorIdentityComparer. IndicatorBase.Consolidators holds the user's consolidator, so it's fine today. A test pinning that would help keep it that way.
• SubscriptionDataConfig.Consolidators now exposes the adapter, not the consolidator the user registered, to anyone who iterates it.

Testing / evidence: The unit tests and regression algorithms cover the expected paths. They don't cover daily bars with precise end times, or consumers that read volume opportunistically. Once this is reworked, please also add the outcome of an end-to-end run for a forex algorithm that registers a trade bar consolidator and indicator.

Other repositories: The public API additions (the RegisterIndicator / ResolveConsolidator calendar overloads, ConsolidatorAdapterBase, QuoteBarToTradeBarAdapter, ConsolidatorIdentityComparer, the ConcurrentSet constructor, the BaseData.IsFillForward protected setter) reach the generated Python stubs on the next release. The documentation still needs an update for calendar periods in register_indicator and for how trade bar consolidators behave on forex/CFD.

Risk: risk:high. The PR adds to the public algorithm API and changes registration behavior users rely on, and the two regressions above are reproducible.

[quantconnect-kronos-reviewer]

subscription.Increment is 1 day for daily data, but with DailyPreciseEndTime a daily equity bar is only 09:30 to 16:00 long. The runtime check in PeriodCountConsolidatorBase.Update compares against data.EndTime - data.Time, so a 12h consolidator on daily SPY works on master (4 bars over 2013-10-07 to 2013-10-11) and is now rejected here in Initialize. This check needs to be no stricter than the runtime one.

[quantconnect-kronos-reviewer]

Some consumers read Volume from any TradeBar they receive: HeikinAshi (line 111), BaseTimelessConsolidator's default volume selector, IntradayVwap, and SessionBar. With this throw they now fail at runtime on adapted feeds. RegisterIndicator(eurusd, new HeikinAshi(), new TradeBarConsolidator(TimeSpan.FromHours(1))) throws in Initialize on master. On this branch it throws on the first bar, with a message telling the user to pick an indicator that doesn't depend on volume.

[quantconnect-kronos-reviewer]

For volume indicators (OBV, VWAP(symbol, period), ...) on forex/CFD, this branch turns an Initialize-time type mismatch into a runtime error on the first consolidated bar. In live trading that can be hours after deployment. Could volume-dependent indicators keep failing at registration?