Repository navigation
Conversation
…ge, loader and sensor APIs The optional parameters of the sequence reader, loader and writer, the point-cloud and camera-label writers, the point-cloud source adapters of ray bundle sensors, the indexed tar archive and store, the tuning parameters of the camera and lidar models and the optional arguments of their world-ray unprojections are now keyword-only. Positional calls silently bound arguments to the wrong parameter whenever a parameter was added in between: e.g., a positional max_threads of SequenceComponentGroupsReader was taken as the index tail retention. The public packages now export the types their APIs take and return: ncore.data.stores exports ZarrStore, NodeCache, IndexTailRetention, ZarrFormat, Selection, FillValue, Attributes, BloscCname and BloscShuffle; ncore.data exports RayBundleSensorProtocol, AsyncRayBundleSensorPointCloudsSourceAdapter and FOV; ncore.data.v4 exports the base sensor component readers and writers and IndexTailRetention. The type aliases are documented on the API pages. ZarrStore is the store base class of the installed zarr-python version at runtime (it was object). BREAKING CHANGE: the following parameters are keyword-only: SequenceComponentGroupsReader(component_group_paths, *, ...) and its open_async; SequenceLoaderV4(reader, *, ...) and SequenceLoaderV4.open_async(component_group_paths, *, ...); IndexedTarArchive(itar_path, *, mode, index_tail_read_size, remote_range_reads) and IndexedTarArchive.open_async(itar_path, *, ...); IndexedTarZarrStore(itar_path, *, mode, index_tail_read_size, remote_range_reads) and IndexedTarZarrStore.open_async(itar_path, *, ...); FThetaCameraModel and OpenCVFisheyeCameraModel(parameters, device, dtype, *, newton_iterations, min_2d_norm); OpenCVPinholeCameraModel(parameters, device, dtype, *, max_normalized_radius_squared); RowOffsetStructuredSpinningLidarModel(parameters, device, dtype, *, angles_to_columns_map_resolution_factor, angles_to_columns_map_dtype, angles_to_columns_map_init, fov_eps_factor), where device and dtype now directly follow the parameters (they came after the angles_to_columns_map arguments). SequenceComponentGroupsWriter(..., generic_meta_data, *, store_type, zarr_format) and SequenceComponentGroupsWriter.from_reader(..., sequence_reader, *, store_type, zarr_format); PointCloudsComponent.Writer(..., coordinate_unit, *, attribute_schemas), PointCloudsComponent.Writer.store_pc(xyz, reference_frame_id, reference_frame_timestamp_us, *, attributes, generic_data, generic_meta_data) and CameraLabelsComponent.Writer.store_label(data, timestamp_us, *, generic_meta_data); RayBundleSensorPointCloudsSourceAdapter(sensor, *, return_index, promote_generic_data) and its asynchronous twin; the camera_rays / sensor_rays, return_T_sensor_worlds and return_timestamps arguments of the CameraModel pixels_to_world_rays_* / image_points_to_world_rays_* methods and of the lidar elements_to_world_rays_shutter_pose.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
First of six PRs that clean up the public API before v20.0.0-rc.1, after which the 20.0 API is frozen. This one makes optional parameters keyword-only across the storage, loader, writer and sensor model APIs, and exports from the public packages every type those APIs take and return.
Positional calls silently bind to the wrong parameter whenever a parameter is added in between. For example, a v19-style
SequenceComponentGroupsReader(paths, True, 1 << 20, 8)now passes8as the index tail retention instead ofmax_threads. Making these parameters keyword-only closes that class of bug before the freeze.Stack (merge from the top to land all six at once):
Group/ArrayAPIValueErrorfor user-facing validationChanges
open_asynctwin:SequenceComponentGroupsReader(component_group_paths, *, ...),SequenceLoaderV4(reader, *, ...),SequenceLoaderV4.open_async(component_group_paths, *, ...)IndexedTarArchive(itar_path, *, ...),IndexedTarZarrStore(itar_path, *, ...)SequenceComponentGroupsWriter(..., *, store_type, zarr_format)and.from_reader(...)PointCloudsComponent.Writer(..., *, attribute_schemas),store_pc(xyz, reference_frame_id, reference_frame_timestamp_us, *, ...),CameraLabelsComponent.Writer.store_label(data, timestamp_us, *, ...)RayBundleSensorPointCloudsSourceAdapter(sensor, *, return_index, promote_generic_data)and its async twin(parameters, device, dtype, *, tuning...)for FTheta, OpenCV fisheye and OpenCV pinholeRowOffsetStructuredSpinningLidarModel(parameters, device, dtype, *, ...):device/dtypemove directly afterparameters, matching the camera modelspixels_to_world_rays_*/image_points_to_world_rays_*methods and of the lidarelements_to_world_rays_shutter_posencore.data.stores:ZarrStore,NodeCache,IndexTailRetention,ZarrFormat,Selection,FillValue,Attributes,BloscCname,BloscShufflencore.data:RayBundleSensorProtocol,AsyncRayBundleSensorPointCloudsSourceAdapter,FOVncore.data.v4:BaseSensorComponentReader,BaseSensorComponentWriter,BaseRayBundleSensorComponentReader,BaseRayBundleSensorComponentWriter,IndexTailRetentionZarrStoreis the store base class of the installed zarr-python at runtime (wasobject), soisinstanceworks.Testing
test.pychecks the exact__all__and that every export is the implementation object.*makes them fail; the reader test reproduces the v19 bug above (unsupported index tail retention 8).bazel test //... --config=no-gpugreen on all four interpreters,bazel build //...(ty), docs with warnings as errors, andbazel run //:format.checkclean. No in-repo caller passed these parameters positionally.Upgrading
TypeError. For example,IndexedTarZarrStore(path, "w")becomesIndexedTarZarrStore(path, mode="w").RowOffsetStructuredSpinningLidarModel: positionaldevice/dtypenow come directly afterparameters.ncore.impl.Type of Change
Checklist
bazel test //...)bazel run //:format)