Serialization

lsst.images.serialization Package

Base classes and utilities for the serialization framework.

This includes the read_archive and write_archive functions, which provide the highest-level interfaces for reading arbitrary objects from and writing them to storage, respectively (for a concrete GeneralizedImage subclass, prefer its read and write methods).

InputArchive and OutputArchive are the abstract interfaces for implementing serialization for a new file format.

The base classes make some concessions to both FITS and ASDF in order to make the (potential) representations in those formats conform to their respective expectations.

For ASDF, this is simple: we use ASDF schemas whenever possible to represent primitive types, from units and times to multidimensional arrays. While the archive interfaces use Pydantic, which maps to JSON, not YAML, the expectation is that by encoding YAML tag information in the JSON Schema (which Pydantic allows us to customize), it should be straightforward for an ASDF archive implementation to have Pydantic dump to a Python dict (etc) tree, and then convert that to tagged YAML by walking the tree along with its schema.

For FITS, the challenge is primarily to populate standard FITS header cards when writing, despite the fact that FITS headers are generally too limiting to be our preferred way of round-tripping any information. To do this, the archive interfaces accept update_header and strip_header callback arguments that are only called by FITS implementations.

Functions

available_schema_classes([package])	Return every ArchiveTree subclass owned by package, sorted by schema name.
backend_for_name(name)	Return the Backend with the given format name.
backend_for_path(path)	Return the Backend for path based on its file extension.
backend_for_stream(stream)	Return the Backend for stream based on its leading bytes.
check_frozen_schemas(directory[, package])	Check the frozen schema files against the current models.
class_for_schema(schema_name)	Return the registered ArchiveTree subclass for schema_name.
dump_schema(tree_cls)	Return the JSON Schema for tree_cls.
frozen_schema_filename(tree_cls)	Return the frozen-schema filename for tree_cls.
frozen_schema_path(directory, tree_cls)	Return the frozen-schema file path for tree_cls under directory.
is_development_version(version)	Return whether a schema version string is a PEP 440 development release.
is_integer(t)	Test whether a NumberType corresponds to an integer type.
no_header_updates(header)	Do not make any modifications to the given FITS header.
open_archive(path[, cls, format, partial])	Open an lsst.images file for incremental, component-wise reads.
parameterize_tree(tree_cls, pointer_type)	Parameterise tree_cls over pointer_type if it is generic.
public_type_for_schema(schema_name)	Return the in-memory Python class produced when reading an archive whose top-level tree has schema name schema_name.
read_archive(path[, cls, format])	Read an archive whose in-memory type is inferred from its schema.
register_schema_class(cls)	Register cls under cls.SCHEMA_NAME.
tree_class_for_info(info, path)	Return the registered ArchiveTree subclass for info's schema.
warn_for_development_schemas(root)	Emit a DevelopmentSchemaWarning if a serialized tree contains any schema still in development.
write_archive(obj, path, **kwargs)	Write obj to path, dispatching by file extension.
write_frozen_schemas(directory[, package])	Write the frozen schema file for every current schema.

Classes

ArchiveAccessRequiredError	Exception raised when a deserialization needs data from the file.
ArchiveInfo	Basic identifying information about an on-disk archive.
ArchiveReadError	Exception raised when the contents of an archive cannot be read.
ArchiveTree	An intermediate base class of pydantic.BaseModel that should be used for all objects that may be used as the top-level tree models written to archives.
ArrayReferenceModel	Model for a subset of the ASDF 'ndarray' schema, in the case where the array data is stored elsewhere.
ArrayReferenceQuantityModel	Model for a subset of the ASDF 'quantity' schema for external arrays.
Backend(name, write, input_archive)	A file-format backend resolved from a path suffix.
ButlerInfo	Information about a butler dataset.
DetachedArchive()	An input archive that is not attached to any file.
DevelopmentSchemaWarning	Warning that a file is being written with a development schema.
FrozenSchemaError	A finalized frozen schema would change without a version bump.
InlineArrayModel	Model for a subset of the ASDF 'ndarray' schema, in the case where the array data is stored inline.
InlineArrayQuantityModel	Model for a subset of the ASDF 'quantity' schema for inline arrays.
InputArchive()	Abstract interface for reading from a file format.
InvalidComponentError	Exception ArchiveTree.deserialize_component when passed an invalid component name.
InvalidParameterError	Exception raised by ArchiveTree.deserialize or ArchiveTree.deserialize_component when passed an invalid keyword argument.
JsonRef	Pydantic model for JSON Reference / Pointer (IETF RFC 6901).
NestedOutputArchive(root, parent)	A proxy output archive that joins a root path into all names before delegating back to its parent archive.
NumberType(value[, names, module, qualname, ...])	Enumeration of array values types supported by the library.
OpaqueArchiveMetadata(*args, **kwargs)	Interface for opaque archive metadata.
OutputArchive()	Abstract interface for writing to a file format.
QuantityModel	Model for a subset of the ASDF 'quantity' schema for scalars.
Reader(archive, tree, info, expected_cls)	A handle to an open lsst.images file.
TableColumnModel	Model for a subset of the ASDF table/column schema.
TableModel	Placeholder for an ASDF-like model for referencing or holding binary tabular data.
TimeModel	Model for a subset of the ASDF 'time' schema.
UnsupportedTableError	Exception raised if a table object has column types or structure that are not supported by this library.

Variables

IntegerType	Type alias.
MetadataValue	Type alias.
SCHEMA_URL_BASE	str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
SignedIntegerType	Type alias.
UnsignedIntegerType	Type alias.