Saving and Loading

Saver

saver.check_and_write_dataset(obj, key, hd)

General function for writing an HDF5 dataset.

Parameters

• obj (main.CAESAR) – Main caesar object to save.

• key (str) – Name of dataset to write.

• hd (h5py.Group) – Open HDF5 group.

saver.save(obj, filename='test.hdf5')

Function to save a CAESAR file to disk.

Parameters

• obj (main.CAESAR) – Main caesar object to save.

• filename (str, optional) – Filename of the output file.

Examples

>>> obj.save('output.hdf5')

saver.serialize_attributes(obj_list, hd, hd_dicts)

Function that goes through a list full of halos/galaxies/clouds and serializes their attributes.

Parameters

• obj (main.CAESAR) – Main caesar object.

• hd (h5py.Group) – Open HDF5 group for lists.

• hd_dicts (h5py.Group) – Open HDF5 group for dictionaries.

saver.serialize_global_attribs(obj, hd)

Function that goes through a caesar object and saves general attributes.

Parameters

• obj (main.CAESAR) – Main caesar object.

• hd (h5py.File) – Open HDF5 dataset.

saver.serialize_list(obj_list, key, hd)

Function that serializes a index list (glist/etc) for objects.

Parameters

• obj (main.CAESAR) – Main caesar object.

• key (str) – Name of the index list.

• hd (h5py.Group) – Open HDF5 group.

Loader

This module is a lazy replacement for caesar.loader

Instead of eagerly constructing every Halo, Galaxy, and Cloud, this module provides a class which lazily constructs Groups and their attributes only as they are accessed, and caches them using functools.lru_cache.

The design of this module was motivated by profiling the previous eager loader, which revealed these things dominated load time, in order of importance: 1) Creating unyt.unty_quantity objects 2) Creating Halo/Galaxy/Cloud objects 3) Reading datasets from the HDF5 file Therefore, this module avoids creating quantities as much as possible and caches them. It might be nice to only load part of the backing HDF5 datasets, but that stage is already quite fast and it looks to me like the HDF5 library (or at least h5py) has some minimum granularity at which it will pull data off disk which is ~1M items, which at the time of writing (April 21, 2020) exceeds the size of most datasets in caesar files, including from the m100n1024 SIMBA run I’ve been testing with.

class loader.CAESAR(filename, skip_hash_check=False)

Bases: object

property central_galaxies

cloudinfo(top=10)

galinfo(top=10)

haloinfo(top=10)

property satellite_galaxies

property yt_dataset

The yt dataset to perform actions on.

class loader.Cloud(obj, index)

Bases: loader.Group

property dlist

property glist

class loader.Galaxy(obj, index)

Bases: loader.Group

property bhlist

property cloud_index_list

property clouds

property dlist

property glist

property satellites

property slist

class loader.Group

Bases: object

contamination_check(lowres=[2, 3, 5], search_factor=2.5, printer=True)

info()

property mass

property metallicity

property temperature

write_IC_mask(ic_ds, filename, search_factor=2.5, radius_type='total')

class loader.Halo(obj, index)

Bases: loader.Group

property bhlist

property central_galaxy

property dlist

property dm2list

property dm3list

property dmlist

property galaxies

property galaxy_index_list

property glist

property satellite_galaxies

property slist

class loader.LazyDataset(obj, dataset_path)

Bases: object

A lazily-loaded HDF5 dataset

class loader.LazyDict(keys, builder)

Bases: collections.abc.Mapping

This type should be indistinguishable from the built-in dict. Any observable difference except the explicit type and performance is considered a bug.

The implementation wraps a dict which initially maps every key to None, and are replaced by calling the passed-in callable as they are accessed.

get(k[, d]) → D[k] if k in D, else d.  d defaults to None.

items() → a set-like object providing a view on D's items

keys() → a set-like object providing a view on D's keys

values() → an object providing a view on D's values

class loader.LazyList(length, builder)

Bases: collections.abc.Sequence

This type should be indistinguishable from the built-in list. Any observable difference except the explicit type and performance is considered a bug.

The implementation wraps a list which is intially filled with None, which is very fast to create at any size because None is a singleton. The initial elements are replaced by calling the passed-in callable as they are accessed.

count(value) → integer -- return number of occurrences of value

index(value[, start[, stop]]) → integer -- return first index of value.

Raises ValueError if the value is not present.

Supporting start and stop arguments is optional, but recommended.

loader.load(filename, skip_hash_check=False)