`antimatter.handlers.pytorch_dataloader`#

Module Contents#

Classes#

PytorchDataLoader

The PytorchDataLoader DataHandler supports a pytorch DataLoader. There are

class antimatter.handlers.pytorch_dataloader.PytorchDataLoader#

Bases: antimatter.handlers.DataHandler

The PytorchDataLoader DataHandler supports a pytorch DataLoader. There are some restrictions on the underlying dataset, which must be iterable, producing two-dimensional dictionaries.

from_generic(cols: List[str], generic_data: List[List[bytes]], extra: Dict[str, Any]) → Any#

from_generic loads the generic data as a dataset into the pytorch DataLoader, passing any extra parameters transparently to the DataLoader constructor

Parameters:

cols – the column names for the underlying data
generic_data – the capsule’s generic data format that is loaded into a pytorch DataLoader
extra – extra data for the DataHandler, passed into the pytorch DataLoader constructor

Returns:

the pytorch DataLoader built with the dataset

to_generic(dl: Any) → Tuple[List[str], List[List[bytes]], Dict[str, Any]]#

to_generic converts a pytorch DataLoader into the generic data format, iterating through the DataLoader’s data set, expecting each iterated item to be a 2-dimensional dictionary.

Parameters:: dl – the DataLoader to extract generic format data from
Returns:: the data in its generic form

field_converter_from_generic(ft: antimatter.fieldtype.fieldtypes.FieldType) → Callable[[bytes], Any]#

field_converter_from_generic gets a field converter function for the given field type that can be used to convert fields from their generic string type to their specific type.

Note that these statement should be true for all implementations, given FieldType ft.

from_gen = field_converter_from_generic(ft) to_gen = field_converter_to_generic(ft)

generic_value == to_gen(from_gen(generic_value)) field_value == from_gen(to_gen(field_value))

Parameters:: ft – the FieldType to get the converter function for
Returns:: a function that can convert field values from generic form

field_converter_to_generic(ft: antimatter.fieldtype.fieldtypes.FieldType) → Callable[[Any], bytes]#

field_converter_to_generic gets a field converter function for the given field type that can be used to convert fields from their specific type to their generic type.