`antimatter.handlers`#

Subpackages#

antimatter.handlers.tests

Submodules#

Package Contents#

Classes#

`Datatype`	Datatype is an enumeration of the compatible datatypes supported by
`DataHandler`	Abstract base DataHandler defining the supporting methods a handler for a
`DictList`	The DictList DataHandler supports a list of dictionaries.
`Dictionary`	The Dictionary DataHandler supports a single dictionary value with string
`LangchainHandler`	The LangchainHandler DataHandler supports an implementation of a langchain
`PandasDataFrame`	The PandasDataFrame DataHandler supports a pandas DataFrame. There are
`ScalarHandler`	The Scalar DataHandler supports a scalar value.

Functions#

factory(→ base.DataHandler)

Factory returns an instance of a DataHandler matching the provided Datatype.

class antimatter.handlers.Datatype#

Bases: str, enum.Enum

Datatype is an enumeration of the compatible datatypes supported by antimatter, plus the ‘Unknown’ default placeholder.

Unknown#

Scalar#

Dict#

DictList#

PandasDataframe#

PytorchDataLoader#

LangchainRetriever#

capitalize()#

Return a capitalized version of the string.

More specifically, make the first character have upper case and the rest lower case.

casefold()#: Return a version of the string suitable for caseless comparisons.

center()#

Return a centered string of length width.

Padding is done using the specified fill character (default is a space).

count()#

S.count(sub[, start[, end]]) -> int

Return the number of non-overlapping occurrences of substring sub in string S[start:end]. Optional arguments start and end are interpreted as in slice notation.

encode()#

Encode the string using the codec registered for encoding.

encoding: The encoding in which to encode the string.
errors: The error handling scheme to use for encoding errors. The default is ‘strict’ meaning that encoding errors raise a UnicodeEncodeError. Other possible values are ‘ignore’, ‘replace’ and ‘xmlcharrefreplace’ as well as any other name registered with codecs.register_error that can handle UnicodeEncodeErrors.

endswith()#

S.endswith(suffix[, start[, end]]) -> bool

Return True if S ends with the specified suffix, False otherwise. With optional start, test S beginning at that position. With optional end, stop comparing S at that position. suffix can also be a tuple of strings to try.

expandtabs()#

Return a copy where all tab characters are expanded using spaces.

If tabsize is not given, a tab size of 8 characters is assumed.

find()#

S.find(sub[, start[, end]]) -> int

Return the lowest index in S where substring sub is found, such that sub is contained within S[start:end]. Optional arguments start and end are interpreted as in slice notation.

Return -1 on failure.

format()#

S.format(*args, **kwargs) -> str

Return a formatted version of S, using substitutions from args and kwargs. The substitutions are identified by braces (‘{’ and ‘}’).

format_map()#

S.format_map(mapping) -> str

Return a formatted version of S, using substitutions from mapping. The substitutions are identified by braces (‘{’ and ‘}’).

index()#

S.index(sub[, start[, end]]) -> int

Return the lowest index in S where substring sub is found, such that sub is contained within S[start:end]. Optional arguments start and end are interpreted as in slice notation.

Raises ValueError when the substring is not found.

isalnum()#

Return True if the string is an alpha-numeric string, False otherwise.

A string is alpha-numeric if all characters in the string are alpha-numeric and there is at least one character in the string.

isalpha()#

Return True if the string is an alphabetic string, False otherwise.

A string is alphabetic if all characters in the string are alphabetic and there is at least one character in the string.

isascii()#

Return True if all characters in the string are ASCII, False otherwise.

ASCII characters have code points in the range U+0000-U+007F. Empty string is ASCII too.

isdecimal()#

Return True if the string is a decimal string, False otherwise.

A string is a decimal string if all characters in the string are decimal and there is at least one character in the string.

isdigit()#

Return True if the string is a digit string, False otherwise.

A string is a digit string if all characters in the string are digits and there is at least one character in the string.

isidentifier()#

Return True if the string is a valid Python identifier, False otherwise.

Call keyword.iskeyword(s) to test whether string s is a reserved identifier, such as “def” or “class”.

islower()#

Return True if the string is a lowercase string, False otherwise.

A string is lowercase if all cased characters in the string are lowercase and there is at least one cased character in the string.

isnumeric()#

Return True if the string is a numeric string, False otherwise.

A string is numeric if all characters in the string are numeric and there is at least one character in the string.

isprintable()#

Return True if the string is printable, False otherwise.

A string is printable if all of its characters are considered printable in repr() or if it is empty.

isspace()#

Return True if the string is a whitespace string, False otherwise.

A string is whitespace if all characters in the string are whitespace and there is at least one character in the string.

istitle()#

Return True if the string is a title-cased string, False otherwise.

In a title-cased string, upper- and title-case characters may only follow uncased characters and lowercase characters only cased ones.

isupper()#

Return True if the string is an uppercase string, False otherwise.

A string is uppercase if all cased characters in the string are uppercase and there is at least one cased character in the string.

join()#

Concatenate any number of strings.

The string whose method is called is inserted in between each given string. The result is returned as a new string.

Example: ‘.’.join([‘ab’, ‘pq’, ‘rs’]) -> ‘ab.pq.rs’

ljust()#

Return a left-justified string of length width.

Padding is done using the specified fill character (default is a space).

lower()#: Return a copy of the string converted to lowercase.

lstrip()#

Return a copy of the string with leading whitespace removed.

If chars is given and not None, remove characters in chars instead.

partition()#

Partition the string into three parts using the given separator.

This will search for the separator in the string. If the separator is found, returns a 3-tuple containing the part before the separator, the separator itself, and the part after it.

If the separator is not found, returns a 3-tuple containing the original string and two empty strings.

removeprefix()#

Return a str with the given prefix string removed if present.

If the string starts with the prefix string, return string[len(prefix):]. Otherwise, return a copy of the original string.

removesuffix()#

Return a str with the given suffix string removed if present.

If the string ends with the suffix string and that suffix is not empty, return string[:-len(suffix)]. Otherwise, return a copy of the original string.

replace()#

Return a copy with all occurrences of substring old replaced by new.

count
Maximum number of occurrences to replace. -1 (the default value) means replace all occurrences.

If the optional argument count is given, only the first count occurrences are replaced.

rfind()#

S.rfind(sub[, start[, end]]) -> int

Return the highest index in S where substring sub is found, such that sub is contained within S[start:end]. Optional arguments start and end are interpreted as in slice notation.

Return -1 on failure.

rindex()#

S.rindex(sub[, start[, end]]) -> int

Return the highest index in S where substring sub is found, such that sub is contained within S[start:end]. Optional arguments start and end are interpreted as in slice notation.

Raises ValueError when the substring is not found.

rjust()#

Return a right-justified string of length width.

Padding is done using the specified fill character (default is a space).

rpartition()#

Partition the string into three parts using the given separator.

This will search for the separator in the string, starting at the end. If the separator is found, returns a 3-tuple containing the part before the separator, the separator itself, and the part after it.

If the separator is not found, returns a 3-tuple containing two empty strings and the original string.

rsplit()#

Return a list of the substrings in the string, using sep as the separator string.

sep
The separator used to split the string.

When set to None (the default value), will split on any whitespace character (including n r t f and spaces) and will discard empty strings from the result.

maxsplit
Maximum number of splits (starting from the left). -1 (the default value) means no limit.

Splitting starts at the end of the string and works to the front.

rstrip()#

Return a copy of the string with trailing whitespace removed.

If chars is given and not None, remove characters in chars instead.

split()#

Return a list of the substrings in the string, using sep as the separator string.

sep
The separator used to split the string.

When set to None (the default value), will split on any whitespace character (including n r t f and spaces) and will discard empty strings from the result.

maxsplit
Maximum number of splits (starting from the left). -1 (the default value) means no limit.

Note, str.split() is mainly useful for data that has been intentionally delimited. With natural text that includes punctuation, consider using the regular expression module.

splitlines()#

Return a list of the lines in the string, breaking at line boundaries.

Line breaks are not included in the resulting list unless keepends is given and true.

startswith()#

S.startswith(prefix[, start[, end]]) -> bool

Return True if S starts with the specified prefix, False otherwise. With optional start, test S beginning at that position. With optional end, stop comparing S at that position. prefix can also be a tuple of strings to try.

strip()#

Return a copy of the string with leading and trailing whitespace removed.

If chars is given and not None, remove characters in chars instead.

swapcase()#: Convert uppercase characters to lowercase and lowercase characters to uppercase.

title()#

Return a version of the string where each word is titlecased.

More specifically, words start with uppercased characters and all remaining cased characters have lower case.

translate()#

Replace each character in the string using the given translation table.

table
Translation table, which must be a mapping of Unicode ordinals to Unicode ordinals, strings, or None.

The table must implement lookup/indexing via __getitem__, for instance a dictionary or list. If this operation raises LookupError, the character is left untouched. Characters mapped to None are deleted.

upper()#: Return a copy of the string converted to uppercase.

zfill()#

Pad a numeric string with zeros on the left, to fill a field of the given width.

The string is never truncated.

name()#: The name of the Enum member.

value()#: The value of the Enum member.

exception antimatter.handlers.HandlerFactoryError#

Bases: HandlerError

Error when creating a handler in the handler factory.

class args#

add_note()#: Exception.add_note(note) – add a note to the exception

with_traceback()#: Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

class antimatter.handlers.DataHandler#

Bases: abc.ABC

Abstract base DataHandler defining the supporting methods a handler for a Datatype must implement. A Datatype must support converting from its native type to the generic internal format and back. This conversion should be lossless so that the data added to a Capsule will behave the same when loaded back out.

abstract from_generic(cols: List[str], generic_data: List[List[bytes]], extra: Dict[str, Any]) → Any#

from_generic takes data in its generic form, with a list of column names and a list of data rows, and converts it into the handler’s specific data type.

Parameters:

cols – list of column names for the data
generic_data – list of dictionaries of data
extra – extra information for the handler use when processing

Returns:

the data in the handler’s specific data format

abstract to_generic(data: Any) → Tuple[List[str], List[List[bytes]], Dict[str, Any]]#

to_generic converts data from the handler’s specific data type into a generic form of a list of column names (if applicable), a list of data rows, and a dictionary containing any extra processing info.

Parameters:: data – the data in the handler’s specific data format
Returns:: the data in its generic form

field_converter_from_generic(ft: antimatter.fieldtype.fieldtypes.FieldType) → Callable[[bytes], Any]#

field_converter_from_generic gets a field converter function for the given field type that can be used to convert fields from their generic string type to their specific type.

Note that these statement should be true for all implementations, given FieldType ft.

from_gen = field_converter_from_generic(ft) to_gen = field_converter_to_generic(ft)

generic_value == to_gen(from_gen(generic_value)) field_value == from_gen(to_gen(field_value))

Parameters:: ft – the FieldType to get the converter function for
Returns:: a function that can convert field values from generic form

field_converter_to_generic(ft: antimatter.fieldtype.fieldtypes.FieldType) → Callable[[Any], bytes]#

field_converter_to_generic gets a field converter function for the given field type that can be used to convert fields from their specific type to their generic type.

Note that these statement should be true for all implementations, given FieldType ft.

from_gen = field_converter_from_generic(ft) to_gen = field_converter_to_generic(ft)

generic_value == to_gen(from_gen(generic_value)) field_value == from_gen(to_gen(field_value))

Parameters:: ft – the FieldType to get the converter function for
Returns:: a function that can convert field values to generic form

class antimatter.handlers.DictList#

Bases: antimatter.handlers.base.DataHandler

The DictList DataHandler supports a list of dictionaries.

from_generic(cols: List[str], generic_data: List[List[bytes]], extra: dict) → List[Dict[str, Any]]#

from_generic takes the generic data and passes it on as a list of dictionaries

Parameters:

cols – the column names
generic_data –

the capsule’s generic data format holding the row values :param extra: extra data for the DataHandler :return: the data in a dictionary list format

to_generic(data: List[Dict[str, Any]]) → Tuple[List[str], List[List[bytes]], Dict[str, Any]]#

to_generic converts a list of dictionaries into the generic data format, which is essentially a no-op as DictList has the same format as generic

Parameters:: data – the list of dictionaries to pass across as generic format
Returns:: the data in its generic form

field_converter_from_generic(ft: antimatter.fieldtype.fieldtypes.FieldType) → Callable[[bytes], Any]#

field_converter_from_generic gets a field converter function for the given field type that can be used to convert fields from their generic string type to their specific type.

Note that these statement should be true for all implementations, given FieldType ft.

from_gen = field_converter_from_generic(ft) to_gen = field_converter_to_generic(ft)

generic_value == to_gen(from_gen(generic_value)) field_value == from_gen(to_gen(field_value))

Parameters:: ft – the FieldType to get the converter function for
Returns:: a function that can convert field values from generic form

field_converter_to_generic(ft: antimatter.fieldtype.fieldtypes.FieldType) → Callable[[Any], bytes]#

field_converter_to_generic gets a field converter function for the given field type that can be used to convert fields from their specific type to their generic type.

Note that these statement should be true for all implementations, given FieldType ft.

from_gen = field_converter_from_generic(ft) to_gen = field_converter_to_generic(ft)

generic_value == to_gen(from_gen(generic_value)) field_value == from_gen(to_gen(field_value))

Parameters:: ft – the FieldType to get the converter function for
Returns:: a function that can convert field values to generic form

class antimatter.handlers.Dictionary#

Bases: antimatter.handlers.base.DataHandler

The Dictionary DataHandler supports a single dictionary value with string keys.

from_generic(cols: List[str], generic_data: List[List[bytes]], extra: dict) → Dict[str, Any]#

from_generic expects at most one dictionary in the generic data list, and extracts and flattens this dictionary if it can be found

Parameters:

cols – the column names; should be the string key values in the dictionary
generic_data – the capsule’s generic data format holding the values of the single row
extra – extra data for the DataHandler

Returns:

the dictionary value held in the generic data format

to_generic(data: Dict[str, Any]) → Tuple[List[str], List[List[bytes]], Dict[str, Any]]#

to_generic converts a single dictionary value into the generic data format, flattening the dictionary into a list and extracting the keys in the key:value pairs as the column names.

Parameters:: data – the dictionary value to wrap into a generic format
Returns:: the data in its generic form

field_converter_from_generic(ft: antimatter.fieldtype.fieldtypes.FieldType) → Callable[[bytes], Any]#

field_converter_from_generic gets a field converter function for the given field type that can be used to convert fields from their generic string type to their specific type.

Note that these statement should be true for all implementations, given FieldType ft.

from_gen = field_converter_from_generic(ft) to_gen = field_converter_to_generic(ft)

generic_value == to_gen(from_gen(generic_value)) field_value == from_gen(to_gen(field_value))

Parameters:: ft – the FieldType to get the converter function for
Returns:: a function that can convert field values from generic form

field_converter_to_generic(ft: antimatter.fieldtype.fieldtypes.FieldType) → Callable[[Any], bytes]#

field_converter_to_generic gets a field converter function for the given field type that can be used to convert fields from their specific type to their generic type.

Note that these statement should be true for all implementations, given FieldType ft.

from_gen = field_converter_from_generic(ft) to_gen = field_converter_to_generic(ft)

generic_value == to_gen(from_gen(generic_value)) field_value == from_gen(to_gen(field_value))

Parameters:: ft – the FieldType to get the converter function for
Returns:: a function that can convert field values to generic form

class antimatter.handlers.LangchainHandler#

Bases: antimatter.handlers.DataHandler

The LangchainHandler DataHandler supports an implementation of a langchain Retriever. This handler assumes that the underlying data is a list of two-dimensional data.

from_generic(cols: List[str], generic_data: List[List[bytes]], extra: Dict[str, Any]) → Any#

from_generic loads the generic data into an implementation of a langchain Retriever with langchain Embeddings.

Parameters:

cols – the column names for the underlying data
generic_data – the capsule’s generic data format that is loaded into a langchain Retriever
extra – extra data for the DataHandler, containing information for the Embeddings

Returns:

the langchain Retriever built with the dataset

to_generic(data: Any) → Tuple[List[str], List[List[bytes]], Dict[str, Any]]#

to_generic converts a langchain Retriever with langchain Embeddings into the generic data format.

Parameters:: data – the langchain Retriever
Returns:: the data in its generic format

field_converter_from_generic(ft: antimatter.fieldtype.fieldtypes.FieldType) → Callable[[bytes], Any]#

field_converter_from_generic gets a field converter function for the given field type that can be used to convert fields from their generic string type to their specific type.

Note that these statement should be true for all implementations, given FieldType ft.

from_gen = field_converter_from_generic(ft) to_gen = field_converter_to_generic(ft)

generic_value == to_gen(from_gen(generic_value)) field_value == from_gen(to_gen(field_value))

Parameters:: ft – the FieldType to get the converter function for
Returns:: a function that can convert field values from generic form

field_converter_to_generic(ft: antimatter.fieldtype.fieldtypes.FieldType) → Callable[[Any], bytes]#

field_converter_to_generic gets a field converter function for the given field type that can be used to convert fields from their specific type to their generic type.

Note that these statement should be true for all implementations, given FieldType ft.

from_gen = field_converter_from_generic(ft) to_gen = field_converter_to_generic(ft)

generic_value == to_gen(from_gen(generic_value)) field_value == from_gen(to_gen(field_value))

Parameters:: ft – the FieldType to get the converter function for
Returns:: a function that can convert field values to generic form

class antimatter.handlers.PandasDataFrame#

Bases: antimatter.handlers.DataHandler

The PandasDataFrame DataHandler supports a pandas DataFrame. There are some restrictions on the underlying dataset which must be a two-dimensional data set, or a list of two-dimensional data sets.

from_generic(cols: List[str], generic_data: List[List[bytes]], extra: Dict[str, Any]) → Any#

from_generic loads the generic data into a pandas DataFrame, passing any extra parameters transparently to the DataFrame constructor.

Parameters:

cols – the column names for the underlying data
generic_data – the data rows that are loaded into a pandas DataFrame
extra – extra data for the DataHandler, passed into the pandas DataFrame

Returns:

the pandas DataFrame built with the dataset

to_generic(df: Any) → Tuple[List[str], List[List[bytes]], Dict[str, Any]]#

to_generic converts a pandas DataFrame into the generic data format, formatting the underlying data based on if the underlying data set is a list of two-dimensional records or a single two-dimensional record.

Parameters:: df – the DataFrame to extract generic format data from the underlying data set
Returns:: the data in its generic form

field_converter_from_generic(ft: antimatter.fieldtype.fieldtypes.FieldType) → Callable[[bytes], Any]#

field_converter_from_generic gets a field converter function for the given field type that can be used to convert fields from their generic string type to their specific type.

Note that these statement should be true for all implementations, given FieldType ft.

from_gen = field_converter_from_generic(ft) to_gen = field_converter_to_generic(ft)

generic_value == to_gen(from_gen(generic_value)) field_value == from_gen(to_gen(field_value))

Parameters:: ft – the FieldType to get the converter function for
Returns:: a function that can convert field values from generic form

field_converter_to_generic(ft: antimatter.fieldtype.fieldtypes.FieldType) → Callable[[Any], bytes]#

field_converter_to_generic gets a field converter function for the given field type that can be used to convert fields from their specific type to their generic type.

Note that these statement should be true for all implementations, given FieldType ft.

from_gen = field_converter_from_generic(ft) to_gen = field_converter_to_generic(ft)

generic_value == to_gen(from_gen(generic_value)) field_value == from_gen(to_gen(field_value))

Parameters:: ft – the FieldType to get the converter function for
Returns:: a function that can convert field values to generic form

class antimatter.handlers.ScalarHandler#

Bases: antimatter.handlers.base.DataHandler

The Scalar DataHandler supports a scalar value.

from_generic(cols: List[str], generic_data: List[List[bytes]], extra: Dict[str, Any]) → Any#

from_generic expects a single value in a list of lists and extracts this value if it can be found.

Parameters:

cols – ignored when converting from generic as the column is a static name.
generic_data – the generic data holder wrapping a single value.
extra – extra data for the DataHandler. Ignored when converting.

Returns:

the value held in the generic data format

to_generic(data: Any) → Tuple[list, List[List[bytes]], Dict[str, Any]]#

to_generic converts a scalar value into the generic data format.

Parameters:: data – the scalar value to wrap into a generic format
Returns:: the data in its generic form

field_converter_from_generic(ft: antimatter.fieldtype.fieldtypes.FieldType) → Callable[[bytes], Any]#

field_converter_from_generic gets a field converter function for the given field type that can be used to convert fields from their generic string type to their specific type.

Note that these statement should be true for all implementations, given FieldType ft.

from_gen = field_converter_from_generic(ft) to_gen = field_converter_to_generic(ft)

generic_value == to_gen(from_gen(generic_value)) field_value == from_gen(to_gen(field_value))

Parameters:: ft – the FieldType to get the converter function for
Returns:: a function that can convert field values from generic form

field_converter_to_generic(ft: antimatter.fieldtype.fieldtypes.FieldType) → Callable[[Any], bytes]#

field_converter_to_generic gets a field converter function for the given field type that can be used to convert fields from their specific type to their generic type.

Note that these statement should be true for all implementations, given FieldType ft.

from_gen = field_converter_from_generic(ft) to_gen = field_converter_to_generic(ft)

generic_value == to_gen(from_gen(generic_value)) field_value == from_gen(to_gen(field_value))

Parameters:: ft – the FieldType to get the converter function for
Returns:: a function that can convert field values to generic form

antimatter.handlers.factory(datatype: antimatter.datatype.datatypes.Datatype) → base.DataHandler#

Factory returns an instance of a DataHandler matching the provided Datatype.

Parameters:: datatype – The Datatype to get a handler for.
Returns:

An implementation of the abstract DataHandler for handling data of the given type.

antimatter.handlers#

Subpackages#

Submodules#

Package Contents#

Classes#

Functions#

`antimatter.handlers`#