lynguine.assess
The assess module provides functionality for processing and analysing data, including custom DataFrame implementations and computation capabilities.
Data Module
- lynguine.assess.data.iskeyword()
x.__contains__(y) <==> y in x.
- class lynguine.assess.data.DataObject(data=None, colspecs=None, index=None, column=None, selector=None, subindex=None)[source]
Bases:
object
- get_subseries()[source]
Get the subseries that is the focus for the DataFrame.
- Returns:
The subseries that is the focus for the DataFrame.
- Return type:
- get_subindices()[source]
Get the subindices that are the focus for the DataFrame.
- Returns:
The subindices that are the focus for the DataFrame.
- Return type:
- get_index()[source]
Get the index that is the focus for the DataFrame.
- Returns:
The index that is the focus for the DataFrame.
- Return type:
- get_column()[source]
Get the column that is the focus for the DataFrame.
- Returns:
The column that is the focus for the DataFrame.
- set_column(column)[source]
Set the column to be the focus for the DataFrame.
- Parameters:
column – The column to set.
- Raises:
KeyError – If the column is not in the DataFrame.
- ismutable(column)[source]
Is a given column mutable?
- Parameters:
column – The column to check.
- Returns:
True if the column is mutable, False otherwise.
- property autocache
Whether the data structure will automatically cache values.
:return True if the data structure will automatically cache values, False otherwise.
- property interface
Return the interface object. :return: The interface object.
- property mutable
Is the data structure mutable.
:return True if the data structure is mutable, False otherwise.
- isparameter(column)[source]
Test if the column is a given column a parameter (i.e. applicable for all rows)?
- Parameters:
column – The column to check.
- Returns:
True if the column is a parameter, False otherwise.
- isseries(column)[source]
Test if a given column is a series column (i.e. one where the index can be repeated and have multiple rows associated with it).
- Parameters:
column – The column to check.
- Returns:
True if the column is a series column, False otherwise.
- get_selector()[source]
Get the selector that is the focus for the DataFrame.
The selector is the column used to disambiguate a series in the focus.
- Returns:
The selector that is the focus for the DataFrame.
- set_selector(column)[source]
Set the selector that is the focus for the DataFrame.
The selector
- Parameters:
column – The column to set the selector to.
- Raises:
KeyError – If the column is not in the DataFrame.
- get_subindex()[source]
Get the subindex that is the focus for the DataFrame. The subindex is the index used to disambiguate a series in the focus.
- Returns:
The subindex that is the focus for the DataFrame.
- Return type:
- set_subindex(index)[source]
Set the subindex that is the focus for the DataFrame.
If the subindex is set to None, the last row of the subseries is returned. The last row is assumed to be the most recent.
- get_selectors()[source]
Return valid selectors, these are columns that are present in the type “series” :return: The valid selectors.
- get_value()[source]
Get the value that is in the cell defined as focus for the DataFrame.
- Returns:
The value that is the focus for the DataFrame.
- set_value(value)[source]
Set the value that is in the cell defined as focus for the DataFrame.
- Parameters:
value – The value to set.
- add_row(index, typ=None, values=None)[source]
Add a new row to the CustomDataFrame for the current column in focus.
- Parameters:
index – The index of the new row to be added.
typ – The type of row being added.
values – A dictionary of column:value pairs for the new row. If None, adds NaN values.
- Raises:
ValueError – If the index already exists or if it’s not possible to add the row.
- head(n=5)[source]
Return the first n rows of the DataFrame.
- Parameters:
n – Number of rows to select.
- Returns:
The first n rows of the DataFrame.
- tail(n=5)[source]
Return the last n rows of the DataFrame.
- Parameters:
n – Number of rows to select.
- Returns:
The last n rows of the DataFrame.
- add_column(column_name, data)[source]
Add a new column to the DataFrame.
- Parameters:
column_name – The name of the new column.
data – The data for the new column.
- Raises:
NotImplementedError – Indicates the method needs to be implemented in a subclass.
- drop_column(column_name)[source]
Drop a column from the DataFrame.
- Parameters:
column_name – The name of the column to drop.
- Raises:
NotImplementedError – Indicates the method needs to be implemented in a subclass.
- filter_rows(condition)[source]
Filter rows based on a specified condition.
- Parameters:
condition – The condition to filter rows.
- Raises:
NotImplementedError – Indicates the method needs to be implemented in a subclass.
- get_shape()[source]
Get the shape of the DataFrame.
- Returns:
A tuple representing the shape of the DataFrame.
- describe()[source]
Generate descriptive statistics.
- Returns:
Descriptive statistics for the DataFrame.
- to_pandas()[source]
Convert the custom DataFrame to a Pandas DataFrame.
- Raises:
NotImplementedError – Indicates the method needs to be implemented in a subclass.
- to_clipboard(*args, **kwargs)[source]
Copy the DataFrame to the system clipboard.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_clipboard.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_clipboard.
- Returns:
Output of Pandas DataFrame to_clipboard method.
- to_feather(*args, **kwargs)[source]
Write the DataFrame to a Feather file.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_feather.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_feather.
- Returns:
Output of Pandas DataFrame to_feather method.
- to_json(*args, **kwargs)[source]
Convert the DataFrame to a JSON string.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_json.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_json.
- Returns:
Output of Pandas DataFrame to_json method.
- to_orc(*args, **kwargs)[source]
Write the DataFrame to an ORC file.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_orc.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_orc.
- Returns:
Output of Pandas DataFrame to_orc method.
- to_records(*args, **kwargs)[source]
Convert the DataFrame to a NumPy record array.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_records.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_records.
- Returns:
Output of Pandas DataFrame to_records method.
- to_timestamp(*args, **kwargs)[source]
Cast to DatetimeIndex of timestamps.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_timestamp.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_timestamp.
- Returns:
Output of Pandas DataFrame to_timestamp method.
- to_csv(*args, **kwargs)[source]
Write the DataFrame to a comma-separated values (csv) file.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_csv.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_csv.
- Returns:
Output of Pandas DataFrame to_csv method.
- to_gbq(*args, **kwargs)[source]
Write the DataFrame to a Google BigQuery table.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_gbq.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_gbq.
- Returns:
Output of Pandas DataFrame to_gbq method.
- to_latex(*args, **kwargs)[source]
Render the DataFrame as a LaTeX tabular environment table.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_latex.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_latex.
- Returns:
Output of Pandas DataFrame to_latex method.
- to_parquet(*args, **kwargs)[source]
Write the DataFrame to a Parquet file.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_parquet.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_parquet.
- Returns:
Output of Pandas DataFrame to_parquet method.
- to_sql(*args, **kwargs)[source]
Write records stored in the DataFrame to a SQL database.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_sql.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_sql.
- Returns:
Output of Pandas DataFrame to_sql method.
- to_xarray(*args, **kwargs)[source]
Convert the DataFrame to an xarray Dataset.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_xarray.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_xarray.
- Returns:
Output of Pandas DataFrame to_xarray method.
- to_dict(*args, **kwargs)[source]
Convert the DataFrame to a dictionary.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_dict.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_dict.
- Returns:
Output of Pandas DataFrame to_dict method.
- to_hdf(*args, **kwargs)[source]
Write the DataFrame to a Hierarchical Data Format (HDF) file.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_hdf.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_hdf.
- Returns:
Output of Pandas DataFrame to_hdf method.
- to_markdown(*args, **kwargs)[source]
Convert the DataFrame to a Markdown string.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_markdown.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_markdown.
- Returns:
Output of Pandas DataFrame to_markdown method.
- to_period(*args, **kwargs)[source]
Convert DataFrame from DatetimeIndex to PeriodIndex.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_period.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_period.
- Returns:
Output of Pandas DataFrame to_period method.
- to_stata(*args, **kwargs)[source]
Export the DataFrame to Stata data format.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_stata.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_stata.
- Returns:
Output of Pandas DataFrame to_stata method.
- to_xml(*args, **kwargs)[source]
Convert the DataFrame to an XML string.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_xml.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_xml.
- Returns:
Output of Pandas DataFrame to_xml method.
- to_excel(*args, **kwargs)[source]
Write the DataFrame to an Excel file.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_excel.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_excel.
- Returns:
Output of Pandas DataFrame to_excel method.
- to_html(*args, **kwargs)[source]
Render the DataFrame as an HTML table.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_html.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_html.
- Returns:
Output of Pandas DataFrame to_html method.
- to_numpy(*args, **kwargs)[source]
Convert the DataFrame to a NumPy array.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_numpy.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_numpy.
- Returns:
Output of Pandas DataFrame to_numpy method.
- to_string(*args, **kwargs)[source]
Render the DataFrame to a console-friendly tabular output.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_string.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_string.
- Returns:
Output of Pandas DataFrame to_string method.
- class property valid_data_types
- classmethod from_pandas(df, colspecs)[source]
Create a CustomDataFrame from a pandas DataFrame.
- Parameters:
- Returns:
A CustomDataFrame object.
- Return type:
- classmethod from_csv(*args, **kwargs)[source]
Read a comma-separated values (csv) file into a CustomDataFrame.
- Parameters:
args – Positional arguments to be passed to pandas.read_csv.
kwargs – Keyword arguments to be passed to pandas.read_csv.
- Returns:
A CustomDataFrame object.
- classmethod from_dict(data, *args, **kwargs)[source]
Construct a CustomDataFrame from a dict of array-like or dicts.
- Parameters:
data – Dictionary of data.
args – Positional arguments to be passed to pandas.DataFrame.from_dict.
kwargs – Keyword arguments to be passed to pandas.DataFrame.from_dict.
- Returns:
A CustomDataFrame object.
- to_flow(interface)[source]
Writes the output data to the flows specified in the interface.
- Parameters:
interface – Interface object.
- classmethod compute_from_flow(interface)[source]
Construct a compute object from an interface object.
- Parameters:
interface (lynguine.config.interface.Interface or dict.) – Interface object.
- Returns:
A compute object.
- Return type:
- classmethod from_flow(interface)[source]
Construct a CustomDataFrame from a interface object.
- Parameters:
interface (lynguine.config.interface.Interface or dict.) – Interface object.
- Returns:
A CustomDataFrame object.
- augment_with_df(df, colspecs)[source]
This method is used to augment the CustomDataFrame with a new DataFrame.
- sort_values(*args, inplace=False, **kwargs)[source]
Sort by the values along either axis.
- Parameters:
by – str or list of str
axis – {0 or ‘index’, 1 or ‘columns’}, default 0
ascending – bool or list of bool, default True
inplace – bool, default False
kind – {‘quicksort’, ‘mergesort’, ‘heapsort’}, default ‘quicksort’
na_position – {‘first’, ‘last’}, default ‘last’
ignore_index – bool, default False
key – callable, optional
- Returns:
sorted_obj : CustomDataFrame
- Raises:
ValueError – If any of the keys are not in the index
- sort_index(*args, inplace=False, **kwargs)[source]
Sort object by labels (along an axis).
- Parameters:
axis – {0 or ‘index’, 1 or ‘columns’}, default 0
level – int or level name or list of ints or list of level names
ascending – bool or list of bool, default True
inplace – bool, default False
kind – {‘quicksort’, ‘mergesort’, ‘heapsort’}, default ‘quicksort’
na_position – {‘first’, ‘last’}, default ‘last’
sort_remaining – bool, default True
ignore_index – bool, default False
key – callable, optional.
- Returns:
sorted_obj : CustomDataFrame
- convert(other)[source]
Convert various data types to a CustomDataFrame.
This method handles conversion from different data types like Pandas DataFrame, Pandas Series, NumPy array, list, and dictionary to the CustomDataFrame format.
- Parameters:
other – The data to be converted.
- Returns:
A CustomDataFrame object.
- Raises:
ValueError – If the data type of other cannot be converted.
- groupby(*args, **kwargs)[source]
Group DataFrame using a mapper or by a Series of columns.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.groupby.
kwargs – Keyword arguments to be passed to pandas.DataFrame.groupby.
- Returns:
A pd.DataFrameGroupBy object.
- pivot_table(*args, **kwargs)[source]
Create a pivot table as a CustomDataFrame.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.pivot_table.
kwargs – Keyword arguments to be passed to pandas.DataFrame.pivot_table.
- Returns:
A pivoted CustomDataFrame object.
- property dtypes: Series
Property to return the data types of the columns across all sub-DataFrames. :return: A Series containing the data types of the columns.
- property log
Access the system log.
- Returns:
Reference to the system log.
- property T
Transpose the DataFrame.
- Returns:
Transposed CustomDataFrame.
- property shape
Return the shape of the DataFrame.
- Returns:
A tuple representing the DataFrame’s dimensions.
- property columns
Return the column labels of the DataFrame.
- Returns:
Index object containing the column labels.
- property index
Return the index (row labels) of the DataFrame.
- Returns:
Index object containing the row labels.
- property empty
Return True if DataFrame is empty.
- property values
Return the values of the DataFrame as a NumPy array.
- Returns:
A NumPy array of the DataFrame’s values.
- property colspecs
Return the column specifications.
- Returns:
Column specifications.
- apply(func, axis=0, raw=False, result_type=None, args=(), **kwargs)[source]
Apply a function along an axis of the DataFrame.
- Parameters:
func – Function to apply to each column or row.
axis – Axis along which the function is applied. 0 or ‘index’: apply function to each column. 1 or ‘columns’: apply function to each row.
raw – Determines if row/column labels are passed to the function.
result_type – Type of the return value. Can be ‘expand’, ‘reduce’, or None.
args – Additional positional arguments to pass to func.
kwargs – Additional keyword arguments to pass to func.
- Returns:
A new CustomDataFrame with the results of the applied function.
- merge(right, how='inner', on=None, left_on=None, right_on=None, suffixes=('_x', '_y'), *args, **kwargs)[source]
Merge CustomDataFrame with another DataFrame or CustomDataFrame.
- Parameters:
right – Another DataFrame or CustomDataFrame to merge with.
kwargs (how, on, left_on, right_on, args,) – Parameters for pandas.DataFrame.merge.
suffixes – A tuple of string suffixes to apply to overlapping column names.
- Returns:
A new CustomDataFrame resulting from the merge operation.
- join(other, on=None, how='left', lsuffix='', rsuffix='', sort=False, validate=None, *args, **kwargs)[source]
Join CustomDataFrame with another DataFrame or CustomDataFrame.
- Parameters:
other – Another DataFrame or CustomDataFrame to merge with.
how (str) – How to join the DataFrames.
lsuffix (str) – Suffix to apply to overlapping column names from the left DataFrame.
rsuffix (str) – Suffix to apply to overlapping column names from the right DataFrame.
sort (bool) – Sort the join keys lexicographically in the result DataFrame.
args – Positional arguments to be passed to pandas.DataFrame.join.
kwargs – Keyword arguments to be passed to pandas.DataFrame.join.
- Returns:
A new CustomDataFrame resulting from the join operation.
- class lynguine.assess.data.CustomDataFrame(data, colspecs=None, index=None, column=None, selector=None, subindex=None, compute=None, interface=None)[source]
Bases:
DataObject
- types = {'cache': ['cache', 'series_cache', 'parameter_cache', 'global_cache'], 'input': ['input', 'data', 'constants', 'global_consts'], 'output': ['output', 'series', 'writedata', 'writeseries', 'parameters', 'globals'], 'parameters': ['constants', 'global_consts', 'parameters', 'globals', 'parameter_cache', 'global_cache'], 'series': ['series', 'writeseries', 'series_cache']}
- property autocache
Whether the data structure will automatically cache values.
:return True if the data structure will automatically cache values, False otherwise.
- property interface
Return the interface object. :return: The interface object.
- get_type_columns(col_type)[source]
Return the columns in the CustomDataFrame that are of a specified type.
- Parameters:
col_type – The type of columns to return.
- Returns:
A list of columns of the specified type.
- get_input_columns()[source]
Return the columns in the CustomDataFrame that are immutable.
- Returns:
A list of immutable (input) columns.
- get_series_columns()[source]
Return the columns in the CustomDataFrame that are of type “series”.
- Returns:
A list of series columns.
- get_parameters_columns()[source]
Return the columns in the CustomDataFrame that are of type “parameters”.
- Returns:
A list of parameters columns.
- get_output_columns()[source]
Return the columns in the CustomDataFrame that are of type “output”.
- Returns:
A list of output columns.
- property colspecs
Return the column specifications.
- Returns:
Column specifications.
- coltype(col)[source]
Return the type of the column.
- Parameters:
col – The column name.
- Returns:
The type of the column.
- to_pandas()[source]
Convert the CustomDataFrame to a pandas DataFrame.
- Returns:
A pandas DataFrame representation of the CustomDataFrame.
- Return type:
- update_from_pandas(df, colspecs=None)[source]
Convert from a pandas data frame to a CustomDataFrame.
- update_name_column_map(name, column)[source]
Update the map from valid variable names to columns in the data frame. Valid variable names are needed e.g. for Liquid filters.
- mapping(mapping=None, series=None)[source]
Generate dictionary of mapping between variable names and column values.
- Parameters:
mapping – mapping to use, if None use default mapping
series – series to use, if None use current series
- Returns:
dictionary of mapping between variable names and column values
- Return type:
- summary_viewer_to_value(viewer, kwargs=None)[source]
Convert a summary viewer structure to populated values.
- property T
Transpose the DataFrame.
- Returns:
Transposed CustomDataFrame.
- add(other)
- add_column(column_name, data)
Add a new column to the DataFrame.
- Parameters:
column_name – The name of the new column.
data – The data for the new column.
- Raises:
NotImplementedError – Indicates the method needs to be implemented in a subclass.
- add_row(index, typ=None, values=None)
Add a new row to the CustomDataFrame for the current column in focus.
- Parameters:
index – The index of the new row to be added.
typ – The type of row being added.
values – A dictionary of column:value pairs for the new row. If None, adds NaN values.
- Raises:
ValueError – If the index already exists or if it’s not possible to add the row.
- apply(func, axis=0, raw=False, result_type=None, args=(), **kwargs)
Apply a function along an axis of the DataFrame.
- Parameters:
func – Function to apply to each column or row.
axis – Axis along which the function is applied. 0 or ‘index’: apply function to each column. 1 or ‘columns’: apply function to each row.
raw – Determines if row/column labels are passed to the function.
result_type – Type of the return value. Can be ‘expand’, ‘reduce’, or None.
args – Additional positional arguments to pass to func.
kwargs – Additional keyword arguments to pass to func.
- Returns:
A new CustomDataFrame with the results of the applied function.
- augment_with_df(df, colspecs)
This method is used to augment the CustomDataFrame with a new DataFrame.
- property columns
Return the column labels of the DataFrame.
- Returns:
Index object containing the column labels.
- classmethod compute_from_flow(interface)
Construct a compute object from an interface object.
- Parameters:
interface (lynguine.config.interface.Interface or dict.) – Interface object.
- Returns:
A compute object.
- Return type:
- convert(other)
Convert various data types to a CustomDataFrame.
This method handles conversion from different data types like Pandas DataFrame, Pandas Series, NumPy array, list, and dictionary to the CustomDataFrame format.
- Parameters:
other – The data to be converted.
- Returns:
A CustomDataFrame object.
- Raises:
ValueError – If the data type of other cannot be converted.
- describe()
Generate descriptive statistics.
- Returns:
Descriptive statistics for the DataFrame.
- dot(other)
- drop_column(column_name)
Drop a column from the DataFrame.
- Parameters:
column_name – The name of the column to drop.
- Raises:
NotImplementedError – Indicates the method needs to be implemented in a subclass.
- drop_duplicates(*args, **kwargs)
- dropna()
- property dtypes: Series
Property to return the data types of the columns across all sub-DataFrames. :return: A Series containing the data types of the columns.
- property empty
Return True if DataFrame is empty.
- equals(other)
- fillna(*args, **kwargs)
- filter_rows(condition)
Filter rows based on a specified condition.
- Parameters:
condition – The condition to filter rows.
- Raises:
NotImplementedError – Indicates the method needs to be implemented in a subclass.
- classmethod from_csv(*args, **kwargs)
Read a comma-separated values (csv) file into a CustomDataFrame.
- Parameters:
args – Positional arguments to be passed to pandas.read_csv.
kwargs – Keyword arguments to be passed to pandas.read_csv.
- Returns:
A CustomDataFrame object.
- classmethod from_dict(data, *args, **kwargs)
Construct a CustomDataFrame from a dict of array-like or dicts.
- Parameters:
data – Dictionary of data.
args – Positional arguments to be passed to pandas.DataFrame.from_dict.
kwargs – Keyword arguments to be passed to pandas.DataFrame.from_dict.
- Returns:
A CustomDataFrame object.
- classmethod from_flow(interface)
Construct a CustomDataFrame from a interface object.
- Parameters:
interface (lynguine.config.interface.Interface or dict.) – Interface object.
- Returns:
A CustomDataFrame object.
- classmethod from_pandas(df, colspecs)
Create a CustomDataFrame from a pandas DataFrame.
- Parameters:
- Returns:
A CustomDataFrame object.
- Return type:
- get_column()
Get the column that is the focus for the DataFrame.
- Returns:
The column that is the focus for the DataFrame.
- get_index()
Get the index that is the focus for the DataFrame.
- Returns:
The index that is the focus for the DataFrame.
- Return type:
- get_selector()
Get the selector that is the focus for the DataFrame.
The selector is the column used to disambiguate a series in the focus.
- Returns:
The selector that is the focus for the DataFrame.
- get_selectors()
Return valid selectors, these are columns that are present in the type “series” :return: The valid selectors.
- get_shape()
Get the shape of the DataFrame.
- Returns:
A tuple representing the shape of the DataFrame.
- get_subindex()
Get the subindex that is the focus for the DataFrame. The subindex is the index used to disambiguate a series in the focus.
- Returns:
The subindex that is the focus for the DataFrame.
- Return type:
- get_subindices()
Get the subindices that are the focus for the DataFrame.
- Returns:
The subindices that are the focus for the DataFrame.
- Return type:
- get_subseries()
Get the subseries that is the focus for the DataFrame.
- Returns:
The subseries that is the focus for the DataFrame.
- Return type:
- get_subseries_values()
Return a pd.Series containing all the values in a column.
- get_value()
Get the value that is in the cell defined as focus for the DataFrame.
- Returns:
The value that is the focus for the DataFrame.
- get_value_column(column)
Get a value from a column in the data frame(s)
- groupby(*args, **kwargs)
Group DataFrame using a mapper or by a Series of columns.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.groupby.
kwargs – Keyword arguments to be passed to pandas.DataFrame.groupby.
- Returns:
A pd.DataFrameGroupBy object.
- head(n=5)
Return the first n rows of the DataFrame.
- Parameters:
n – Number of rows to select.
- Returns:
The first n rows of the DataFrame.
- property index
Return the index (row labels) of the DataFrame.
- Returns:
Index object containing the row labels.
- ismutable(column)
Is a given column mutable?
- Parameters:
column – The column to check.
- Returns:
True if the column is mutable, False otherwise.
- isna()
- isnull()
- isparameter(column)
Test if the column is a given column a parameter (i.e. applicable for all rows)?
- Parameters:
column – The column to check.
- Returns:
True if the column is a parameter, False otherwise.
- isseries(column)
Test if a given column is a series column (i.e. one where the index can be repeated and have multiple rows associated with it).
- Parameters:
column – The column to check.
- Returns:
True if the column is a series column, False otherwise.
- join(other, on=None, how='left', lsuffix='', rsuffix='', sort=False, validate=None, *args, **kwargs)
Join CustomDataFrame with another DataFrame or CustomDataFrame.
- Parameters:
other – Another DataFrame or CustomDataFrame to merge with.
how (str) – How to join the DataFrames.
lsuffix (str) – Suffix to apply to overlapping column names from the left DataFrame.
rsuffix (str) – Suffix to apply to overlapping column names from the right DataFrame.
sort (bool) – Sort the join keys lexicographically in the result DataFrame.
args – Positional arguments to be passed to pandas.DataFrame.join.
kwargs – Keyword arguments to be passed to pandas.DataFrame.join.
- Returns:
A new CustomDataFrame resulting from the join operation.
- property log
Access the system log.
- Returns:
Reference to the system log.
- mean(axis=0)
- merge(right, how='inner', on=None, left_on=None, right_on=None, suffixes=('_x', '_y'), *args, **kwargs)
Merge CustomDataFrame with another DataFrame or CustomDataFrame.
- Parameters:
right – Another DataFrame or CustomDataFrame to merge with.
kwargs (how, on, left_on, right_on, args,) – Parameters for pandas.DataFrame.merge.
suffixes – A tuple of string suffixes to apply to overlapping column names.
- Returns:
A new CustomDataFrame resulting from the merge operation.
- multiply(other)
- property mutable
Is the data structure mutable.
:return True if the data structure is mutable, False otherwise.
- notna()
- pivot_table(*args, **kwargs)
Create a pivot table as a CustomDataFrame.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.pivot_table.
kwargs – Keyword arguments to be passed to pandas.DataFrame.pivot_table.
- Returns:
A pivoted CustomDataFrame object.
- save_flows()
Save the output flows.
- set_column(column)
Set the column to be the focus for the DataFrame.
- Parameters:
column – The column to set.
- Raises:
KeyError – If the column is not in the DataFrame.
- set_index(index)
Set the index to be the focus for the DataFrame.
- set_selector(column)
Set the selector that is the focus for the DataFrame.
The selector
- Parameters:
column – The column to set the selector to.
- Raises:
KeyError – If the column is not in the DataFrame.
- set_subindex(index)
Set the subindex that is the focus for the DataFrame.
If the subindex is set to None, the last row of the subseries is returned. The last row is assumed to be the most recent.
- set_value(value)
Set the value that is in the cell defined as focus for the DataFrame.
- Parameters:
value – The value to set.
- set_value_column(value, column)
Set a value to a column in the write data frame
- property shape
Return the shape of the DataFrame.
- Returns:
A tuple representing the DataFrame’s dimensions.
- sort_index(*args, inplace=False, **kwargs)
Sort object by labels (along an axis).
- Parameters:
axis – {0 or ‘index’, 1 or ‘columns’}, default 0
level – int or level name or list of ints or list of level names
ascending – bool or list of bool, default True
inplace – bool, default False
kind – {‘quicksort’, ‘mergesort’, ‘heapsort’}, default ‘quicksort’
na_position – {‘first’, ‘last’}, default ‘last’
sort_remaining – bool, default True
ignore_index – bool, default False
key – callable, optional.
- Returns:
sorted_obj : CustomDataFrame
- sort_values(*args, inplace=False, **kwargs)
Sort by the values along either axis.
- Parameters:
by – str or list of str
axis – {0 or ‘index’, 1 or ‘columns’}, default 0
ascending – bool or list of bool, default True
inplace – bool, default False
kind – {‘quicksort’, ‘mergesort’, ‘heapsort’}, default ‘quicksort’
na_position – {‘first’, ‘last’}, default ‘last’
ignore_index – bool, default False
key – callable, optional
- Returns:
sorted_obj : CustomDataFrame
- Raises:
ValueError – If any of the keys are not in the index
- subtract(other)
- sum(axis=0)
- tail(n=5)
Return the last n rows of the DataFrame.
- Parameters:
n – Number of rows to select.
- Returns:
The last n rows of the DataFrame.
- to_clipboard(*args, **kwargs)
Copy the DataFrame to the system clipboard.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_clipboard.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_clipboard.
- Returns:
Output of Pandas DataFrame to_clipboard method.
- to_csv(*args, **kwargs)
Write the DataFrame to a comma-separated values (csv) file.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_csv.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_csv.
- Returns:
Output of Pandas DataFrame to_csv method.
- to_dict(*args, **kwargs)
Convert the DataFrame to a dictionary.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_dict.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_dict.
- Returns:
Output of Pandas DataFrame to_dict method.
- to_excel(*args, **kwargs)
Write the DataFrame to an Excel file.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_excel.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_excel.
- Returns:
Output of Pandas DataFrame to_excel method.
- to_feather(*args, **kwargs)
Write the DataFrame to a Feather file.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_feather.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_feather.
- Returns:
Output of Pandas DataFrame to_feather method.
- to_flow(interface)
Writes the output data to the flows specified in the interface.
- Parameters:
interface – Interface object.
- to_gbq(*args, **kwargs)
Write the DataFrame to a Google BigQuery table.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_gbq.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_gbq.
- Returns:
Output of Pandas DataFrame to_gbq method.
- to_hdf(*args, **kwargs)
Write the DataFrame to a Hierarchical Data Format (HDF) file.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_hdf.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_hdf.
- Returns:
Output of Pandas DataFrame to_hdf method.
- to_html(*args, **kwargs)
Render the DataFrame as an HTML table.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_html.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_html.
- Returns:
Output of Pandas DataFrame to_html method.
- to_json(*args, **kwargs)
Convert the DataFrame to a JSON string.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_json.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_json.
- Returns:
Output of Pandas DataFrame to_json method.
- to_latex(*args, **kwargs)
Render the DataFrame as a LaTeX tabular environment table.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_latex.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_latex.
- Returns:
Output of Pandas DataFrame to_latex method.
- to_markdown(*args, **kwargs)
Convert the DataFrame to a Markdown string.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_markdown.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_markdown.
- Returns:
Output of Pandas DataFrame to_markdown method.
- to_numpy(*args, **kwargs)
Convert the DataFrame to a NumPy array.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_numpy.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_numpy.
- Returns:
Output of Pandas DataFrame to_numpy method.
- to_orc(*args, **kwargs)
Write the DataFrame to an ORC file.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_orc.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_orc.
- Returns:
Output of Pandas DataFrame to_orc method.
- to_parquet(*args, **kwargs)
Write the DataFrame to a Parquet file.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_parquet.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_parquet.
- Returns:
Output of Pandas DataFrame to_parquet method.
- to_period(*args, **kwargs)
Convert DataFrame from DatetimeIndex to PeriodIndex.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_period.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_period.
- Returns:
Output of Pandas DataFrame to_period method.
- to_records(*args, **kwargs)
Convert the DataFrame to a NumPy record array.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_records.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_records.
- Returns:
Output of Pandas DataFrame to_records method.
- to_sql(*args, **kwargs)
Write records stored in the DataFrame to a SQL database.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_sql.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_sql.
- Returns:
Output of Pandas DataFrame to_sql method.
- to_stata(*args, **kwargs)
Export the DataFrame to Stata data format.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_stata.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_stata.
- Returns:
Output of Pandas DataFrame to_stata method.
- to_string(*args, **kwargs)
Render the DataFrame to a console-friendly tabular output.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_string.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_string.
- Returns:
Output of Pandas DataFrame to_string method.
- to_timestamp(*args, **kwargs)
Cast to DatetimeIndex of timestamps.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_timestamp.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_timestamp.
- Returns:
Output of Pandas DataFrame to_timestamp method.
- to_xarray(*args, **kwargs)
Convert the DataFrame to an xarray Dataset.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_xarray.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_xarray.
- Returns:
Output of Pandas DataFrame to_xarray method.
- to_xml(*args, **kwargs)
Convert the DataFrame to an XML string.
- Parameters:
args – Positional arguments to be passed to pandas.DataFrame.to_xml.
kwargs – Keyword arguments to be passed to pandas.DataFrame.to_xml.
- Returns:
Output of Pandas DataFrame to_xml method.
- transpose()
- valid_data_types = {'cache', 'constants', 'data', 'global_cache', 'global_consts', 'globals', 'input', 'output', 'parameter_cache', 'parameters', 'series', 'series_cache', 'writedata', 'writeseries'}
- property values
Return the values of the DataFrame as a NumPy array.
- Returns:
A NumPy array of the DataFrame’s values.
- display_to_value(display, kwargs=None, local={})[source]
Convert a display string to a string.
- Parameters:
- Returns:
The string extracted from the display string.
- Return type:
- Raises:
KeyError – If the mapping doesn’t contain the key requested in the display string.
- lynguine.assess.data.concat(objs, *args, **kwargs)[source]
Concatenate a sequence of CustomDataFrame objects into a single CustomDataFrame.
- Parameters:
objs – A sequence of CustomDataFrame objects to concatenate.
args – Positional arguments to be passed to pandas.concat.
kwargs – Keyword arguments to be passed to pandas.concat.
- Returns:
A new CustomDataFrame representing the concatenation of the input dataframes.
- Raises:
ValueError – If objs is empty or contains non-CustomDataFrame objects.
Compute Module
- class lynguine.assess.compute.Compute(interface)[source]
Bases:
object
The Compute class handles computation operations on data frames.
This class is responsible for executing various computation operations specified in an interface configuration. It supports precomputation, computation, and post-computation operations, along with liquid template rendering.
The computation operations are specified in the interface using a dictionary or list of dictionaries with fields like ‘function’, ‘field’, ‘args’, etc.
- _computes
Dictionary of computation operations divided by type (‘precompute’, ‘compute’, ‘postcompute’).
- Type:
- _liquid_env
The liquid template environment for rendering templates.
- logger
Logger instance for logging messages.
Initialize the compute object.
Creates a new Compute object with the specified interface, setting up computation operations, liquid environment, and logger.
- Parameters:
interface (lynguine.config.interface.Interface or dict) – The interface to be used for computation operations.
- Returns:
None
- setup_logger()[source]
Setup the logger for the Compute class.
Initializes the logger with the appropriate name, log level, and output file based on the context configuration.
- Returns:
None
- property computes
Return the compute operations.
- Returns:
The list of compute operations defined in the interface.
- Return type:
- property precomputes
Return the precompute operations.
These operations are executed before the main compute operations.
- Returns:
The list of precompute operations defined in the interface.
- Return type:
- property postcomputes
Return the postcompute operations.
These operations are executed after the main compute operations.
- Returns:
The list of postcompute operations defined in the interface.
- Return type:
- prep(settings, data)[source]
Prepare a compute entry for use.
Processes the settings dictionary to create a standardized compute entry with function, arguments, and refresh flag.
- Parameters:
settings (dict) – The settings to be used for the computation.
data (lynguine.assess.data.CustomDataFrame or pandas.DataFrame) – The data to be used in the computation.
- Returns:
The prepared compute entry.
- Return type:
- gca_(function, field=None, refresh=False, args={}, row_args={}, view_args={}, function_args={}, subseries_args={}, column_args={})[source]
Args generator for compute functions.
- Parameters:
function (str) – The name of the function to be used.
field (str) – The field to be used.
refresh (bool) – Whether to refresh the field.
args (dict) – The arguments to be used.
row_args (dict) – The row arguments to be used.
view_args (dict) – The view arguments to be used.
function_args (dict) – The function arguments to be used.
subseries_args (dict) – The subseries arguments to be used.
column_args (dict) – The column arguments to be used.
- Returns:
The arguments to be used.
- Return type:
- gcf_(function, data)[source]
Function generator for compute functions.
Creates and returns a function to be used for computation based on the function name specified. The returned function will access values from the data object according to the argument specifications.
- Parameters:
function (str) – The name of the function to be used.
data (lynguine.assess.data.CustomDataFrame) – The data object to be used in the function.
- Returns:
The compute function that will be executed.
- Return type:
function
- Raises:
ValueError – If the specified function is not found in the list of available functions.
- run(data, interface)[source]
Run the compute operations on the data.
This method executes the compute operations specified in the interface on the data. First it runs any precompute operations, then the main compute operations, and finally any postcompute operations.
For each compute operation, it: 1. Prepares the operation with the settings and data 2. Gets the current value for the target field 3. Determines whether to compute (based on refresh flag and existing value) 4. Computes the new value and sets it in the data frame
- Parameters:
data (lynguine.assess.data.CustomDataFrame) – The data to be processed.
interface (lynguine.config.interface.Interface or dict) – The interface defining the compute operations.
- Returns:
None
- Raises:
ValueError – If a compute operation is missing required fields.
- Return type:
None
- preprocess(data, interface)[source]
Run preprocessing operations on the data.
Executes the precompute operations specified in the interface on the data. These operations are run before the main compute operations.
Similar to run(), but only processes the ‘precompute’ operations.
- Parameters:
data (lynguine.assess.data.CustomDataFrame) – The data to be preprocessed.
interface (lynguine.config.interface.Interface or dict) – The interface defining the precompute operations.
- Returns:
None
- Raises:
ValueError – If a precompute operation is missing required fields.
- Return type:
None
- run_onchange(data, index, column)[source]
Run compute operations when a specific cell changes.
This method is used to run compute operations when a specific cell in the data frame changes. It checks if the changed cell is used as an argument in any compute operation and runs those operations if needed.
- Parameters:
data (lynguine.assess.data.CustomDataFrame) – The data frame containing the changed cell.
index (object) – The index of the changed cell.
column (str) – The column of the changed cell.
- Returns:
None
- Return type:
None
- run_all(data, interface)[source]
Run compute operations on all indices in the data frame.
Iterates through all indices in the data frame and runs the compute operations for each index. This is useful for batch processing the entire data set.
- Parameters:
data (lynguine.assess.data.CustomDataFrame) – The data to process.
interface (lynguine.config.interface.Interface or dict) – The interface defining the compute operations.
- Returns:
None
- Return type:
None
- filter(data, interface)[source]
Apply filters to the data based on the interface.
This method applies filters to the data frame based on filter specifications in the interface. It can select or exclude rows based on column values.
- Parameters:
data (lynguine.assess.data.CustomDataFrame) – The data to filter.
interface (lynguine.config.interface.Interface or dict) – The interface defining the filters.
- Returns:
None
- Raises:
ValueError – If a filter operation is invalid.
- Return type:
None
- load_liquid(interface)[source]
Initialize the liquid template environment.
Sets up the liquid template rendering environment with custom filters and configurations from the interface.
- Parameters:
interface (lynguine.config.interface.Interface or dict) – The interface containing liquid configuration.
- Returns:
None
- add_liquid_filters()[source]
Add custom filters to the liquid environment.
This method can be extended to add additional custom filters to the liquid template environment.
- Returns:
None
- classmethod from_flow(interface)[source]
Create a Compute instance from an interface.
Factory method to create a Compute instance from an interface specification, which can be either an Interface object or a dictionary.
- Parameters:
interface (lynguine.config.interface.Interface or dict) – The interface to create the Compute from.
- Returns:
A new Compute instance.
- Return type:
Display Module
- class lynguine.assess.display.WidgetCluster(name, parent, viewer=False, **kwargs)[source]
Bases:
object
A class to hold and manage a cluster of widgets.
Initialize a WidgetCluster.
- Parameters:
- add(cluster=None, **kwargs)[source]
Add widgets or a sub-cluster to this cluster.
- Parameters:
cluster (WidgetCluster, optional) – A WidgetCluster to add as a sub-cluster, defaults to None
kwargs – Widgets to add to the cluster
- update(**kwargs)[source]
Update existing widgets in the cluster.
- Parameters:
kwargs – The widgets to update, with their new values
- Raises:
ValueError – If attempting to update a non-existent widget
- class lynguine.assess.display.DisplaySystem(index=None, data=None, interface=None, system=None)[source]
Bases:
object
A class to manage the display system for data visualization and interaction.
Initialize a DisplaySystem.
- Parameters:
- property index
Get the current index of the data.
- Returns:
The current index
- Return type:
pd.Index or None
- get_index()[source]
Get the current index of the data.
- Returns:
The current index
- Return type:
object or None
- set_index(value)[source]
Set the index of the data.
- Parameters:
value (str) – The new index value
- Return type:
None
- get_value()[source]
Get the current value of the data.
- Returns:
The current value
- Return type:
object or None
- set_value_by_element(value, element, trigger_update=True)[source]
Set the value of a specific element in the data.
- get_column()[source]
Get the current column of the data.
- Returns:
The current column
- Return type:
str or None
- set_column(column)[source]
Set the current column of the data.
- Parameters:
column (str) – The new column to set
- Return type:
None
- get_indices()[source]
Get all indices of the data.
- Returns:
All indices
- Return type:
pd.Index or None
- add_downstream_display(display)[source]
Add a downstream display to update when this display updates.
- Parameters:
display (DisplaySystem) – The downstream display to add