monitor_schema.models.analyzer

Analyzer module.

Submodules

• monitor_schema.models.analyzer.algorithms
• monitor_schema.models.analyzer.analyzer
• monitor_schema.models.analyzer.baseline
• monitor_schema.models.analyzer.targets

Package Contents

Classes

ReferenceProfileId	A baseline based on a static reference profile.
SingleBatchBaseline	Using current batch.
TimeRangeBaseline	A static time range.
TrailingWindowBaseline	A dynamic trailing window.
DatasetMetric	Metrics that are applicable at the dataset level.
SimpleColumnMetric	Simple column metrics that are basically just a single number.
ComplexMetrics	Sketch-based metrics that can only be processed by certain algorithms.
ComparisonConfig	Compare whether the target against a value or against a baseline's metric.
ColumnListChangeConfig	Compare whether the target is equal to a value or not.
FixedThresholdsConfig	Fixed threshold analysis.
StddevConfig	Calculates upper bounds and lower bounds based on stddev from a series of numbers.
SeasonalConfig	An analyzer using stddev for a window of time range.
DriftConfig	An analyzer using stddev for a window of time range.
ExperimentalConfig	Experimental algorithm that is not standardized by the above ones yet.
DiffConfig	Detecting the differences between two numerical metrics.
Analyzer	Configuration for running an analysis.
BaselineType	Supported baseline types.
ReferenceProfileId	A baseline based on a static reference profile.
TrailingWindowBaseline	A dynamic trailing window.
TimeRangeBaseline	A static time range.
SingleBatchBaseline	Using current batch.
TargetLevel	Which nested level we are targeting.
DatasetMatrix	Define the matrix of fields and segments to fan out for monitoring.
ColumnMatrix	Define the matrix of columns and segments to fan out for monitoring.

class monitor_schema.models.analyzer.ReferenceProfileId(**data: Any)

Bases: _Baseline

A baseline based on a static reference profile.

A typical use case is to use a “gold” dataset and upload its profile to WhyLabs. This can be a training dataset as well for an ML model.

type :Literal[BaselineType]

profileId :str

class monitor_schema.models.analyzer.SingleBatchBaseline(**data: Any)

Bases: _SegmentBaseline

Using current batch.

This is used when you want to use one batch to monitor another batch in a different metric entity.

type :Literal[BaselineType]

offset :Optional[int]

datasetId :str

class monitor_schema.models.analyzer.TimeRangeBaseline(**data: Any)

Bases: _SegmentBaseline

A static time range.

Instead of using a single profile or a trailing window, user can lock in a “good” period.

type :Literal[BaselineType]

range :monitor_schema.models.commons.TimeRange

class monitor_schema.models.analyzer.TrailingWindowBaseline(**data: Any)

Bases: _SegmentBaseline

A dynamic trailing window.

This is useful if you don’t have a static baseline to monitor against. This is the default mode for most monitors.

type :Literal[BaselineType]

size :int

offset :Optional[int]

exclusionRanges :Optional[List[monitor_schema.models.commons.TimeRange]]

class monitor_schema.models.analyzer.DatasetMetric

Bases: str, enum.Enum

Metrics that are applicable at the dataset level.

profile_count = profile.count

profile_last_ingestion_time = profile.last_ingestion_time

profile_first_ingestion_time = profile.first_ingestion_time

column_row_count_sum = column_row_count_sum

shape_column_count = shape_column_count

shape_row_count = shape_row_count

input_count = input.count

output_count = output.count

classification_f1 = classification.f1

classification_precision = classification.precision

classification_recall = classification.recall

classification_accuracy = classification.accuracy

classification_auc = classification.auc

regression_mse = regression.mse

regression_mae = regression.mae

regression_rmse = regression.rmse

class monitor_schema.models.analyzer.SimpleColumnMetric

Bases: str, enum.Enum

Simple column metrics that are basically just a single number.

count = count

median = median

max = max

min = min

mean = mean

stddev = stddev

variance = variance

unique_upper = unique_upper

unique_upper_ratio = unique_upper_ratio

unique_est = unique_est

unique_est_ratio = unique_est_ratio

unique_lower = unique_lower

unique_lower_ratio = unique_lower_ratio

count_bool = count_bool

count_bool_ratio = count_bool_ratio

count_integral = count_integral

count_integral_ratio = count_integral_ratio

count_fractional = count_fractional

count_fractional_ratio = count_fractional_ratio

count_string = count_string

count_string_ratio = count_string_ratio

count_null = count_null

count_null_ratio = count_null_ratio

inferred_data_type = inferred_data_type

quantile_p5 = quantile_5

quantile_p75 = quantile_75

quantile_p25 = quantile_25

quantile_p90 = quantile_90

quantile_p95 = quantile_95

quantile_p99 = quantile_99

class monitor_schema.models.analyzer.ComplexMetrics

Bases: str, enum.Enum

Sketch-based metrics that can only be processed by certain algorithms.

histogram = histogram

frequent_items = frequent_items

unique_sketch = unique_sketch

column_list = column_list

class monitor_schema.models.analyzer.ComparisonConfig(**data: Any)

Bases: AlgorithmConfig

Compare whether the target against a value or against a baseline’s metric.

This is useful to detect data type change, for instance.

type :Literal[AlgorithmType]

operator :ComparisonOperator

expected :Optional[ExpectedValue]

baseline :Optional[Union[monitor_schema.models.analyzer.baseline.TrailingWindowBaseline, monitor_schema.models.analyzer.baseline.ReferenceProfileId, monitor_schema.models.analyzer.baseline.TimeRangeBaseline, monitor_schema.models.analyzer.baseline.SingleBatchBaseline]]

class monitor_schema.models.analyzer.ColumnListChangeConfig(**data: Any)

Bases: AlgorithmConfig

Compare whether the target is equal to a value or not.

This is useful to detect data type change, for instance.

type :Literal[AlgorithmType]

mode :Literal[ON_ADD_AND_REMOVE, ON_ADD, ON_REMOVE] = ON_ADD_AND_REMOVE

metric :Literal[ComplexMetrics]

exclude :Optional[List[monitor_schema.models.utils.COLUMN_NAME_TYPE]]

baseline :Union[monitor_schema.models.analyzer.baseline.TrailingWindowBaseline, monitor_schema.models.analyzer.baseline.ReferenceProfileId, monitor_schema.models.analyzer.baseline.TimeRangeBaseline, monitor_schema.models.analyzer.baseline.SingleBatchBaseline]

class monitor_schema.models.analyzer.FixedThresholdsConfig(**data: Any)

Bases: AlgorithmConfig

Fixed threshold analysis.

If user fails to set both upper bound and lower bound, this algorithm becomes a no-op. WhyLabs might enforce the present of either fields in the future.

type :Literal[AlgorithmType]

upper :Optional[float]

lower :Optional[float]

class monitor_schema.models.analyzer.StddevConfig(**data: Any)

Bases: _ThresholdBaseConfig

Calculates upper bounds and lower bounds based on stddev from a series of numbers.

An analyzer using stddev for a window of time range.

This calculation will fall back to Poisson distribution if there is only 1 value in the baseline. For 2 values, we use the formula sqrt((x_i - avg(x))^2 / n - 1)

type :Literal[AlgorithmType]

factor :Optional[float]

minBatchSize :Optional[int]

baseline :Union[monitor_schema.models.analyzer.baseline.TrailingWindowBaseline, monitor_schema.models.analyzer.baseline.TimeRangeBaseline, monitor_schema.models.analyzer.baseline.ReferenceProfileId]

class monitor_schema.models.analyzer.SeasonalConfig(**data: Any)

Bases: _ThresholdBaseConfig

An analyzer using stddev for a window of time range.

This will fall back to Poisson distribution if there is only 1 value in the baseline.

This only works with TrailingWindow baseline (TODO: add backend validation)

type :Literal[AlgorithmType]

algorithm :Literal[arima, rego, stastforecast]

minBatchSize :Optional[int]

alpha :Optional[float]

baseline :monitor_schema.models.analyzer.baseline.TrailingWindowBaseline

stddevTimeRanges :Optional[List[monitor_schema.models.commons.TimeRange]]

stddevMaxBatchSize :Optional[int]

stddevFactor :Optional[float]

class monitor_schema.models.analyzer.DriftConfig(**data: Any)

Bases: AlgorithmConfig

An analyzer using stddev for a window of time range.

This analysis will detect whether the data drifts or not. By default, we use hellinger distance with a threshold of 0.7.

type :Literal[AlgorithmType]

algorithm :Literal[hellinger, ks_test, kl_divergence, variation_distance]

metric :Literal[ComplexMetrics, ComplexMetrics]

threshold :float

minBatchSize :Optional[int]

baseline :Union[monitor_schema.models.analyzer.baseline.TrailingWindowBaseline, monitor_schema.models.analyzer.baseline.ReferenceProfileId, monitor_schema.models.analyzer.baseline.TimeRangeBaseline, monitor_schema.models.analyzer.baseline.SingleBatchBaseline]

class monitor_schema.models.analyzer.ExperimentalConfig(**data: Any)

Bases: AlgorithmConfig

Experimental algorithm that is not standardized by the above ones yet.

type :Literal[AlgorithmType]

implementation :str

baseline :Union[monitor_schema.models.analyzer.baseline.TrailingWindowBaseline, monitor_schema.models.analyzer.baseline.ReferenceProfileId, monitor_schema.models.analyzer.baseline.TimeRangeBaseline, monitor_schema.models.analyzer.baseline.SingleBatchBaseline]

stub :Optional[AlgorithmType]

class monitor_schema.models.analyzer.DiffConfig(**data: Any)

Bases: AlgorithmConfig

Detecting the differences between two numerical metrics.

type :Literal[AlgorithmType]

mode :DiffMode

thresholdType :Optional[ThresholdType]

threshold :float

baseline :Union[monitor_schema.models.analyzer.baseline.TrailingWindowBaseline, monitor_schema.models.analyzer.baseline.ReferenceProfileId, monitor_schema.models.analyzer.baseline.TimeRangeBaseline, monitor_schema.models.analyzer.baseline.SingleBatchBaseline]

class monitor_schema.models.analyzer.Analyzer(**data: Any)

Bases: monitor_schema.models.commons.NoExtrasBaseModel

Configuration for running an analysis.

An analysis targets a metric (note that a metric could be a complex object) for one or multiple fields in one or multiple segments. The output is a list of ‘anomalies’ that might show issues with data.

class Config

Updates JSON schema anyOf to oneOf.

static schema_extra(schema: Dict[str, Any], model: pydantic.BaseModel) → None

Update specific fields here (for Union type, specifically).

metadata :Optional[monitor_schema.models.commons.Metadata]

id :str

displayName :Optional[str]

tags :Optional[List[constr(min_length=3, max_length=32, regex='[0-9a-zA-Z\\-_]')]]

schedule :Optional[Union[monitor_schema.models.commons.CronSchedule, monitor_schema.models.commons.FixedCadenceSchedule]]

disabled :Optional[bool]

targetMatrix :Union[monitor_schema.models.analyzer.targets.ColumnMatrix, monitor_schema.models.analyzer.targets.DatasetMatrix]

dataReadinessDuration :Optional[str]

batchCoolDownPeriod :Optional[str]

backfillGracePeriodDuration :Optional[str]

config :Union[monitor_schema.models.analyzer.algorithms.DiffConfig, monitor_schema.models.analyzer.algorithms.ComparisonConfig, monitor_schema.models.analyzer.algorithms.ListComparisonConfig, monitor_schema.models.analyzer.algorithms.ColumnListChangeConfig, monitor_schema.models.analyzer.algorithms.FixedThresholdsConfig, monitor_schema.models.analyzer.algorithms.StddevConfig, monitor_schema.models.analyzer.algorithms.DriftConfig, monitor_schema.models.analyzer.algorithms.ExperimentalConfig, monitor_schema.models.analyzer.algorithms.SeasonalConfig]

class monitor_schema.models.analyzer.BaselineType

Bases: str, enum.Enum

Supported baseline types.

BatchTimestamp = BatchTimestamp

Reference = Reference

TrailingWindow = TrailingWindow

TimeRange = TimeRange

CurrentBatch = CurrentBatch

class monitor_schema.models.analyzer.ReferenceProfileId(**data: Any)

Bases: _Baseline

A baseline based on a static reference profile.

A typical use case is to use a “gold” dataset and upload its profile to WhyLabs. This can be a training dataset as well for an ML model.

type :Literal[BaselineType]

profileId :str

class monitor_schema.models.analyzer.TrailingWindowBaseline(**data: Any)

Bases: _SegmentBaseline

A dynamic trailing window.

This is useful if you don’t have a static baseline to monitor against. This is the default mode for most monitors.

type :Literal[BaselineType]

size :int

offset :Optional[int]

exclusionRanges :Optional[List[monitor_schema.models.commons.TimeRange]]

class monitor_schema.models.analyzer.TimeRangeBaseline(**data: Any)

Bases: _SegmentBaseline

A static time range.

Instead of using a single profile or a trailing window, user can lock in a “good” period.

type :Literal[BaselineType]

range :monitor_schema.models.commons.TimeRange

class monitor_schema.models.analyzer.SingleBatchBaseline(**data: Any)

Bases: _SegmentBaseline

Using current batch.

This is used when you want to use one batch to monitor another batch in a different metric entity.

type :Literal[BaselineType]

offset :Optional[int]

datasetId :str

class monitor_schema.models.analyzer.TargetLevel

Bases: str, enum.Enum

Which nested level we are targeting.

dataset = dataset

column = column

class monitor_schema.models.analyzer.DatasetMatrix(**data: Any)

Bases: _BaseMatrix

Define the matrix of fields and segments to fan out for monitoring.

.

type :Literal[TargetLevel]

class monitor_schema.models.analyzer.ColumnMatrix(**data: Any)

Bases: _BaseMatrix

Define the matrix of columns and segments to fan out for monitoring.

type :Literal[TargetLevel]

include :Optional[List[Union[ColumnGroups, monitor_schema.models.utils.COLUMN_NAME_TYPE]]]

exclude :Optional[List[Union[ColumnGroups, monitor_schema.models.utils.COLUMN_NAME_TYPE]]]