filters

galactic.filters.apply_bloom_filter(self, fields, inplace=True)

Applies a Bloom filter to the dataset to filter out duplicate examples based on the specified fields.

# Example usage:
ds.apply_bloom_filter(fields=['field1', 'field2'], inplace=False)

Parameters:

• fields (Sequence[str]) – Sequence of field names used to apply the Bloom filter.

• inplace (bool) – Whether to apply the filter in-place or return a new object. Defaults to True.

Returns:

If inplace is False, returns a new object of the same type with the filtered dataset.

Return type:

type(self) if not inplace else None

galactic.filters.filter_regex(self, fields, regex, inplace=True)

Filters the dataset by removing examples that match the specified regex pattern in the specified fields.

# Example usage:
ds.filter_regex(fields=['field1', 'field2'], regex='[0-9]+', inplace=False)

Parameters:

• fields (Sequence[str]) – Sequence of field names to apply the regex filter on.

• regex (str) – The regex pattern used to filter out matching examples from the specified fields.

• inplace (bool) – Whether to apply the filter in-place or return a new object. Defaults to True.

Returns:

If inplace is False, returns a new object of the same type with the filtered dataset.

Return type:

type(self) if not inplace else None

galactic.filters.filter_string(self, fields, values, inplace=True)

Filters the dataset by removing examples that contain any of the specified strings in the specified fields.

# Example usage:
ds.filter_string(fields=['field1', 'field2'], values=['value1', 'value2'], inplace=False)

Parameters:

• fields (Sequence[str]) – Sequence of field names to apply the filter on.

• values (Sequence[str]) – Sequence of string values to filter out from the specified fields.

• inplace (bool) – Whether to apply the filter in-place or return a new object. Defaults to True.

Returns:

If inplace is False, returns a new object of the same type with the filtered dataset.

Return type:

type(self) if not inplace else None