classifiers

classifiers#

galactic.classifiers.ai_classifier(self, new_column, field, classes, prompt=None, backend='openai')[source]#

Classify a field using OpenAI’s API or a HF zero-shot classifier to generate a new column based on an existing column.

Parameters:

  • self – The instance of the class.

  • new_column (str) – Name of the new column to be added to the dataset.

  • field (Optional[str]) – Name of the field to use for classification. Can be None if backend is ‘embeddings’.

  • classes (Union[list[str], dict[str, str]]) – Classes to classify into. Can be a list of labels or a dict of label: description.

  • prompt (Optional[str], optional) – Prompt template (Jinja2) to use for the API request, with a template slot for the field to classify. Defaults to None.

  • backend (str, optional) – Which backend to use. Currently supports ‘embeddings’, ‘openai’, and ‘huggingface’. Defaults to “openai”.

galactic.classifiers.embeddings_classifier(self, new_column, model_path, field='__embedding')[source]#

Use an embeddings model to classify a field.

Parameters:

  • new_column (str) –

  • model_path (str) –

  • field (str) –

galactic.classifiers.fasttext_classifier(self, new_column, model_path, field, normalize=['lower', 'strip'], split_punctuation=True, replace_newlines_with=' __newline__ ')[source]#

Classify a field using a trained fastText model to generate a new column based on an existing column.

Parameters:

  • self – The instance of the class.

  • fasttext_model (str) – The path to the trained fastText model file.

  • new_column (str) – Name of the new column to be added to the dataset.

  • field (Optional[str]) – Name of the field to use for classification. Can be None if the fastText model is trained on embeddings.

  • k (int, optional) – Number of labels to predict. Defaults to 1.

  • model_path (str) –

  • normalize (list[str]) –

  • split_punctuation (bool) –

  • replace_newlines_with (str) –

galactic.classifiers.train_embeddings_classifier(self, model_name, save_dir, label_field, model_type='svm', input_field='__embedding', validation_split=0.1, test_split=0.1, random_seed=42)[source]#

Trains a classifier on the dataset using embeddings.

Parameters:

  • self – The instance of the class.

  • model_name (str) – The name of the model to be saved.

  • save_dir (str) – The directory where the model should be saved.

  • label_field (str) – The field name from the dataset to be used as label.

  • model_type (str, optional) – The type of model to train. Either “svm” or “logistic”. Defaults to “svm”.

  • input_field (str, optional) – The field name from the dataset to be used as input. Defaults to “__embedding”.

  • validation_split (float, optional) – The proportion of the dataset to include in the validation split. Defaults to 0.1.

  • test_split (float, optional) – The proportion of the dataset to include in the test split. Defaults to 0.1.

  • random_seed (int, optional) – Seed for reproducing results. Defaults to 42.

galactic.classifiers.train_fasttext_classifier(self, model_name, save_dir, input_field, label_field, validation_split=0.1, test_split=0.1, normalize=['lower', 'strip'], split_punctuation=True, replace_newlines_with=' __newline__ ', target_model_size='2M', training_duration=300, random_seed=42)[source]#

Trains a fasttext classifier on the dataset provided to the instance.

Parameters:

  • self – The instance of the class.

  • model_name (str) – The name of the model to be saved.

  • save_dir (str) – The directory where the model should be saved.

  • input_field (str) – The field name from the dataset to be used as input.

  • label_field (str) – The field name from the dataset to be used as label.

  • validation_split (float, optional) – The proportion of the dataset to include in the validation split. Defaults to 0.1.

  • test_split (float, optional) – The proportion of the dataset to include in the test split. Defaults to 0.1.

  • normalize (list[str], optional) – List of normalization to apply to the text. Defaults to [“lower”, “strip”].

  • split_punctuation (bool, optional) – Whether to split the text by punctuation. Defaults to True.

  • replace_newlines_with (str, optional) – String to replace newlines with in the text. Defaults to “ __newline__ “.

  • target_model_size (str, optional) – The desired size of the fasttext model. Defaults to “2M”.

  • training_duration (int, optional) – The duration in seconds for the fasttext autotune. Defaults to 300.

  • random_seed (int, optional) – Seed for reproducing results. Defaults to 42.

Contents