conversations

conversations#

Functions for processing conversation data may require you to provide a MessageConfig (which can either be a MessageConfig object from galactic.utils, or just a dictionary with the same fields). It it, you just need to specify the following fields:

  • role_field (str): What field in a message specifies the speaker? For OpenAI chats, it’s “role” In ShareGPT, it’s “from”. etc.

  • content_field (str): What field in a message specifies the content? For OpenAI chats, it’s “content”. It may also be “value”, “text”, etc. in other formats.

  • user_role (str): In the role field, what value refers to the human user? It may be “user” (for OpenAI), but other formats may have things like “human”.

  • assistant_role (str): In the role field, what value refers to the assistant? It may be “assistant” (for OpenAI), but other formats may have things like “bot”, “gpt”, etc.

  • system_role (str): In the role field, what value refers to the system? It may be “system” (for OpenAI), or some formats may not use system prompts (in this case, it’s fine to just use “system”).

galactic.conversations.add_initial_system_message(self, input_column, input_message_config, system_message, output_column=None)[source]#

Adds (the same) initial system message to every conversation.

Parameters:

  • input_column (str) –

  • input_message_config (galactic.utils.MessageConfig | dict) –

  • system_message (str) –

  • output_column (str | None) –

galactic.conversations.conversation_from_dicts(self, input_column, input_message_config, output_column=None, output_message_config=MessageConfig(role_field='role', content_field='content', user_role='user', assistant_role='assistant', system_role='system'), conversation_field=None, metadata_fields=[])[source]#

Parses (somewhat) arbitrary JSON containing conversations into standard format for OpenAI/etc. conversational fine-tuning. If output column is not provided, overwrites the input column.

Parameters:

  • input_column (str) –

  • input_message_config (galactic.utils.MessageConfig | dict) –

  • output_column (str | None) –

  • output_message_config (galactic.utils.MessageConfig) –

  • conversation_field (str | None) –

  • metadata_fields (list[str] | None) –

galactic.conversations.conversation_from_string(self, input_column, user_delimiter, assistant_delimiter, system_delimiter, output_column=None, output_message_config=MessageConfig(role_field='role', content_field='content', user_role='user', assistant_role='assistant', system_role='system'), strip_whitespace=True, replace_values=None)[source]#

Parses a conversation from a string ‘transcript’ into a structured object with a list of messages.

Parameters:

  • input_column (str) –

  • user_delimiter (str) –

  • assistant_delimiter (str) –

  • system_delimiter (str) –

  • output_column (str | None) –

  • output_message_config (galactic.utils.MessageConfig) –

  • strip_whitespace (bool) –

  • replace_values (dict[str, str] | None) –

galactic.conversations.convert_conversation_to_string(self, input_column, input_message_config, output_column, user_delimiter, assistant_delimiter, system_delimiter, strip_whitespace=True)[source]#

Converts a conversation from the standard format to a string, using the provided delimiters (e.g. User:, Assistant:)

Parameters:

  • input_column (str) –

  • input_message_config (galactic.utils.MessageConfig | dict) –

  • output_column (str) –

  • user_delimiter (str) –

  • assistant_delimiter (str) –

  • system_delimiter (str) –

  • strip_whitespace (bool) –

galactic.conversations.get_conversation_length(self, input_column, output_column)[source]#

Get the length (in messages) of a conversation. If you want length in tokens, convert it to a string, and then use count_tokens from the taggers module.

Parameters:

  • input_column (str) –

  • output_column (str) –

galactic.conversations.get_conversation_speakers(self, input_column, input_message_config, output_column='__speakers')[source]#

Computes list of just the speakers for each message in each conversation.

Parameters:

  • input_column (str) –

  • input_message_config (galactic.utils.MessageConfig | dict) –

  • output_column (str) –

galactic.conversations.get_last_speaker(self, input_column, input_message_config, output_column='__last_speaker')[source]#

Gets the role of the last speaker for each conversation.

Parameters:

  • input_column (str) –

  • input_message_config (galactic.utils.MessageConfig | dict) –

  • output_column (str) –

galactic.conversations.get_shared_prefix(self, col1, col2, prefix_col, suffix1_col=None, suffix2_col=None, strip_whitespace=True)[source]#

When you have “chosen” and “rejected” conversations, this method will find the longest shared prefix between them. Operates on strings, not lists of messages. Resulting prefix will be put in ‘prefix_col’. If provided, each suffix will be put in the corresponding suffix columns.

Parameters:

  • col1 (str) –

  • col2 (str) –

  • prefix_col (str) –

  • suffix1_col (str | None) –

  • suffix2_col (str | None) –

  • strip_whitespace (bool) –

galactic.conversations.is_alternating(self, input_column, input_message_config, output_column='__alternating')[source]#

Returns true if no speaker goes 2x in a row. Useful if you make assumptions about conversations being alternating and want to catch ones where they’re not.

Parameters:

  • input_column (str) –

  • input_message_config (galactic.utils.MessageConfig | dict) –

  • output_column (str) –

galactic.conversations.standardize_last_turn(self, input_column, input_message_config, last_speaker_role, output_column=None)[source]#

Strip messages from the end of the conversation until the last message is from the specified speaker. This is often helpful for training in the case where you want the last reply to be from the assistant, and you want to train a model to generate that message given the conversation so far.

Parameters:

  • input_column (str) –

  • input_message_config (galactic.utils.MessageConfig | dict) –

  • last_speaker_role (Literal['user', 'assistant', 'system']) –

  • output_column (str | None) –

galactic.conversations.take_initial_system_message(self, input_column, input_message_config, output_column='__initial_system_message', output_type='string', remove_from_input=True)[source]#

Takes initial message and puts it in another column if it’s a system message. If there’s no initial system message, it will be [None? empty?]

Parameters:

  • input_column (str) –

  • input_message_config (galactic.utils.MessageConfig | dict) –

  • output_column (str) –

  • output_type (Literal['dict', 'string']) –

  • remove_from_input (bool) –

galactic.conversations.take_last_message(self, input_column, input_message_config, output_column, output_type='string', remove_from_input=True)[source]#

Take the last message from a conversation and put it in a separate column. It will be removed from the input if remove_from_input is True.

Parameters:

  • input_column (str) –

  • input_message_config (galactic.utils.MessageConfig | dict) –

  • output_column (str) –

  • output_type (Literal['dict', 'string']) –

  • remove_from_input (bool) –

Contents