nlp_data_py.commons.utils package

Submodules

nlp_data_py.commons.utils.fileutils module

class nlp_data_py.commons.utils.fileutils.FileUtils

Bases: object

Simple util to quickly read and write files. Nothing much here

static file_exist(path)

Checks if file exists

logger = <Logger FileUtils (WARNING)>

static mkdir(path)

Make directory if it dose not already exists

static read_file(file)

Read contents from file.

Parameters:file – str: Path of file to read Returns:contents of file as strin Raises:Usual file handling exceptions

static read_pickle(path)

Read Pickled file and return read object

Parameters:path – str: Path to Pickle file Raises:Usual file ops and pickle Exceptions

static write_content_tofile(content, file, mode='a')

Write content to file. By default it writes in append mode

Parameters:

• content – str: Contents to write to file
• file – str: Path where to write
• mode – str: Mode in which to write. Default is append mode

Returns:

Nothing

Raises:

Usual file handling exceptions

static write_pickle(obj, path)

Write object as pickle file

Parameters:

• obj – Any: Object to write
• path – str: Path to write

Raises:

Usual file and pickle exceptions

nlp_data_py.commons.utils.helpers module

class nlp_data_py.commons.utils.helpers.Helpers

Bases: object

Generic helper methods.

static extend_list(lst: List, ext_with, times)

Extends given list with elements. This is with side effects

Parameters:

• lst (List) – List to be extended
• ext_with (Any) – Element with which to extend the list
• times (Int) – ext_with with be added to list times times

Returns:

None

Example:

extend_list([1,2,3], 0, 5) will produce [1, 2, 3, 0, 0, 0, 0, 0]

static extend_shorter_list(list1: List, list2: List, ext_with)

Compares 2 lists and extends the shorter with to longer ones length. Shorter list is extended by the element provided in ext_with parameter

Parameters:

• list1 (List) – First list
• list2 (List) – Second list
• ext_with (Any) – Element with which to extend the list

Returns:

None

Example:

extend_shorter_list([1,2,3], [1, 2], 0) will
produce keep first the same but changes 2nd one to [1, 2, 0]

static extend_shorter_lists(lists: [typing.List[typing.List]], ext_with)

Compares lists of lists and extends shorter lists with ext_with to match the length of largest list.

Parameters:

• lists (List[List]) – List of lists
• ext_with (Any) – Element with which to extend the list

Returns:

None

Example:

list1 = [1, 3, 4, 5, 8]
list2 = [1, 3]
list3 = [5]
list4 = [3, 4, 2, 2, 2]
Helpers.extend_shorter_lists([list1, list2, list3, list4], 9)
assert list1 == [1, 3, 4, 5, 8]
assert list2 == [1, 3, 9, 9, 9]
assert list3 == [5, 9, 9, 9, 9]
assert list4 == [3, 4, 2, 2, 2]

static generate_random_shuffle(length: int)

Randomly shuffles the range from 0 to given length

Parameters:length – Length for range Returns:Shuffled list of length = length

Example:

generate_random_shuffle(10) may produce
shuffled list rangning from 0 and 9

logger = <Logger Helpers (WARNING)>

static normalize_ratios(ratio_list: List)

Softmax of list.

Parameters:ratio_list – List of numbers Returns:Softmaxed list

Example:

normalize_ratios([8, 2, 2]) will produce
[0.8, 0.2, 0.2]

nlp_data_py.commons.utils.logging module

class nlp_data_py.commons.utils.logging.Logging

Bases: object

Generic Logging with logging module. This class has a property: DEFAULT_LEVEL. which is a Log Level variable. Global level will be used when no specific log levels are specified with get_logger. Default global level is DEBUG. This can be overridden

DEFAULT_HANDLERS = [<class 'logging.StreamHandler'>]

DEFAULT_LEVEL = 30

DEFAULT_STREAM = <_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>

static get_logger(name: str, level=None, propagate=False, handlers=[<class 'logging.StreamHandler'>], args=[[<_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>]])

Returns a logger of the name provided.

Parameters:

• name (str) – Name of the logger to be created.
• level (int) – Level at which logs should be written. (same as logging.Levels) default value is None. If None, Logging.global_level be used
• propagate (Boolean) – Propagate logs to parent. Default value is False
• handlers (List of functions) – Function that returns a logging handler. These functions will be called with logger.addHandler. default value is [logging.StreamHandler].
• args (List[List]) – List of parameters for the handlers. It should match with the number of handlers. If the function takes no parameters leave an empty list. default value: empty list of empty list. default value is args=[[DEFAULT_STREAM]]

Returns: logger

Module contents