nlp_data_py.commons.utils package


nlp_data_py.commons.utils.fileutils module

class nlp_data_py.commons.utils.fileutils.FileUtils[source]

Bases: object

Simple util to quickly read and write files. Nothing much here

static file_exist(path)[source]

Checks if file exists

logger = <Logger FileUtils (WARNING)>
static mkdir(path)[source]

Make directory if it dose not already exists

static read_file(file)[source]

Read contents from file.

Parameters:file – str: Path of file to read
Returns:contents of file as strin
Raises:Usual file handling exceptions
static read_pickle(path)[source]

Read Pickled file and return read object

Parameters:path – str: Path to Pickle file
Raises:Usual file ops and pickle Exceptions
static write_content_tofile(content, file, mode='a')[source]

Write content to file. By default it writes in append mode

  • content – str: Contents to write to file
  • file – str: Path where to write
  • mode – str: Mode in which to write. Default is append mode



Usual file handling exceptions

static write_pickle(obj, path)[source]

Write object as pickle file

  • obj – Any: Object to write
  • path – str: Path to write

Usual file and pickle exceptions

nlp_data_py.commons.utils.helpers module

class nlp_data_py.commons.utils.helpers.Helpers[source]

Bases: object

Generic helper methods.

static extend_list(lst: List, ext_with, times)[source]

Extends given list with elements. This is with side effects

  • lst (List) – List to be extended
  • ext_with (Any) – Element with which to extend the list
  • times (Int) – ext_with with be added to list times times



extend_list([1,2,3], 0, 5) will produce [1, 2, 3, 0, 0, 0, 0, 0]
static extend_shorter_list(list1: List, list2: List, ext_with)[source]

Compares 2 lists and extends the shorter with to longer ones length. Shorter list is extended by the element provided in ext_with parameter

  • list1 (List) – First list
  • list2 (List) – Second list
  • ext_with (Any) – Element with which to extend the list



extend_shorter_list([1,2,3], [1, 2], 0) will
produce keep first the same but changes 2nd one to [1, 2, 0]
static extend_shorter_lists(lists: [typing.List[typing.List]], ext_with)[source]

Compares lists of lists and extends shorter lists with ext_with to match the length of largest list.

  • lists (List[List]) – List of lists
  • ext_with (Any) – Element with which to extend the list



list1 = [1, 3, 4, 5, 8]
list2 = [1, 3]
list3 = [5]
list4 = [3, 4, 2, 2, 2]
Helpers.extend_shorter_lists([list1, list2, list3, list4], 9)
assert list1 == [1, 3, 4, 5, 8]
assert list2 == [1, 3, 9, 9, 9]
assert list3 == [5, 9, 9, 9, 9]
assert list4 == [3, 4, 2, 2, 2]
static generate_random_shuffle(length: int)[source]

Randomly shuffles the range from 0 to given length

Parameters:length – Length for range
Returns:Shuffled list of length = length


generate_random_shuffle(10) may produce
shuffled list rangning from 0 and 9
logger = <Logger Helpers (WARNING)>
static normalize_ratios(ratio_list: List)[source]

Softmax of list.

Parameters:ratio_list – List of numbers
Returns:Softmaxed list


normalize_ratios([8, 2, 2]) will produce
[0.8, 0.2, 0.2]

nlp_data_py.commons.utils.logging module

class nlp_data_py.commons.utils.logging.Logging[source]

Bases: object

Generic Logging with logging module. This class has a property: DEFAULT_LEVEL. which is a Log Level variable. Global level will be used when no specific log levels are specified with get_logger. Default global level is DEBUG. This can be overridden

DEFAULT_HANDLERS = [<class 'logging.StreamHandler'>]
DEFAULT_STREAM = <_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>
static get_logger(name: str, level=None, propagate=False, handlers=[<class 'logging.StreamHandler'>], args=[[<_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>]])[source]

Returns a logger of the name provided.

  • name (str) – Name of the logger to be created.
  • level (int) – Level at which logs should be written. (same as logging.Levels) default value is None. If None, Logging.global_level be used
  • propagate (Boolean) – Propagate logs to parent. Default value is False
  • handlers (List of functions) – Function that returns a logging handler. These functions will be called with logger.addHandler. default value is [logging.StreamHandler].
  • args (List[List]) – List of parameters for the handlers. It should match with the number of handlers. If the function takes no parameters leave an empty list. default value: empty list of empty list. default value is args=[[DEFAULT_STREAM]]

Returns: logger

Module contents