metawards.OutputFiles

class metawards.OutputFiles(output_dir: str = 'output', check_empty: bool = True, force_empty: bool = False, prompt=<built-in function input>, auto_bzip: bool = False)

This is a class that manages all of the output files that are written to during a model outbreak. This object is used to hold the ‘FILE’ objects for open files, and will ensure that these files are closed and written to disk as needed. It will also ensure that files are written to the correct output directory, and that they are only opened when they are needed (e.g. only the first call to open the file will actually open it - subsequent calls will return the already-open file handler)

Examples

>>> output = OutputFiles(output_dir="output", check_empty=True)
>>> FILE = output.open("output.txt")
>>> FILE.write("some output\n")
>>> FILE = output.open("something.csv.bz2", auto_bzip=True)
>>> FILE.write("something,else,is,here\n")
>>> output.flush()
>>> FILE = output.open("output.txt")
>>> FILE.write("some more output\n")
>>> output.close()

Note that you can also use OutputFiles in a contexthandler, to ensure that all output files are automatically closed, e.g.

>>> with OutputFiles(output_dir="output") as output:
>>>     FILE = output.open("output.txt")
>>>     FILE.write("something\n")

__init__(output_dir: str = 'output', check_empty: bool = True, force_empty: bool = False, prompt=<built-in function input>, auto_bzip: bool = False)

Construct a set of OutputFiles. These will all be written to ‘output_dir’.

Parameters

• output_dir (str) – The directory in which to create all of the output files. This directory will be created automatically if it doesn’t exist

• check_empty (bool) – Whether or not to check if the directory is empty before continuing. If the directory is not empty, then the user will be prompted to make a decision to either keep going, choose a different directory, remove existing output or exit

• force_empty (bool) – Force the output directory to be empty. BE CAREFUL as this will remove all files in that directory! There are checks to stop you doing something silly, but these are not fool-proof. The user will be prompted to confirm that the files should be removed

• prompt – This is the function that should be called to prompt the user for input, e.g. to confirm whether or not files should be deleted. This defaults to input. Set this to None if you really want MetaWards to remove files silently (e.g. useful if you are running batch jobs on a cluster and you really know what you are doing)

• auto_bzip (bool) – The default flag for auto_bzip when opening files. If this is true then all files will be automatically bzipped (compressed) as they are written, unless the code opening the file has explicitly asked otherwise

Methods

__delattr__(name, /)	Implement delattr(self, name).
__dir__()	Default dir() implementation.
__enter__()
__eq__(value, /)	Return self==value.
__exit__(exc_type, exc_value, exc_traceback)
__format__(format_spec, /)	Default object formatter.
__ge__(value, /)	Return self>=value.
__getattribute__(name, /)	Return getattr(self, name).
__gt__(value, /)	Return self>value.
__hash__()	Return hash(self).
__init__([output_dir, check_empty, ...])	Construct a set of OutputFiles.
__init_subclass__	This method is called when a class is subclassed.
__le__(value, /)	Return self<=value.
__lt__(value, /)	Return self<value.
__ne__(value, /)	Return self!=value.
__new__(**kwargs)
__reduce__()	Helper for pickle.
__reduce_ex__(protocol, /)	Helper for pickle.
__repr__()	Return repr(self).
__setattr__(name, value, /)	Implement setattr(self, name, value).
__sizeof__()	Size of object in memory, in bytes.
__str__()	Return str(self).
__subclasshook__	Abstract classes can override this to customize issubclass().
_close_dir()	Internal function used to close all of the output files
_open_dir()	Internal function used to open the directory in which all output files will be placed
auto_bzip()	Return whether the default is to automatically bzip2 files
close()	Close all of the files and this directory
flush()	Flush the contents of all files to disk
get_filename(filename)	Return the full expanded filename for 'filename'
get_path()	Return the full expanded path to this directory
is_closed()	Return whether or not the output files are closed
is_database(filename)	Return whether or not 'filename' is an open database
is_open()	Return whether or not the output files are open
open(filename[, auto_bzip, mode, headers, sep])	Open the file called 'filename' in the output directory, returning a handle to that file.
open_db(filename[, auto_bzip, initialise])	Open up a SQLite3 database connection to a file called
open_subdir(dirname)	Create and open a sub-directory in this OutputFiles called 'dirname'.
output_dir()	Return the absolute path of the directory to which the output files will be written
remove(path[, prompt])	Remove the passed filename or directory

Attributes

__dict__
__doc__
__module__
__weakref__	list of weak references to the object (if defined)

__init__(output_dir: str = 'output', check_empty: bool = True, force_empty: bool = False, prompt=<built-in function input>, auto_bzip: bool = False)

Construct a set of OutputFiles. These will all be written to ‘output_dir’.

Parameters

• output_dir (str) – The directory in which to create all of the output files. This directory will be created automatically if it doesn’t exist

• check_empty (bool) – Whether or not to check if the directory is empty before continuing. If the directory is not empty, then the user will be prompted to make a decision to either keep going, choose a different directory, remove existing output or exit

• force_empty (bool) – Force the output directory to be empty. BE CAREFUL as this will remove all files in that directory! There are checks to stop you doing something silly, but these are not fool-proof. The user will be prompted to confirm that the files should be removed

• prompt – This is the function that should be called to prompt the user for input, e.g. to confirm whether or not files should be deleted. This defaults to input. Set this to None if you really want MetaWards to remove files silently (e.g. useful if you are running batch jobs on a cluster and you really know what you are doing)

• auto_bzip (bool) – The default flag for auto_bzip when opening files. If this is true then all files will be automatically bzipped (compressed) as they are written, unless the code opening the file has explicitly asked otherwise

auto_bzip()

Return whether the default is to automatically bzip2 files

close()

Close all of the files and this directory

flush()

Flush the contents of all files to disk

get_filename(filename)

Return the full expanded filename for ‘filename’

get_path()

Return the full expanded path to this directory

is_closed()

Return whether or not the output files are closed

is_database(filename)

Return whether or not ‘filename’ is an open database

is_open()

Return whether or not the output files are open

open(filename: str, auto_bzip=None, mode='t', headers=None, sep=' ')

Open the file called ‘filename’ in the output directory, returning a handle to that file. Note that this will open the file once, and will return the already-open file handle on all subsequent calls.

Parameters

• filename (str) – The name of the file to open. This must be relative to the output directory, and within that directory. It is an error to try to open a file that is not contained within this directory.

• auto_bzip (bool) – Whether or not to open the file in auto-bzip (compression) mode. If this is True then the file will be automatically compressed as it is written. The filename will have ‘.bz2’ automatically appended so that this is clear. If this is False then the file will be written uncompressed. If ‘None’ is passed (the default) then the value of auto_bzip that was passed to the constructor of this OutputFiles will be used. Note that this flag is ignored if the file is already open.

• mode (str) – The mode of opening the file, e.g. ‘t’ for text mode, and ‘b’ for binary mode. The default is text mode

• headers (list[str] or plain str or function) – The headers to add to the top of the file, e.g. if it will contain column data. This will be written to the first line when the file is opened. If a list is passed, then this will be written joined together using ‘sep’. If a plain string is passed then this will be written. If this is a function then this function will be called with “FILE” as the argument. If nothing is passed then no headers will be written.

• sep (str) – The separator used for the headers (e.g. ” ” or “,” are good choices). By default things are space-separated

Returns

The handle to the open file

Return type

file

open_db(filename: str, auto_bzip=None, initialise=None)

Open up a SQLite3 database connection to a file called

‘filename’ in the output directory, returning the SQLite3 connection to the database. Note that this will open the database one, and will return the already-made connection on all subsequence calls.

filename: str

The name of the file containing the database to open. This must be relative to the output directory, and within that directory. It is an error to try to open a database that is not contained in this directory.

auto_bzip: bool

Whether or not to automatically compress the file using bzip2 when it is closed. The filename will automatically have ‘.bz2’ appended so that this is clear. If ‘None’ is passed (the default) then the value of ‘auto_bzip’ that was passed to the constructor of this OutputFiles will be used. Note that this flag is ignored if the database is already open

initialise: function

A function that is called to initialise the database the first time that it is opened. The function is called with the argument “CONN” (representing the sqlite3 database connection). Use this to create the tables that you need

open_subdir(dirname)

Create and open a sub-directory in this OutputFiles called ‘dirname’. This will inherit all properties, e.g. check_empty, auto_bzip etc from this OutputFiles

Parameters

dirname (str) – The name of the subdirectory to open

Returns

subdir – The open subdirectory

Return type

OutputFiles

output_dir()

Return the absolute path of the directory to which the output files will be written

static remove(path, prompt=<built-in function input>)

Remove the passed filename or directory

Parameters

• path (str) – The path to the file or directory or remove

• prompt – Prompt to use to ask the user - if None then no checks!