Unit 10: Python Modules and Packages¶
Overview and Learning Objectives
This final unit of the PCP notebooks serves several purposes. First, we give a general introduction to Python modules and Python packages, which are fundamental concepts for organizing and making Python code available. Second, we introduce the Python package libpcp (that accompanies the PCP notebooks) and use this package as a concrete example for illustrating the Python concepts. At the same time, this unit (together with Unit 1) also documents the technical backbone underlying the PCP notebooks. Last but not least, we will also uncover in this unit the secret of where one can find the sample solutions for all exercises. In summary, we hope that the PCP notebooks help students naturally transition from learning about Python programming and signal processing to beginning independent research following good scientific practices. Another main motivation of the notebooks is to indirectly guide students to employ open-source tools for software development and reproducible research.
Python Modules¶
A Python module is basically a file with an extension .py containing Python code. The content of a module can be accessed with the import statement. As an example, we consider the file module.py contained in the folder libpcp. When the import statement is executed, the interpreter searches for module.py in a list of directories which specifies the search paths for modules. The variable sys.path (which is part of the module sys) yields the list of directories. It is initialized from the environment variable PYTHONPATH (plus an installation-dependent default). The list contained in sys.path can be extended using the function sys.path.append. The following example illustrates these concepts:
import sys
print(sys.path, '\n')
import os
sys.path.append('libpcp')
print(sys.path, '\n')
Once the directory of the module is in the search path, we can use the import statement. Let us come back to our example module.py, which has the following content:
import os
fn = os.path.join('libpcp', 'module.py')
with open(fn, 'r', encoding='utf-8') as stream:
content_text = stream.read()
print(content_text)
The following options import the module module or some of its elements:
import libpcp.module
result = libpcp.module.add(libpcp.module.a, libpcp.module.b, libpcp.module.c)
from libpcp.module import add
result = add(4, 5)
from libpcp.module import add as s
result = s(6)
from libpcp.module import *
result = add(a, b, c)
The file variable module.__file__ determines the path where the module was found. Furthermore, when a .py-file is imported as a module, Python sets the variable __name__ to the name of the module. Finally, the help-function shows the documentation of the specified module.
print('Directory of module:', libpcp.module.__file__)
print('Name of module:', libpcp.module.__name__)
print('=======================================')
help(libpcp.module)
Note that any .py-file that contains a module can also be executed as a Python script (e.g., module.py). In the case that a file is run as a script, the variable __name__ is set to the string '__main__'. This allows for placing additional statements in the module that are executed only when being run as a script (and not when imported as a module). For example, one can place these elements in a conditional (if) block as follows:
if (__name__ == '__main__'):
Statements only executed when run as a script
Python Packages¶
A Python package is a namespace that consists of a directory, which in turn may contain subdirectories (sub-packages) and files (modules). The naming convention follows the hierarchical file structure using dot notation. Opposed to normal directories, a package in Python typically contains a particular file called __init__.py (until Python 3.3, the existence of such a file was even mandatory). This file is automatically executed when the package (or a module in the package) is imported. For example, this allows for initializing package-specific data or for automatically importing specific modules from a package. Continuing our example above, the directory libpcp can be regarded as a package. The content of its initialization file __init__.py is output in the next code cell.
import os
fn = os.path.join('libpcp', '__init__.py')
with open(fn, 'r', encoding='utf-8') as stream:
content_text = stream.read()
print(content_text)
Using __init__.py allows a user to call the functions without the need to specify the module name the functions are contained in. In the above example, the string string_init as well as the function test_function_init (which are contained in the module module) can be directly accessed via libpcp. This is demonstrated by the next example.
import libpcp
print(libpcp.string_init)
libpcp.test_function_init()
libpcp.test_function_init('Hallo')
To call the other functions and variables in the module module, which are not specified in __init__.py, one needs to first import the module. The following code cell illustrates some options.
import libpcp.module
print(libpcp.module.string)
print(libpcp.module.string_init)
print(libpcp.string_init)
from libpcp import module
print(module.string)
print(module.string_init)
from libpcp import module as other_name
print(other_name.string)
print(other_name.string_init)
The Package libpcp¶
As an illustration, we included in the Python package libpcp most of the functions specified in the PCP notebooks. Furthermore, the libpcp also contains the solutions of the exercises in form of functions, which are called in the respective notebooks to show the results (yet, note the source code to be implemented). The package libpcp contains ten modules that correspond to the ten units of the PCP notebooks. In the following code cell, the package LibFMP is imported and its help page is displayed.
import libpcp
help(libpcp)
The __init__.py-file of libpcp was discussed before. Furthermore, using the module module as an example, we showed how to access the functions by first importing the module. In the following example, we import the module complex and show its content.
import libpcp.complex
help(libpcp.complex)
Next, we call the function libpcp.complex.exercise_mandelbrot_fancy, which produces a visualization of the Mandelbrot Set.
libpcp.complex.exercise_mandelbrot_fancy(save_file=True)
Documentation of Functions¶
For documenting the functions contained in libpcp, we follow standard Python style conventions as formulated in the Google Python Style Guide. Except for the solutions to the exercise, the other libpcp-functions are contained in some PCP notebook, where one finds a detailed explanation of the application, the underlying theory, and implementation issues. In the Docstring of a libpcp-function, we specify the PCP notebook where the function is explained and developed. Using the help-function, the following example shows the docstring of the function libpcp.complex.plot_vector. In particular, the information Notebook: PCP_06_complex.ipynb shows that this function is introduced in the PCP Notebook on Complex Numbers with the filename PCP_06_complex.ipynb.
from libpcp import complex leads to a collision of the Python method complex(), which returns a complex number. To avoid such collisions, one can use import libpcp.complex (as used above) or uses a renaming options (e.g., from libpcp import complex as pcp_complex).
help(libpcp.complex.plot_vector)
