Tool Time: Profiling and Tracing of Python Code with Score-P

Friday, January 28, 2022

The parallel performance and instrumentation framework Score-P only supports Fortran and C/C++ applications by default. However, with the freely available Score-P Python module, it supports Python code as well.

Download and Installation

First, install Score-P. Be sure to use the configuration option “--enable-shared” and to check whether Score-P is using the gcc-compiler plugin for user function instrumentation. You might also want to install the Cube browser, which allows you to visualize the measured profile files. For more details on how to install Score-P and Cube, see the POP online training module “Installing POP Tools: Score-P, Scalasca, Cube”.

Next, make sure the command “scorep-config” is in your PATH. Then download the Score-P Python bindings and install them:

    git clone https://github.com/score-p/scorep_binding_python
    cd scorep_python_bindings/
    pip3 install .

Basic Usage

After installation, you can instrument and measure a specific Python script by

    python –m scorep <script.py>

If internally MPI functions are used in the Python script, also pass the “--mpp=mpi” flag to the call

    python –m scorep –-mpp=mpi <script.py>

By default, the script is profiled and the results are stored in a Cube profile (profile.cubex). Tracing is enabled (as usual) by setting the environment variable SCOREP_ENABLE_TRACING to true. The resulting OTF2 trace can then be found in the file traces.otf2.

By default, all Python functions are instrumented and measured. To reduce measurement overhead, selective manual instrumentation can be used. This can be activated by passing the flag “--noinstrumenter”:

    python –m scorep --noinstrumenter -–mpp=mpi <script.py>

Then, manual selective instrumentation can be enabled for specific regions in the code. For example, in the following script

    import numpy as np
    import scorep

    [...]
    with scorep.instrumenter.enable():
        c = np.dot(a,b)
    [...]

only the “dot” call (and anything inside it) is instrumented and measured. The necessary instrumentation statements are indicated in blue.

Python functions that should be instrumented and measured can be labeled with a decorator:

    @scorep.user.region()
    def do_something():
        [...]

More Information

A more detailed description of the Score-P Python bindings can be found at https://github.com/score-p/scorep_binding_python. There is also a small HowTo in the wiki, which also shows how the measurement results can be visualized and analyzed with the profile browser Cube and the trace visualizer Vampir. Finally, the paper “Advanced Python Performance Monitoring with Score-P” provides some technical background and information.

Tags: 
Tool time
Performance tools
Score-P