Python

Python Tips and Tricks

Debugger

pdb is a python native debugger. To call the debugger with a program main.py just run the following command

pdb python -m main.py

You can introduce breakpoint manually in your code by typing command

breakpoint()

Alternatively you can introduce breakpoint directly in pdb shell by typing

>> bN

where N is a line number for execution, e.g. 100 Few useful commands:

Useful PDB commands

if one liner

Sometimes it is convenient to use one liner instead of long if-else statement:

device_type = 'cuda' if 'cuda' in device else 'cpu' # for later use in torch.autocast

F string

You may print the string with a nice format:

number = 2
print(f' The number is {number:2f}')

The cheat sheet for f-number formatting.

Cache

You can cache a number results of a deterministic function by using the following decorator from functools module

import functools 
@functools.lru_cache(maxsize = 4096)
def foo(x):
  .....

maxsize is size of cache. The argument of None removes the limit

Array enumeration

For numpy

a = np.array([[1, 2], [3, 4]])
for index, x in np.ndenumerate(a):
    print(index, x)

For usual python

a = np.array([[1, 2], [3, 4]])
for index, x in enumerate(a):
    print(index, x)

Partial Functions

You can define new function out of already defined one with some argument being applied.

from functools import partial

def foo(x,y) -> float:
   return x*y

new_func = partial(foo,y=5)

Zip

For iterating along the several list it is a good practice to use zip:

a = [1,2,3,4]
b = [5,6,7,8]
    for av, bv in zip(a, b):
        print(av)
        print(bv)

Multiprocessing

from multiprocessing import Pool
def run_with_mp_map(items, do_work, processes=None, chunksize=None):
    print(f"running using multiprocessing with {processes=}, {chunksize=}")
    start_t = time.perf_counter()
    with Pool(processes=processes) as pool:
        results = pool.imap(do_work, items, chunksize=chunksize)
    end_t = time.perf_counter()
    wall_duration = end_t - start_t
    print(f"it took: {wall_duration:.2f}s")
    return results

Context manager

 with open(filename, "w") as f:
        f.write("hello!\n")
    # close automatic, even if exception

PEP8 Checker

It is a good practice to follow PEP8 standard when writing code. In jupyter the PEP8 syntax checker may help highlight the errors. To load the checker into a jupyter notebook:

%load_ext pycodestyle_magic

To switch on the checker:

%pycodestyle_on

To switch off the checker:

%pycodestyle_off

Main function

It is a good practice to indicate whether your python file will be executed as a script by including "if __name__ == '__main__':" statement:

def main():
    pass
# Do Something:

if __name__ == '__main__':
    main()

Measure execution timing

In jupyter it is possible to use "magic" command to measure execution time. For exmple, the code below run function foo for 1000 times and returns average execution time

%timeit n 1000 -r1 foo(x)

Alternatively, the time can be measured directly by including the following code:

import time
start = time.perf_counter()
time.sleep(1)
end = time.perf_counter()
print(end - start)