How to run python tests in PyCharm/IntelliJ

1

I have mine set up so that I can run python tests by right-clicking. I do have to compile from the command line first, though – it’s possible to add a gradle task to be done before the python test, but that wastes many seconds.

  1. Ensure that your Python interpreter is set up correctly. cmd ; > modules can bring up the module menu.
  2. From the top menu, go to Run > Edit Configurations
  3. From the left menu, navitage to Defaults > Python tests > Unittests
  4. edit your default to look like this, and hit save:
    image
    image.png2366×1732 404 KB

The environment variable hidden for me is

PYTHONPATH=$PYTHONPATH:/Users/tpoterba/spark-2.0.2-bin-hadoop2.7//python:/Users/tpoterba/spark-2.0.2-bin-hadoop2.7//python/lib/py4j-0.10.3-src.zip

We need this to add py4j and pyspark to the python path.

2

Update: I no longer use the former method. I now have the following lines in my ~/.profile (important that it is this dotfile in particular, intelliJ will not look at shell-specific dot files):

export HAIL_HOME=/Users/tpoterba/hail/hail
export PYTHONPATH="$HAIL_HOME/python:$PYTHONPATH"
JAR_PATH=/Users/tpoterba/hail/hail/build/libs/hail-all-spark.jar
export PYSPARK_SUBMIT_ARGS="--conf spark.driver.extraClassPath=$JAR_PATH --conf spark.executor.extraClassPath=$JAR_PATH --driver-memory 8G pyspark-shell"

Then I use the following configuration:

image
image.png1075×722 79.1 KB

This will require a reboot of intellij and possibly the OS.