python for replicable analysis

notebook format

this code is written in a jupyter notebook (.ipynb file extension) and can be previewed on nbviewer

setup

conda/pip

it can be possible to install jupyter notebooks and all the data science dependencies using conda and/or pip - installing conda is apparently best done by downloading it from anaconda.com

python environments

assuming you have conda (or miniconda) installed, a way to set up python libraries in a portable way is to use python environments using a small analysis-environment.yml config file:

name: analysis-environment

channels:
  - defaults

dependencies:
  - python=3.6
  - jupyter=1.0.0
  - pandas=0.19.2
  - pyproj=1.9.5.1
  - plotly=2.5.1
  - geopandas=0.3.0
  - folium=0.2.1
  - xlrd=1.1.0

1. Save the environment yaml into your working directory
2. Create a virtual environment from the environment yaml - in the command prompt type conda env create -f analysis-environment.yml
3. Activate the environment with the name of your environment from the top of the environment yaml: In the command prompt type either: activate analysis-environment (windows) / source activate analysis-environment (macOS/linux) / conda activate analysis-environment (linux)
4. Use jupyter notebook --port 8889 if the standard 8888 port runs intro trouble with e.g kapersky

at the end of using the environment, use deactivate from within it to leave for the main (base) conda environment

conda update -n base conda

docker

if you have (ideally {linu|os}x and) docker, you can use it to download and run a self-contained, disposable setup - datacamp has all the details, but here's a couple of useful shell commands (there are loads :( )

there's a lot to learn, but docker may be good thing in a hotdesking environment

depending on how the install went, the docker service will need starting

systemctl start docker.service

download and run the premade data science container

docker run -it --rm -p 8888:8888 jupyter/datascience-notebook
jupyter notebook

open a shell inside the container, useful if you want system level access to contents

docker exec jupyter/datascience-notebook bash
docker exec -it 'name-of-container' bash
# later use the exit command to leave the container

docker commit is the way to saves a container state, next time you run the container, the commit will reprise; this command uses an image id (auto-generated as a random word_pair like nifty_bassi in the example below) and the container name

docker commit nifty_bassi jupyter/datascience-notebook:latest