Getting started

Jonathon Sumner
Physics Department
Dawson College
Sameer Bhatnagar
Physics Department
Dawson College
March 8, 2020
March 8, 2020

Meeting 1

Up and running with python, virtualenv, git, and jupyter notebooks

Today's plan

Let's get started with the nitty-gritty of setting up a new project. We need to grab some code from github, setup our Python environment, and introduce ourselves to the Python programming language.

Everything is ready for you at this link:

Github repo

Postscript

Ok, so that was a bit of a mess. For those who didn't attend our first meeting, we didn't get very far!

Let's take a minute to review the basics. Regardless of what operating system you have (Mac, Linux, Windows), we need three things:

1. Python

This is our programming language. Many computers will have Python 2 already installed, but we don't want that one. It will be deprecated as of January 2020 and some of its syntax is not valid in Python 3. It is perfectly fine to have more than one version of Python on your system so ideally you will install Python 3.6+ in parallel with any older 2.x version you may have.

This is where a few (ok maybe many) of us got stuck during our first meeting. If you are still having trouble, read on.

For Mac OS X users:

Code sample: Checking for Python 3 on Mac OS X
jsumner$ python3 --version
Python 3.7
jsumner$ which python3
/Library/Frameworks/Python.framework/Versions/3.7/bin/python3

You can also type 'python3' to start the Python terminal. Use 'quit()' to exit.

For Windows users:

Code sample: Checking for Python 3 on Windows
H:\>python --version
Python 3.7.2
H:\>where python
C:\Users\jsumner\AppData\Local\Programs\Python\Python37-32\python.exe

We don't need to specify 'python3'; just 'py' will do!

2. A virtual environment

Python is just the core language, we will also need a bunch of packages. These are useful pieces of Python code written by members of the Python community to handle specific tasks. For example, maybe we want our Python script to get data from a website, the requests package will do that for us. Obviously, if we want to use an interesting package, we will need to download it and stick it somewhere Python can find it.

You might consider installing all the packages alongside your native Python installation and this would work, at least for a little while. But there is something else you need to be aware of: dependencies. When we write a piece of code that imports packages, our code will likely be dependent on the exact version of the package we downloaded. If the requests package is upgraded by its authors, it is possible that some of its functionality will change and it won't work with our code anymore.

Of course, if you are the only person to work on your code this isn't a problem. You already downloaded the version that you need. But what if you want to collaborate with others? How do you ensure that they have the same setup on their machine as you do on your machine? This could turn into quite a nightmare; thankfully smart people have already solved this problem using virtual environments.

The idea is fairly straightforward: we create a small (possibly hidden) directory and within it place all the packages we need for a given project. If anyone else wants to work on that project, they just need a copy of that directory.

But we skipped over an important part: how do we find and get the packages in the first place? The answer is through the Python Package Index (Pypi) and its package manager pip. Let's use pip to install the package that will make virtual environments for us. Be careful to call pip3 and not pip (if you have Python 2 on your system)!

Code sample: Using pip3 to install the virtualenv package (same for Mac and Windows)
jsumner$ pip3 install virtualenv

And now we can make a new virtual environment to hold any packages we may need. We will make a new project directory, move into it and then call virtualenv. Note that we are passing an extra argument to let the environment know exactly which version of Python this virtualenv will use.

Code sample: Creating a new virtual environment called 'venv_cnc' on Mac
jsumner$ mkdir new-project
jsumner$ cd new-project
jsumner$ virtualenv -p /Library/Frameworks/Python.framework/Versions/3.7/bin/python3 venv_cnc
Code sample: Creating a new virtual environment called 'venv_cnc' on Windows
H:\>mkdir new-project
H:\>cd new-project
H:\new-project>virtualenv -p C:\Users\jsumner\AppData\Local\Programs\Python\Python37-32\python.exe venv_cnc

To activate our virtual environment:

Code sample: Activating our virtual environment on Mac
jsumner$ source venv_cnc/bin/activate
(venv_cnc)jsumner$
Code sample: Activating our virtual environment on Windows
H:\new-project>venv_cnc\Scripts\activate
(venv_cnc) H:\new-project>

And to deactivate:

Code sample: Deactivating our virtual environment on Mac
(venv_cnc)jsumner$ deactivate
jsumner$
Code sample: Deactivating our virtual environment on Windows
(venv_cnc) H:\new-project>venv_cnc\Scripts\deactivate
H:\new-project>

If you want to go a step further, you can also install virtualenvwrapper which makes life a little easier if you have a lot of virtual environments to manage.

3. Version control

We are making progress, even though we have no code yet. But before we enter even one character of code in this project, we need version control. It will keep track of all the changes we make to our code and, if we share our code, it will also keep track of all changes anyone else makes. I don't really have the words to express how important version control is, so I will just say this: Use it.

There are several version control tools out there to choose from, but git is the most popular.

Code sample: Turn your project directory into a git repository on Mac
jsumner$ git init
jsumner$ ls -a
.	..	.git	venv_cnc
Code sample: Turn your project directory into a git repository on Windows
H:\git init
H:\dir /A
.	..	.git	venv_cnc

On Mac OS X, there is a decent chance it will ask you to install command line developer tools. That isn't a terrible idea. Or you can download and install git here. If you need to install git for Windows, go here.

With git installed and our repo initialized, we have a slightly less empty project. The new .git is actually a directory that contains all the stuff required to track your changes. You never need to mess with it.

And that is the basic setup. Our workflow will look like this:

Code sample: Typical workflow
jsumner$ cd new-project
jsumner$ source venv_cnc/bin/activate
(venv_cnc)jsumner$ pip install <new package to install>
(venv_cnc)jsumner$ python <name of python file to run>
(venv_cnc)jsumner$ deactivate
jsumner$

Note that on Mac we don't need to specify pip3 or python3 anymore as we tied Python 3 to this virtualenv when we created it. Try it! Also note that we haven't actually done anything with git yet, but it is there, ready.

4. Text editor or IDE*

I said there were only three things we needed. And that is true. Items 4 and 5 on this list are starred because they fall into the category of things we want but don't need in order work on a Python project. The first is a nice text editor. Sameer and I suggest to go for the smallest tool that does the job. I like Atom, Sameer likes Sublime. Both of them are bare bones but have a plethora of different plugins you can add to customize the work environment. They also integrate with git out-of-the-box, which is a nice feature. On the other end of the spectrum, you will find things like PyCharm which will take care of almost everything you need to set up and code in a Python project. We prefer a minimalist approach: it is certainly a better learning experience, but PyCharm is immensely popular, so feel free to give it a go.

For any purists out there who believe that a terminal/console is all you really need, there is vim.

This is your call: find a place where you like to code.

5. Jupyter notebook*

We definitely do not need jupyter notebooks to run or develop Python code... but it is a very nice place for both because it allows us:

Jupyter notebooks are fantastic for developing and debugging new code and are also an excellent way to publish reproducible research: your work and report are in one place for all to see.

More to come on jupyter...

Hello world

Time to create a 'hello world' app to check if everything is installed properly. Python code is notoriously succinct, so this script won't take long. Open up your text editor of choice and create a new file called 'hello.py' with the following contents:

Code sample: Our first Python script
print('Hello world!')

Save it in your project directory and run:

Code sample: Running our script from the terminal
(venv_cnc)jsumner$ python hello.py
Hello world!

That was easy! Now let's install our first package. Maybe we want to print our results to the terminal using different colours. Python doesn't support that, but the colorama package does!

Code sample: Installing our first package using pip
(venv_cnc)jsumner$ pip install colorama
Collecting colorama
  Downloading https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Installing collected packages: colorama
Successfully installed colorama-0.4.1
Code sample: Add a little spice to hello.py
from colorama import init, Fore, Back, Style

init()
print(Fore.BLUE + Back.WHITE + 'Hello world!')
print(Style.RESET_ALL + 'Back to normal')

When you run the script now, the output should be a little different than before. Notice how readable Python syntax is, it almost reads like plain English: "From the colorama package import the functions init, Fore, Back, and Style"...

As a last step, let's commit our progress to git.

Code sample: Our first git commit
(venv_cnc)jsumner$: git add .
(venv_cnc)jsumner$: git commit -m "Hello world app created"
[master (root-commit) aabbccdd] Hello world app created
 1 file changed, 5 insertions(+)
 create mode 100644 hello.py
(venv_cnc)jsumner$

Ok, that's enough for now. See you next time!