Up and running with python, virtualenv, git, and jupyter notebooks
Let's get started with the nitty-gritty of setting up a new project. We need to grab some code from github, setup our Python environment, and introduce ourselves to the Python programming language.
Everything is ready for you at this link:
Ok, so that was a bit of a mess. For those who didn't attend our first meeting, we didn't get very far!
Let's take a minute to review the basics. Regardless of what operating system you have (Mac, Linux, Windows), we need three things:
This is our programming language. Many computers will have Python 2 already installed, but we don't want that one. It will be deprecated as of January 2020 and some of its syntax is not valid in Python 3. It is perfectly fine to have more than one version of Python on your system so ideally you will install Python 3.6+ in parallel with any older 2.x version you may have.
This is where a few (ok maybe many) of us got stuck during our first meeting. If you are still having trouble, read on.
For Mac OS X users:
jsumner$ python3 --version Python 3.7 jsumner$ which python3 /Library/Frameworks/Python.framework/Versions/3.7/bin/python3
You can also type 'python3' to start the Python terminal. Use 'quit()' to exit.
For Windows users:
H:\>python --version Python 3.7.2 H:\>where python C:\Users\jsumner\AppData\Local\Programs\Python\Python37-32\python.exe
We don't need to specify 'python3'; just 'py' will do!
Python is just the core language, we will also need a bunch of packages. These are useful pieces of Python code written by members of the Python community to handle specific tasks. For example, maybe we want our Python script to get data from a website, the requests package will do that for us. Obviously, if we want to use an interesting package, we will need to download it and stick it somewhere Python can find it.
You might consider installing all the packages alongside your native Python installation and this would work, at least for a little while. But there is something else you need to be aware of: dependencies. When we write a piece of code that imports packages, our code will likely be dependent on the exact version of the package we downloaded. If the requests package is upgraded by its authors, it is possible that some of its functionality will change and it won't work with our code anymore.
Of course, if you are the only person to work on your code this isn't a problem. You already downloaded the version that you need. But what if you want to collaborate with others? How do you ensure that they have the same setup on their machine as you do on your machine? This could turn into quite a nightmare; thankfully smart people have already solved this problem using virtual environments.
The idea is fairly straightforward: we create a small (possibly hidden) directory and within it place all the packages we need for a given project. If anyone else wants to work on that project, they just need a copy of that directory.
But we skipped over an important part: how do we find and get the packages in the first place? The answer is through the Python Package Index (Pypi) and its package manager pip. Let's use pip to install the package that will make virtual environments for us. Be careful to call pip3 and not pip (if you have Python 2 on your system)!
jsumner$ pip3 install virtualenv
And now we can make a new virtual environment to hold any packages we may need. We will make a new project directory, move into it and then call virtualenv. Note that we are passing an extra argument to let the environment know exactly which version of Python this virtualenv will use.
jsumner$ mkdir new-project jsumner$ cd new-project jsumner$ virtualenv -p /Library/Frameworks/Python.framework/Versions/3.7/bin/python3 venv_cnc
H:\>mkdir new-project H:\>cd new-project H:\new-project>virtualenv -p C:\Users\jsumner\AppData\Local\Programs\Python\Python37-32\python.exe venv_cnc
To activate our virtual environment:
jsumner$ source venv_cnc/bin/activate (venv_cnc)jsumner$
H:\new-project>venv_cnc\Scripts\activate (venv_cnc) H:\new-project>
And to deactivate:
(venv_cnc)jsumner$ deactivate jsumner$
(venv_cnc) H:\new-project>venv_cnc\Scripts\deactivate H:\new-project>
If you want to go a step further, you can also install virtualenvwrapper which makes life a little easier if you have a lot of virtual environments to manage.
We are making progress, even though we have no code yet. But before we enter even one character of code in this project, we need version control. It will keep track of all the changes we make to our code and, if we share our code, it will also keep track of all changes anyone else makes. I don't really have the words to express how important version control is, so I will just say this: Use it.
There are several version control tools out there to choose from, but git is the most popular.
jsumner$ git init jsumner$ ls -a . .. .git venv_cnc
H:\git init H:\dir /A . .. .git venv_cnc
On Mac OS X, there is a decent chance it will ask you to install command line developer tools. That isn't a terrible idea. Or you can download and install git here. If you need to install git for Windows, go here.
With git installed and our repo initialized, we have a slightly less empty project. The new .git is actually a directory that contains all the stuff required to track your changes. You never need to mess with it.
And that is the basic setup. Our workflow will look like this:
jsumner$ cd new-project jsumner$ source venv_cnc/bin/activate (venv_cnc)jsumner$ pip install <new package to install> (venv_cnc)jsumner$ python <name of python file to run> (venv_cnc)jsumner$ deactivate jsumner$
Note that on Mac we don't need to specify pip3 or python3 anymore as we tied Python 3 to this virtualenv when we created it. Try it! Also note that we haven't actually done anything with git yet, but it is there, ready.
I said there were only three things we needed. And that is true. Items 4 and 5 on this list are starred because they fall into the category of things we want but don't need in order work on a Python project. The first is a nice text editor. Sameer and I suggest to go for the smallest tool that does the job. I like Atom, Sameer likes Sublime. Both of them are bare bones but have a plethora of different plugins you can add to customize the work environment. They also integrate with git out-of-the-box, which is a nice feature. On the other end of the spectrum, you will find things like PyCharm which will take care of almost everything you need to set up and code in a Python project. We prefer a minimalist approach: it is certainly a better learning experience, but PyCharm is immensely popular, so feel free to give it a go.
For any purists out there who believe that a terminal/console is all you really need, there is vim.
This is your call: find a place where you like to code.
We definitely do not need jupyter notebooks to run or develop Python code... but it is a very nice place for both because it allows us:
Jupyter notebooks are fantastic for developing and debugging new code and are also an excellent way to publish reproducible research: your work and report are in one place for all to see.
More to come on jupyter...
Time to create a 'hello world' app to check if everything is installed properly. Python code is notoriously succinct, so this script won't take long. Open up your text editor of choice and create a new file called 'hello.py' with the following contents:
Save it in your project directory and run:
(venv_cnc)jsumner$ python hello.py Hello world!
That was easy! Now let's install our first package. Maybe we want to print our results to the terminal using different colours. Python doesn't support that, but the colorama package does!
(venv_cnc)jsumner$ pip install colorama Collecting colorama Downloading https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl Installing collected packages: colorama Successfully installed colorama-0.4.1
from colorama import init, Fore, Back, Style init() print(Fore.BLUE + Back.WHITE + 'Hello world!') print(Style.RESET_ALL + 'Back to normal')
When you run the script now, the output should be a little different than before. Notice how readable Python syntax is, it almost reads like plain English: "From the colorama package import the functions init, Fore, Back, and Style"...
As a last step, let's commit our progress to git.
(venv_cnc)jsumner$: git add . (venv_cnc)jsumner$: git commit -m "Hello world app created" [master (root-commit) aabbccdd] Hello world app created 1 file changed, 5 insertions(+) create mode 100644 hello.py (venv_cnc)jsumner$
Ok, that's enough for now. See you next time!