python-foundation.Rmd

---
title: "Python Foundation for Spatial Analysis (Full Course)"
author: "Ujaval Gandhi"
subtitle: A gentle introduction to Python programming with a focus on spatial data.
output:
  # pdf_document:
  #   toc: yes
  #   toc_depth: '3'
  #   latex_engine: xelatex 
  html_document:
    df_print: paged
    highlight: pygments
    toc: yes
    toc_depth: 2
    includes:
      in_header:
        - copy-code.html
        - GA_Script.html
      after_body: comment.html
fontsize: 12pt
header-includes:
- \usepackage{fancyhdr}
- \pagestyle{fancy}
- \renewcommand{\footrulewidth}{0.4pt}
- \fancyhead[LE,RO]{\thepage}
- \geometry{left=1in,top=0.75in,bottom=0.75in}
- \fancyfoot[CE,CO]{{\includegraphics[height=0.5cm]{images/cc-by-nc.png}} Ujaval Gandhi
  http://www.spatialthoughts.com}
classoption: a4paper
---
\newpage
 
***

```{r echo=FALSE, fig.align='center', out.width='75%', out.width='250pt'}
knitr::include_graphics('images/spatial_thoughts_logo.png')
```

***

\newpage

# Introduction 

This class covers Python from the very basics. Suitable for GIS practitioners with no programming background or python knowledge. The course will introduce participants to basic programming concepts, libraries for spatial analysis, geospatial APIs and techniques for building spatial data processing pipelines.

[![Watch the video](https://img.youtube.com/vi/rHgsslH-_y0/mqdefault.jpg)](https://www.youtube.com/watch?v=rHgsslH-_y0&list=PLppGmFLhQ1HJspXSA0asH9kw1OhlLrxHT&index=1){target="_blank"}

[Watch the Video ↗](https://www.youtube.com/watch?v=rHgsslH-_y0&list=PLppGmFLhQ1HJspXSA0asH9kw1OhlLrxHT&index=1){target="_blank"}

[Access the Presentation ↗](https://docs.google.com/presentation/d/125LV0qR47S_OGaZZr5Zv-7lbWfdZ5pNo3iFm6IOX7uI/edit?usp=sharing){target="_blank"}


# Get the Data Package

The code examples in this class use a variety of datasets. All the required datasets and Jupyter notebooks are supplied to you in the ``python_foundation.zip`` file. Unzip this file to a directory - preferably to the ``<home folder>/Downloads/python_foundation/`` folder. 

Download [python_foundation.zip](https://github.com/spatialthoughts/courses/releases/download/data/python_foundation.zip){target="_blank"}.

> Note: Certification and Support are only available for participants in our paid instructor-led classes.

# Get the Course Videos

The course is accompanied by a set of videos covering the all the modules. These videos are recorded from our live instructor-led classes and are edited to make them easier to consume for self-study. We have 2 versions of the videos:

### YouTube

We have created a YouTube Playlist with separate videos for each notebook and exercise to enable effective online-learning. [Access the YouTube Playlist &#8599;](https://www.youtube.com/playlist?list=PLppGmFLhQ1HJspXSA0asH9kw1OhlLrxHT){target="_blank"}

### Vimeo

We are also making combined full-length video for each module available on Vimeo. These videos can be downloaded for offline learning. [Access the Vimeo Playlist &#8599;](https://vimeo.com/showcase/11034058){target="_blank"}


# Installation and Setting up the Environment 

## Install Conda

Follow our step-by-step [Conda Installation Guide](install-conda.html){target="_blank"} to install Miniconda for your operating system.

## Create an Environment and Install Packages

Once conda has been installed and configured, we will create a new *environment* and install the required packages. An environment is an isolated space where you will install required packages. Many packages may contain conflicting requirements, and it is a good practice to create a new environment for each of your projects and install only the required packages there. We will now type commands in a terminal to create a new environment. 

1, *(Windows users)* Search for Anaconda Powershell Prompt in the Start Menu and launch it. *(Mac/Linux users)*: Open a Terminal window.  Enter the command below and press *Enter* to create your new environment.

```
conda create --name python_foundation -y
```

```{r echo=FALSE, fig.align='center', out.width='75%'}
knitr::include_graphics('images/python_foundation/environment1.png')
```

2. Now that the environment is created, you need to activate it. Type the command below and press *Enter*. Once the environment activates, the ``(base)`` will change to ``(python_foundation)``. 

```
conda activate python_foundation
```

```{r echo=FALSE, fig.align='center', out.width='75%'}
knitr::include_graphics('images/python_foundation/environment2.png')
```

3. Now we are ready to install the required packages using the `conda install` command. We can now install other required packages for this class. Run the command below to install `geopandas`, `geopy`, `rioxarray`, `matplotlib` and `jupyterlab` packages. We use the [conda-forge](https://conda-forge.org/docs/user/introduction.html){target="_blank"} channel which has a broader range of packages available than the default channel.

```
conda install --channel conda-forge geopandas geopy rioxarray matplotlib jupyterlab -y
```

```{r echo=FALSE, fig.align='center', out.width='75%'}
knitr::include_graphics('images/python_foundation/environment3.png')
```

4. Once the command finishes, you should see a screen such as below.

```{r echo=FALSE, fig.align='center', out.width='75%'}
knitr::include_graphics('images/python_foundation/environment4.png')
```

5. Your Python environment is now ready. Launch the *JupyterLab* application using the command below. This will initiate and run a local server in your system and opens in your default browser. 

> Note: Do not close your anaconda prompt after *JupyterLab* opens up. You need to keep it running as long as you want to use JupyterLab.

```
jupyter-lab
```

```{r echo=FALSE, fig.align='center', out.width='75%'}
knitr::include_graphics('images/python_foundation/environment5.png')
```

Note that JupyterLab application can browse the files only on the Drive from where it was launched from. If your data is stored on a different drive, you will need an additional step to switch to that drive before launching Jupyterlab.

*Windows*

On the command prompt, type the drive letter followed by `:` and press *Enter* to switch to the drive.
```
D:
```
```
jupyter-lab
```

```{r echo=FALSE, fig.align='center', out.width='75%'}
knitr::include_graphics('images/python_foundation/environment5a.png')
```

*Mac/Linux*

Check the drives mounted on your system by entering `ls /Volumes`. After that use `cd` command to switch to the drive.
```
cd /Volumes/<NameofYourDrive>
```
```
jupyter-lab
```

6. A new browser tab will open with an instance of JupyterLab. If prompted to *Select Kernel*, choose `Python 3 (ipykernel)` option. Check the *Always start the preferred kernel* box and click *Select*.

```{r echo=FALSE, fig.align='center', out.width='75%'}
knitr::include_graphics('images/python_foundation/environment6.png')
```

7. Enter the following statements in the first cell and click the *Run* button. **If nothing happens - it means your installation was successful!**. Your environment is now ready for the course. If you get an *ImportError*, repeat the installation steps carefully again.

```
import geopandas
import geopy
import rioxarray
```

```{r echo=FALSE, fig.align='center', out.width='75%'}
knitr::include_graphics('images/python_foundation/environment7.png')
```


## Debugging Python Installation Errors

The following section describes common installation errors with suggested fixes.

#### Package Conflicts

Sometimes if you have older version of packages in your base environment, it can cause conflicts when you try installing other packages. You can try updating your base environment and try again. The following sequence of commands will delete the python_foundation environment and update packages in your base environment.

```
conda deactivate
conda env remove -n python_foundation
conda update --all
```

Follow the installation steps again to create the environment and required packages.

#### OpenSSL Error

On many Windows systems, you may get an error such **CondaSSLError: OpenSSL appears to be unavailable on this machine. OpenSSL is required to download and install packages**. This means the OpenSSL module is missing. Please download and install the [Win32/Win64 OpenSSL packages](https://slproweb.com/products/Win32OpenSSL.html){target="_blank"} and try again.

If the error persists, you can manually fix the issue by copying the required DLL files in the correct place as [described in this issue](https://github.com/conda/conda/issues/11982#issuecomment-1285538983){target="_blank"}.

#### RTree spatialindex Error

When importing GeoPandas, you may see an error **Could not find or load spatialindex_c-64.dll**. This error is likely caused by a corrupted installation. This error is easily fixed by deleting the conda environment and reinstalling geopandas. Run the following sequence of commands to delete the environment.

```
conda deactivate
conda env remove -n python_foundation
```

Follow the installation steps again to create the environment and required packages.

# Using AI-Coding Assistance

LLM-based coding assistants are useful in writing and debugging Python code. There are many models and agents available and you may use your favorite coding assistant in this course. We recommend using Claude for this course. The exercises and assignments are tested using the free version (using Sonnet 4.6 model). You can sign-up for a free account at [Claude AI](https://claude.ai/){target="_blank"}.

# Using Jupyter Notebooks

[![Watch the Video](https://img.youtube.com/vi/xsId8Yt-S-Y/mqdefault.jpg)](https://www.youtube.com/watch?v=xsId8Yt-S-Y&list=PLppGmFLhQ1HJspXSA0asH9kw1OhlLrxHT&index=3){target="_blank"}

[Watch the Video &#8599;](https://www.youtube.com/watch?v=xsId8Yt-S-Y&list=PLppGmFLhQ1HJspXSA0asH9kw1OhlLrxHT&index=3){target="_blank"}

Your class data package contain multiple Jupyter notebooks containing code and exercises for this class. 

1. Launch the *JupyterLab* application. It will open your Web Browser and load the application in a new tab. From the left-hand panel, navigate to the directory where you extracted the data package.

```{r echo=FALSE, fig.align='center', out.width='75%'}
knitr::include_graphics('images/python_foundation/setup1.png')
```

2. Jupyter notebooks have a `.ipynb` extensions. Double-click on a notebook file to open it. Code in the notebook is executed cell-by-cell. You can select a cell and click the *Run* button to execute the code and see the output.

```{r echo=FALSE, fig.align='center', out.width='75%'}
knitr::include_graphics('images/python_foundation/setup2.png')
```

3. At the end of each notebook, you will find an exercise. Before adding a new cell and attempting to complete the exercise, make sure you go to **Run &rarr; Run All Cells** to execute all the code in the notebook. Doing this will ensure all the required variables are avalable to you to use in the exervise.

```{r echo=FALSE, fig.align='center', out.width='75%'}
knitr::include_graphics('images/python_foundation/setup3.png')
```

\newpage

----

Open the notebook named ``01_variables.ipynb``.

[![Watch the Video](https://img.youtube.com/vi/JPZOxlOLTKU/mqdefault.jpg)](https://www.youtube.com/watch?v=JPZOxlOLTKU&list=PLppGmFLhQ1HJspXSA0asH9kw1OhlLrxHT&index=4){target="_blank"}

[Watch the Video &#8599;](https://www.youtube.com/watch?v=JPZOxlOLTKU&list=PLppGmFLhQ1HJspXSA0asH9kw1OhlLrxHT&index=4){target="_blank"}

----

```{r child='python-foundation-output/01_variables.md'}
```

\newpage

Open the notebook named ``02_data_structures.ipynb``.

[![Watch the Video](https://img.youtube.com/vi/BxKrHhGAodY/mqdefault.jpg)](https://www.youtube.com/watch?v=BxKrHhGAodY&list=PLppGmFLhQ1HJspXSA0asH9kw1OhlLrxHT&index=6){target="_blank"}

[Watch the Video &#8599;](https://www.youtube.com/watch?v=BxKrHhGAodY&list=PLppGmFLhQ1HJspXSA0asH9kw1OhlLrxHT&index=6){target="_blank"}


----

```{r child='python-foundation-output/02_data_structures.md'}
```

\newpage

Open the notebook named ``03_string_operations.ipynb``.

[![Watch the Video](https://img.youtube.com/vi/RNp5sF-ZEkM/mqdefault.jpg)](https://www.youtube.com/watch?v=RNp5sF-ZEkM&list=PLppGmFLhQ1HJspXSA0asH9kw1OhlLrxHT&index=8){target="_blank"}

[Watch the Video &#8599;](https://www.youtube.com/watch?v=RNp5sF-ZEkM&list=PLppGmFLhQ1HJspXSA0asH9kw1OhlLrxHT&index=8){target="_blank"}

----

```{r child='python-foundation-output/03_string_operations.md'}
```

\newpage
Open the notebook named ``04_loops_and_conditionals.ipynb``.

[![Watch the Video](https://img.youtube.com/vi/NkKuzVkYOzk/mqdefault.jpg)](https://www.youtube.com/watch?v=NkKuzVkYOzk&list=PLppGmFLhQ1HJspXSA0asH9kw1OhlLrxHT&index=10){target="_blank"}

[Watch the Video &#8599;](https://www.youtube.com/watch?v=NkKuzVkYOzk&list=PLppGmFLhQ1HJspXSA0asH9kw1OhlLrxHT&index=10){target="_blank"}

----

```{r child='python-foundation-output/04_loops_and_conditionals.md'}
```

\newpage

Open the notebook named ``05_functions.ipynb``.

[![Watch the Video](https://img.youtube.com/vi/rcO29Q5Nd3Y/mqdefault.jpg)](https://www.youtube.com/watch?v=rcO29Q5Nd3Y&list=PLppGmFLhQ1HJspXSA0asH9kw1OhlLrxHT&index=12){target="_blank"}

[Watch the Video &#8599;](https://www.youtube.com/watch?v=rcO29Q5Nd3Y&list=PLppGmFLhQ1HJspXSA0asH9kw1OhlLrxHT&index=12){target="_blank"}

[Access the Presentation &#8599;](https://docs.google.com/presentation/d/1fFCOHSEPnGamcymzTZGE3Sc9lMJGo4tQ063leEyZMU8/edit?usp=sharing){target="_blank"}

----

```{r child='python-foundation-output/05_functions.md'}
```

\newpage

Open the notebook named ``06_standard_library.ipynb``.

[![Watch the Video](https://img.youtube.com/vi/kgY2npOQFM8/mqdefault.jpg)](https://www.youtube.com/watch?v=kgY2npOQFM8&list=PLppGmFLhQ1HJspXSA0asH9kw1OhlLrxHT&index=14){target="_blank"}

[Watch the Video &#8599;](https://www.youtube.com/watch?v=kgY2npOQFM8&list=PLppGmFLhQ1HJspXSA0asH9kw1OhlLrxHT&index=14){target="_blank"}

----

```{r child='python-foundation-output/06_standard_library.md'}
```

\newpage

Open the notebook named ``07_third_party_modules.ipynb``.

[![Watch the Video](https://img.youtube.com/vi/BqBPWQt9PL0/mqdefault.jpg)](https://www.youtube.com/watch?v=BqBPWQt9PL0&list=PLppGmFLhQ1HJspXSA0asH9kw1OhlLrxHT&index=16){target="_blank"}

[Watch the Video &#8599;](https://www.youtube.com/watch?v=BqBPWQt9PL0&list=PLppGmFLhQ1HJspXSA0asH9kw1OhlLrxHT&index=16){target="_blank"}

----

```{r child='python-foundation-output/07_third_party_modules.md'}
```

\newpage

Open the notebook named ``08_using_web_apis.ipynb``.

[![Watch the Video](https://img.youtube.com/vi/WBip-hbQYnM/mqdefault.jpg)](https://www.youtube.com/watch?v=WBip-hbQYnM&list=PLppGmFLhQ1HJspXSA0asH9kw1OhlLrxHT&index=18){target="_blank"}

[Watch the Video &#8599;](https://www.youtube.com/watch?v=WBip-hbQYnM&list=PLppGmFLhQ1HJspXSA0asH9kw1OhlLrxHT&index=18){target="_blank"}

----

```{r child='python-foundation-output/08_using_web_apis.md'}
```

\newpage

Open the notebook named ``assignment.ipynb``.

[![Watch the Video](https://img.youtube.com/vi/HaC3-y7c_F8/mqdefault.jpg)](https://www.youtube.com/watch?v=HaC3-y7c_F8&list=PLppGmFLhQ1HJspXSA0asH9kw1OhlLrxHT&index=20){target="_blank"}

[Watch the Video &#8599;](https://www.youtube.com/watch?v=HaC3-y7c_F8&list=PLppGmFLhQ1HJspXSA0asH9kw1OhlLrxHT&index=20){target="_blank"}

----

```{r child='python-foundation-output/assignment.md'}
```


----

\newpage

----

Open the notebook named ``common_errors.ipynb``.

[![Watch the Video](https://img.youtube.com/vi/nMS_al2za3U/mqdefault.jpg)](https://www.youtube.com/watch?v=nMS_al2za3U&list=PLppGmFLhQ1HJspXSA0asH9kw1OhlLrxHT&index=21){target="_blank"}

[Watch the Video &#8599;](https://www.youtube.com/watch?v=nMS_al2za3U&list=PLppGmFLhQ1HJspXSA0asH9kw1OhlLrxHT&index=21){target="_blank"}

----

```{r child='python-foundation-output/common_errors.md'}
```

----

\newpage

Open the notebook named ``09_reading_files.ipynb``.

[![Watch the Video](https://img.youtube.com/vi/v1dbrcIKL_E/mqdefault.jpg)](https://www.youtube.com/watch?v=v1dbrcIKL_E&list=PLppGmFLhQ1HJspXSA0asH9kw1OhlLrxHT&index=22){target="_blank"}

[Watch the Video &#8599;](https://www.youtube.com/watch?v=v1dbrcIKL_E&list=PLppGmFLhQ1HJspXSA0asH9kw1OhlLrxHT&index=22){target="_blank"}

----

```{r child='python-foundation-output/09_reading_files.md'}
```

\newpage

Open the notebook named ``10_reading_csv_files.ipynb``.

[![Watch the Video](https://img.youtube.com/vi/rgALt7FzOMY/mqdefault.jpg)](https://www.youtube.com/watch?v=rgALt7FzOMY&list=PLppGmFLhQ1HJspXSA0asH9kw1OhlLrxHT&index=24){target="_blank"}

[Watch the Video &#8599;](https://www.youtube.com/watch?v=rgALt7FzOMY&list=PLppGmFLhQ1HJspXSA0asH9kw1OhlLrxHT&index=24){target="_blank"}

----

```{r child='python-foundation-output/10_reading_csv_files.md'}
```

\newpage

Open the notebook named ``11_working_with_pandas.ipynb``.

[![Watch the Video](https://img.youtube.com/vi/rgALt7FzOMY/mqdefault.jpg)](https://www.youtube.com/watch?v=lYqoohizNNw&list=PLppGmFLhQ1HJspXSA0asH9kw1OhlLrxHT&index=26){target="_blank"}

[Watch the Video &#8599;](https://www.youtube.com/watch?v=lYqoohizNNw&list=PLppGmFLhQ1HJspXSA0asH9kw1OhlLrxHT&index=26){target="_blank"}

[Access the Presentation &#8599;](https://docs.google.com/presentation/d/1fyNe15yhbMq_435yaHRZefKzos4Vpyf54mqNfhC_flQ/edit?usp=sharing){target="_blank"}

----

```{r child='python-foundation-output/11_working_with_pandas.md'}
```

\newpage

Open the notebook named ``12_working_with_geopandas.ipynb``.

[![Watch the Video](https://img.youtube.com/vi/wsqJQX8sZaA/mqdefault.jpg)](https://www.youtube.com/watch?v=wsqJQX8sZaA&list=PLppGmFLhQ1HJspXSA0asH9kw1OhlLrxHT&index=28){target="_blank"}

[Watch the Video &#8599;](https://www.youtube.com/watch?v=wsqJQX8sZaA&list=PLppGmFLhQ1HJspXSA0asH9kw1OhlLrxHT&index=28){target="_blank"}

----

```{r child='python-foundation-output/12_working_with_geopandas.md'}
```

\newpage

Open the notebook named ``13_creating_spatial_data.ipynb``.

[![Watch the Video](https://img.youtube.com/vi/6fheYLAGuzc/mqdefault.jpg)](https://www.youtube.com/watch?v=6fheYLAGuzc&list=PLppGmFLhQ1HJspXSA0asH9kw1OhlLrxHT&index=30){target="_blank"}

[Watch the Video &#8599;](https://www.youtube.com/watch?v=6fheYLAGuzc&list=PLppGmFLhQ1HJspXSA0asH9kw1OhlLrxHT&index=30){target="_blank"}

----

```{r child='python-foundation-output/13_creating_spatial_data.md'}
```

\newpage

Open the notebook named ``14_introduction_to_numpy.ipynb``.

[![Watch the Video](https://img.youtube.com/vi/HfvkZFsx3ys/mqdefault.jpg)](https://www.youtube.com/watch?v=HfvkZFsx3ys&list=PLppGmFLhQ1HJspXSA0asH9kw1OhlLrxHT&index=32){target="_blank"}

[Watch the Video &#8599;](https://www.youtube.com/watch?v=HfvkZFsx3ys&list=PLppGmFLhQ1HJspXSA0asH9kw1OhlLrxHT&index=32){target="_blank"}

----

```{r child='python-foundation-output/14_introduction_to_numpy.md'}
```

\newpage

Open the notebook named ``15_working_with_rioxarray.ipynb``.

[![Watch the Video](https://img.youtube.com/vi/N3sePRkzK9s/mqdefault.jpg)](https://www.youtube.com/watch?v=N3sePRkzK9s&list=PLppGmFLhQ1HJspXSA0asH9kw1OhlLrxHT&index=34){target="_blank"}

[Watch the Video &#8599;](https://www.youtube.com/watch?v=N3sePRkzK9s&list=PLppGmFLhQ1HJspXSA0asH9kw1OhlLrxHT&index=34){target="_blank"}

----

```{r child='python-foundation-output/15_working_with_rioxarray.md'}
```

\newpage

# Writing Standalone Python Scripts

[![Watch the Video](https://img.youtube.com/vi/38GeIR4zom0/mqdefault.jpg)](https://www.youtube.com/watch?v=38GeIR4zom0&list=PLppGmFLhQ1HJspXSA0asH9kw1OhlLrxHT&index=36){target="_blank"}

[Watch the Video &#8599;](https://www.youtube.com/watch?v=38GeIR4zom0&list=PLppGmFLhQ1HJspXSA0asH9kw1OhlLrxHT&index=36){target="_blank"}


So far we have used Jupyter Notebooks to write and execute Python code. A notebook is a great choice to interactively explore, visualize and document workflows. But they are not suited for writing scripts for automation. If you have tasks that are long running or want to execute certain tasks on a schedule, you have to write scripts in a standalone `.py` file and run it from a Terminal or Console.

## Get a Text Editor

Any kind of software development requires a good text editor. If you already have a favorite text editor or an IDE (Integrated Development Environment), you may use it for this course. Otherwise, each platform offers a wide variety of free or paid options for text editors. Choose the one that fits your needs.

Below are my recommendations editors that are simple to use for beginners.

- Windows: [Notepad++](https://notepad-plus-plus.org/downloads/){target="_blank"} is a good free editor for windows. Download and install the Notepad++ editor.
- Mac: [TextMate](https://macromates.com/){target="_blank"} is an open-source editor for Mac that is currently available for free.

Once you get some experience with Python, we suggest graduating to a professional editor. We recommend [Visual Studio Code (VS Code)](https://code.visualstudio.com/){target="_blank"} - which can work with both text files and Jupyter Notebooks and has in-editor support for AI-Assisted coding.

## Writing a Script

Copy the following code and paste it in your text editor. Browse to the data package directory and save the file as `get_distance.py`. Make sure that there is no `.txt` extension at the end. 

```{python eval=FALSE, code=readLines('code/python_foundation/get_distance.py')}
```

## Executing a Script

### Windows

1. Open Command Prompt/Terminal.

2. Navigate to the directory containing the script using the `cd` command.

```
cd Downloads\python_foundation
```

```{r echo=FALSE, fig.align='center', out.width='75%'}
knitr::include_graphics('images/python_foundation/win_script2.png')
```

3. Run the script using the `python` command. The script will run and print the distance.

```
python get_distance.py
```

```{r echo=FALSE, fig.align='center', out.width='75%'}
knitr::include_graphics('images/python_foundation/win_script3.png')
```

### Mac and Linux

1. Open a Terminal Window.

```{r echo=FALSE, fig.align='center', out.width='75%'}
knitr::include_graphics('images/python_foundation/mac_script1.png')
```

2. Switch to the correct conda environment. 

```
conda activate python_foundation
```

```{r echo=FALSE, fig.align='center', out.width='75%'}
knitr::include_graphics('images/python_foundation/mac_script2.png')
```

3. Navigate to the directory containing the script using the `cd` command.

```
cd Downloads/python_foundation
```

```{r echo=FALSE, fig.align='center', out.width='75%'}
knitr::include_graphics('images/python_foundation/mac_script3.png')
```

4. Run the script using the `python` command. The script will run and print the distance.

> If you have multiple python installations on your system, you will have to pick the right Python binary. If the command fails, try `python3.7` instead of just `python` in the command below. The script will run and print the distance.

```
python get_distance.py
```

```{r echo=FALSE, fig.align='center', out.width='75%'}
knitr::include_graphics('images/python_foundation/mac_script4.png')
```

\newpage

# What next?

To achieve mastery in programming, you need to practice writing code and solve many diverse type of problems. I strongly recommend picking up a project and implementing it in Python to improve your skills. You can also take other courses to learn aspects of Python and spatial data processing.

[![Watch the Video](https://img.youtube.com/vi/a4oDDxQyaus/mqdefault.jpg)](https://www.youtube.com/watch?v=a4oDDxQyaus&list=PLppGmFLhQ1HJspXSA0asH9kw1OhlLrxHT&index=37){target="_blank"}

[Watch the Video &#8599;](https://www.youtube.com/watch?v=a4oDDxQyaus&list=PLppGmFLhQ1HJspXSA0asH9kw1OhlLrxHT&index=37){target="_blank"}

## Do a Project

An ideal project is something that is both critical to your work and also interesting to you. We encourage you to pick a project in your domain that helps you automate a workflow or solves a problem otherwise not possible.

You may want to check out this [Awesome Geospatial](https://github.com/sacridini/Awesome-Geospatial/blob/master/README.md#python){target="_blank"} that has a curated list of geospatial Python packages that you can use for your project.


### Example Projects

Below are some projects that our previous participants worked on. These are good examples of what complete beginners can achieve after completing this course. Use these as inspiration to come up with your own project and start working on it!

* [Data-driven Framework for Quantifying Helicopter Demand](https://colab.research.google.com/github/SiloSolutionist/Abitibi-QC-Heli-Risk-Analysis/blob/main/Abitibi_QC_Heli_Logistics_Risk_Analysis.ipynb){target="_blank"} by Kayla Helt
* [Karnataka Assembly Election Dashboard](https://karnatakaassemblyresultsdashboard.netlify.app/){target="_blank"} by Srinidhi HV
* [Geocoding Addresses from a spreadsheet](https://colab.research.google.com/github/KlausAgerskov/Python_geocode_internships/blob/master/klaus_danish_geocode_final.ipynb){target="_blank"} by Klaus Agerskov
* [Retrieving and visualizing data from OpenStreetMap](https://colab.research.google.com/github/luicarboni/My_Projects/blob/master/Starting_with_OSMnx.ipynb){target="_blank"} by Luigi Carboni
* [Route Optimization](https://colab.research.google.com/github/amitkb3/python_spatial_analysis_final_project/blob/master/PythonSpatialAnalysisCourse_FinalProject.ipynb){target="_blank"} by Amit Bothra
* [Unsupervised Clustering of industrial locations](https://github.com/AshishA2812/Python_Spatial_Project1/blob/master/DBscan%20clustering_AshishA.ipynb){target="_blank"} by Ashish Andhale
* [Mapping VIIRS Active Fires in South America](https://nbviewer.jupyter.org/github/acoiman/mapping_wildfires/blob/master/mapping_wildfires.ipynb){target="_blank"} by Abraham Coiman
* [Analyzing Music using Spotify API](https://colab.research.google.com/github/deepikajpsharma/Spotify/blob/main/Spotify%20data%20analysis.ipynb){target="_blank"} by Deepika Sharma
* [Analyzing Primate Distribution in Protected Areas](https://github.com/ulfboge/galago-wdpa/tree/main){target="_blank"} by Johan Karlsson
* [Finding Solar Sites from OpenStreetMap](https://colab.research.google.com/github/cosmith57/Python-Foundations-for-Spatial-Analysis/blob/main/Colby_Smith_PFSA.ipynb){target="_blank"} by Colby Smith
* [Saline System Visualizer](https://colab.research.google.com/github/valegio/saline_map/blob/main/saline_map.ipynb){target="_blank"} by Valentina Giovanetti


### Project Ideas

If you do not have a project of your own, here are some recommendation for projects that are a good fit for beginners, but are still challenging to help you become proficient. You can try to work on as many of the following problems as you like.

- **Geo-processing**: I have several intermediate level geoprocessing tutorials that use QGIS. You can take any of the following tutorials and implement it using libraries such as *geopandas* and *rioxarray*.
    - [Performing Table Joins](https://www.qgistutorials.com/en/docs/3/performing_table_joins.html){target="_blank"} 
    - [Performing Spatial Joins](https://www.qgistutorials.com/en/docs/3/performing_spatial_joins.html){target="_blank"} 
    - [Nearest Neighbor Analysis](https://www.qgistutorials.com/en/docs/3/nearest_neighbor_analysis.html){target="_blank"}
    - [Sampling Raster Data using Points or Polygons](https://www.qgistutorials.com/en/docs/3/sampling_raster_data.html){target="_blank"}
- **Network Analysis**: Use the [osmnx package](https://github.com/gboeing/osmnx){target="_blank"} to do network analysis with OpenStreetMap data and Python. Start by looking at [examples](https://github.com/gboeing/osmnx-examples/tree/master/notebooks){target="_blank"}
- **Visualization and Interactive Mapping**: Use the plotting library [geoplot](https://residentmario.github.io/geoplot/index.html){target="_blank"} or mapping library [folium](https://python-visualization.github.io/folium/){target="_blank"} to create an interactive map. See this [example](https://medium.com/analytics-vidhya/interactive-choropleth-map-in-python-using-folium-4e1479d9e568){target="_blank"}
- **Text Processing using LLMs**: Use Large-Language Models to process text and build applications. See our tutorial [Natural Language Processing using OpenAI API](https://www.geopythontutorials.com/notebooks/openai_mapping_news_articles.html){target="_blank"} for mapping news articles.
- **Image Segmentation**: [Segment Geospatial](https://samgeo.gishub.org/){target="_blank"} is a Python package to extract features from satellite imagery. Use this to extract objects from satellite or drone imagery. See our tutorials for [Extracting Farm Boundaries](https://www.geopythontutorials.com/notebooks/samgeo_farm_boundary_extraction.html){target="_blank"} and [Detecting Mine Perimeter](https://www.geopythontutorials.com/notebooks/samgeo_mine_perimeter_detection.html){target="_blank"}.


## Keep Learning

Here are some recommendations for courses and tutorials you could  do after you have completed this course.

* Spatial Thoughts' [Geospatial Python Tutorials](https://www.geopythontutorials.com/){target="_blank"} covering step-by-step tutorials for GeoPandas, XArray and more.
* Spatial Thoughts' Intermediate Python course [Mapping and Data Visualization with Python](https://spatialthoughts.com/courses/python-dataviz/){target="_blank"}
* University of Helsinki's Free [Geo-Python Course](https://geo-python-site.readthedocs.io/en/latest/){target="_blank"}
* University of Helsinki's Free [Automating GIS Processes Course](https://autogis-site.readthedocs.io/en/latest/){target="_blank"}
* Kaggle's Free [Python Courses](https://www.kaggle.com/learn){target="_blank"}, including [Geospatial Analysis](https://www.kaggle.com/learn/geospatial-analysis){target="_blank"}
* Allen Downey's Free [Elements of Data Science Course](https://allendowney.github.io/ElementsOfDataScience/){target="_blank"}
* [Doing Geospatial in Python](https://geopython.github.io/geopython-workshop/){target="_blank"} online workshop. Covers some more geospatial libraries, with a focus on OGC services and metadata.
* Andrew Ng's [ChatGPT Prompt Engineering for Developers](https://www.deeplearning.ai/short-courses/chatgpt-prompt-engineering-for-developers/){target="_blank"} that teaches how to use OpenAI API for building applications.


\newpage

# Data Credits
* [World Cities Database](https://simplemaps.com/data/world-cities){target="_blank"}: Basic database. Copyright SimpleMaps.com 2010-2020
* [Geonames](https://www.geonames.org){target="_blank"} places database for USA, Canada and Mexico
* Karnataka Admin Boundaries: Downloaded from [Datameet Spatial Data repository](https://github.com/datameet/maps/tree/master/States){target="_blank"}.
* OpenStreetMap (osm) data layers: Data/Maps Copyright 2019 Geofabrik GmbH and OpenStreetMap Contributors. [OSM India free extract](https://download.geofabrik.de/asia/india.html){target="_blank"} downloaded from Geofabrik.
* Bangalore Sentinel-2 Imagery. Downloaded from [Copernicus Open Access Hub](https://scihub.copernicus.eu/){target="_blank"}. Copyright European Space Agency - ESA.
* NASA Shuttle Radar Topography Mission (SRTM) Elevation Dataset. Downloaded from [30m SRTM Tile Downloader](https://dwtkns.com/srtm30m/]){target="_blank"}.


# License

This course material is licensed under a [Creative Commons Attribution 4.0 International (CC BY 4.0)](https://creativecommons.org/licenses/by/4.0/){target="_blank"}. You are free to re-use and adapt the material but are required to give appropriate credit to the original author as below:

*Python Foundation for Spatial Analysus Course* by Ujaval Gandhi [www.spatialthoughts.com](https://spatialthoughts.com){target="_blank"}

&copy; 2023 Ujaval Gandhi [www.spatialthoughts.com](https://spatialthoughts.com){target="_blank"}

***

**This course is offered as an instructor-led online class. Visit [Spatial Thoughts](https://spatialthoughts.com/events/){target="_blank"} to know details of upcoming sessions.**