{ "nbformat": 4, "nbformat_minor": 0, "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.9" }, "colab": { "name": "Fatal Force (start).ipynb", "provenance": [] } }, "cells": [ { "cell_type": "markdown", "metadata": { "id": "0KrR-Qd_wXf-" }, "source": [ "# Introduction" ] }, { "cell_type": "markdown", "metadata": { "id": "I6e2m4XsuRoL" }, "source": [ "Since Jan. 1, 2015, [The Washington Post](https://www.washingtonpost.com/) has been compiling a database of every fatal shooting in the US by a police officer in the line of duty. \n", "\n", "
\n", "\n", "While there are many challenges regarding data collection and reporting, The Washington Post has been tracking more than a dozen details about each killing. This includes the race, age and gender of the deceased, whether the person was armed, and whether the victim was experiencing a mental-health crisis. The Washington Post has gathered this supplemental information from law enforcement websites, local new reports, social media, and by monitoring independent databases such as \"Killed by police\" and \"Fatal Encounters\". The Post has also conducted additional reporting in many cases.\n", "\n", "There are 4 additional datasets: US census data on poverty rate, high school graduation rate, median household income, and racial demographics. [Source of census data](https://factfinder.census.gov/faces/nav/jsf/pages/community_facts.xhtml)." ] }, { "cell_type": "markdown", "metadata": { "id": "JwGJl3mnw2Af" }, "source": [ "### Upgrade Plotly\n", "\n", "Run the cell below if you are working with Google Colab" ] }, { "cell_type": "code", "metadata": { "id": "KvA9PBoRwvQG", "outputId": "ca6ee2a1-8da5-4f2d-f2b2-ea019cd92cae", "colab": { "base_uri": "https://localhost:8080/" } }, "source": [ "%pip install --upgrade plotly" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "Collecting plotly\n", "\u001b[?25l Downloading https://files.pythonhosted.org/packages/a6/66/af86e9d9bf1a3e4f2dabebeabd02a32e8ddf671a5d072b3af2b011efea99/plotly-4.12.0-py2.py3-none-any.whl (13.1MB)\n", "\u001b[K |████████████████████████████████| 13.1MB 310kB/s \n", "\u001b[?25hRequirement already satisfied, skipping upgrade: six in /usr/local/lib/python3.6/dist-packages (from plotly) (1.15.0)\n", "Requirement already satisfied, skipping upgrade: retrying>=1.3.3 in /usr/local/lib/python3.6/dist-packages (from plotly) (1.3.3)\n", "Installing collected packages: plotly\n", " Found existing installation: plotly 4.4.1\n", " Uninstalling plotly-4.4.1:\n", " Successfully uninstalled plotly-4.4.1\n", "Successfully installed plotly-4.12.0\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "jqXsMLNJxELp" }, "source": [ "## Import Statements" ] }, { "cell_type": "code", "metadata": { "id": "TPhEXC2FxGSr" }, "source": [ "import numpy as np\n", "import pandas as pd\n", "import plotly.express as px\n", "import matplotlib.pyplot as plt\n", "import seaborn as sns\n", "\n", "# This might be helpful:\n", "from collections import Counter" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "SwKGmmaLxS2s" }, "source": [ "## Notebook Presentation" ] }, { "cell_type": "code", "metadata": { "id": "ayQQqhKtxBxj" }, "source": [ "pd.options.display.float_format = '{:,.2f}'.format" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "nKxJMOiMyAUF" }, "source": [ "## Load the Data" ] }, { "cell_type": "code", "metadata": { "_cell_guid": "abb0685f-f844-43f8-84c0-4cd54361c7bf", "_uuid": "9c06ed7c41db79ae9d940f2d52aeff23621b3037", "id": "8KmGeg6ouRoP" }, "source": [ "df_hh_income = pd.read_csv('Median_Household_Income_2015.csv', encoding=\"windows-1252\")\n", "df_pct_poverty = pd.read_csv('Pct_People_Below_Poverty_Level.csv', encoding=\"windows-1252\")\n", "df_pct_completed_hs = pd.read_csv('Pct_Over_25_Completed_High_School.csv', encoding=\"windows-1252\")\n", "df_share_race_city = pd.read_csv('Share_of_Race_By_City.csv', encoding=\"windows-1252\")\n", "df_fatalities = pd.read_csv('Deaths_by_Police_US.csv', encoding=\"windows-1252\")" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "6RuA5SvUyEaB" }, "source": [ "# Preliminary Data Exploration\n", "\n", "* What is the shape of the DataFrames? \n", "* How many rows and columns do they have?\n", "* What are the column names?\n", "* Are there any NaN values or duplicates?" ] }, { "cell_type": "code", "metadata": { "id": "3U9RcS1WyrRJ" }, "source": [ "" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "DmIARadGyrgs" }, "source": [ "" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "OSgTJoX7yrcS" }, "source": [ "" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "p72TfMvayr2C" }, "source": [ "## Data Cleaning - Check for Missing Values and Duplicates\n", "\n", "Consider how to deal with the NaN values. Perhaps substituting 0 is appropriate. " ] }, { "cell_type": "code", "metadata": { "id": "WEZuPaXvyxxp" }, "source": [ "" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "550aSczryx01" }, "source": [ "" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": { "_uuid": "52b308c936f5835c5cd186497fe3f96e96372483", "id": "n3DrcXVQuRoZ" }, "source": [ "# Chart the Poverty Rate in each US State\n", "\n", "Create a bar chart that ranks the poverty rate from highest to lowest by US state. Which state has the highest poverty rate? Which state has the lowest poverty rate? Bar Plot" ] }, { "cell_type": "code", "metadata": { "id": "mGxEzMDqzqmn" }, "source": [ "" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "B3AM6oJQ1D_P" }, "source": [ "" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "GhUibVz5z4sn" }, "source": [ "# Chart the High School Graduation Rate by US State\n", "\n", "Show the High School Graduation Rate in ascending order of US States. Which state has the lowest high school graduation rate? Which state has the highest?" ] }, { "cell_type": "code", "metadata": { "id": "laD6W9eG0JFR" }, "source": [ "" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "ENGTEmPQ0Rvt" }, "source": [ "# Visualise the Relationship between Poverty Rates and High School Graduation Rates\n", "\n", "#### Create a line chart with two y-axes to show if the rations of poverty and high school graduation move together. " ] }, { "cell_type": "code", "metadata": { "id": "MkNk5U8v0JPB" }, "source": [ "" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "urRMqQG_0Nmo" }, "source": [ "" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "Zavx_8PL1Jy9" }, "source": [ "#### Now use a Seaborn .jointplot() with a Kernel Density Estimate (KDE) and/or scatter plot to visualise the same relationship" ] }, { "cell_type": "code", "metadata": { "id": "u77CHfdX0pjW" }, "source": [ "" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "CYy89o-u1zu0" }, "source": [ "" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "PXNFQmHl10Fx" }, "source": [ "#### Seaborn's `.lmplot()` or `.regplot()` to show a linear regression between the poverty ratio and the high school graduation ratio. " ] }, { "cell_type": "code", "metadata": { "id": "F1xrjFgT1znI" }, "source": [ "" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "QJTbZZKg0p6i" }, "source": [ "# Create a Bar Chart with Subsections Showing the Racial Makeup of Each US State\n", "\n", "Visualise the share of the white, black, hispanic, asian and native american population in each US State using a bar chart with sub sections. " ] }, { "cell_type": "code", "metadata": { "id": "YQbXb6Ex09D9" }, "source": [ "" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "LqrimlLo09Na" }, "source": [ "" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": { "_uuid": "77493cfd86a031612be8fb8d60f27713e74e6d80", "id": "K7xvQXkjuRo7" }, "source": [ "# Create Donut Chart by of People Killed by Race\n", "\n", "Hint: Use `.value_counts()`" ] }, { "cell_type": "code", "metadata": { "id": "1nOafPdy1u9Z" }, "source": [ "" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "YN3hzJTe1vHJ" }, "source": [ "" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "kr-8Rezn3Bep" }, "source": [ "# Create a Chart Comparing the Total Number of Deaths of Men and Women\n", "\n", "Use `df_fatalities` to illustrate how many more men are killed compared to women. " ] }, { "cell_type": "code", "metadata": { "id": "joH1L4303BsJ" }, "source": [ "" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "X4MrMkVa3MTk" }, "source": [ "" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": { "_uuid": "cd1531c8421bfb6967cebf7039056779f9b960a6", "id": "ZxbBdvM0uRpQ" }, "source": [ "# Create a Box Plot Showing the Age and Manner of Death\n", "\n", "Break out the data by gender using `df_fatalities`. Is there a difference between men and women in the manner of death? " ] }, { "cell_type": "code", "metadata": { "id": "WdPPZLJY2shA" }, "source": [ "" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "tpPq4uFc2sql" }, "source": [ "" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "EnFbXlRm3XK6" }, "source": [ "" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "h1pAgkyb3Xb0" }, "source": [ "# Were People Armed? \n", "\n", "In what percentage of police killings were people armed? Create chart that show what kind of weapon (if any) the deceased was carrying. How many of the people killed by police were armed with guns versus unarmed? " ] }, { "cell_type": "code", "metadata": { "id": "0d9B3Pec4TdM" }, "source": [ "" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "vdGTO3pW4Tp5" }, "source": [ "" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "qnO_rckl4WuJ" }, "source": [ "" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "IgD2a5-L4W_I" }, "source": [ "# How Old Were the People Killed?" ] }, { "cell_type": "markdown", "metadata": { "id": "BNPrdT-W9lf3" }, "source": [ "Work out what percentage of people killed were under 25 years old. " ] }, { "cell_type": "code", "metadata": { "id": "zSyDgzb542O9" }, "source": [ "" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "5XrAD_ey9qBB" }, "source": [ "Create a histogram and KDE plot that shows the distribution of ages of the people killed by police. " ] }, { "cell_type": "code", "metadata": { "id": "U6B1Qp7l42H8" }, "source": [ "" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "zcqqPDo79rTx" }, "source": [ "Create a seperate KDE plot for each race. Is there a difference between the distributions? " ] }, { "cell_type": "code", "metadata": { "id": "ubUnMjok43Zo" }, "source": [ "" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "JkcHvMUt43uE" }, "source": [ "# Race of People Killed\n", "\n", "Create a chart that shows the total number of people killed by race. " ] }, { "cell_type": "code", "metadata": { "id": "e8wZvBrt5BEb" }, "source": [ "" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "_aqN0wC45BOl" }, "source": [ "" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "SJaNm4rC5nWm" }, "source": [ "# Mental Illness and Police Killings\n", "\n", "What percentage of people killed by police have been diagnosed with a mental illness?" ] }, { "cell_type": "code", "metadata": { "id": "dbuGvCnN5ytC" }, "source": [ "" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "BsKDVLiY5CP_" }, "source": [ "" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "R0WSPeKD5Cir" }, "source": [ "# In Which Cities Do the Most Police Killings Take Place?\n", "\n", "Create a chart ranking the top 10 cities with the most police killings. Which cities are the most dangerous? " ] }, { "cell_type": "code", "metadata": { "id": "NYVwo3qy5UV3" }, "source": [ "" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "T_FU4GFk5Upa" }, "source": [ "" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "AqcOW6ca8JMD" }, "source": [ "# Rate of Death by Race\n", "\n", "Find the share of each race in the top 10 cities. Contrast this with the top 10 cities of police killings to work out the rate at which people are killed by race for each city. " ] }, { "cell_type": "code", "metadata": { "id": "FkjAUlKy8I2g" }, "source": [ "" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "dA3Hs9b55V59" }, "source": [ "" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "1HUJksrI5WFb" }, "source": [ "# Create a Choropleth Map of Police Killings by US State\n", "\n", "Which states are the most dangerous? Compare your map with your previous chart. Are these the same states with high degrees of poverty? " ] }, { "cell_type": "code", "metadata": { "id": "u3UuX4Yo5dBA" }, "source": [ "" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "Y6VLPDeN5dKO" }, "source": [ "" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "37UsmVYF6hA2" }, "source": [ "# Number of Police Killings Over Time\n", "\n", "Analyse the Number of Police Killings over Time. Is there a trend in the data? " ] }, { "cell_type": "code", "metadata": { "id": "nITf1IhP6gpI" }, "source": [ "" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "Q_vxjD_A7JRh" }, "source": [ "" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "mOqtie707JU8" }, "source": [ "" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "5fjhL8Lk7JKa" }, "source": [ "" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "9o6uLFpS61nw" }, "source": [ "# Epilogue\n", "\n", "Now that you have analysed the data yourself, read [The Washington Post's analysis here](https://www.washingtonpost.com/graphics/investigations/police-shootings-database/)." ] }, { "cell_type": "code", "metadata": { "id": "8PjgzuNa61I1" }, "source": [ "" ], "execution_count": null, "outputs": [] } ] }