{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Introduction" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Google Trends gives us an estimate of search volume. Let's explore if search popularity relates to other kinds of data. Perhaps there are patterns in Google's search volume and the price of Bitcoin or a hot stock like Tesla. Perhaps search volume for the term \"Unemployment Benefits\" can tell us something about the actual unemployment rate? \n", "\n", "Data Sources:
\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Import Statements" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import pandas as pd\n", "import matplotlib.pyplot as plt" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Read the Data\n", "\n", "Download and add the .csv files to the same folder as your notebook." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "df_tesla = pd.read_csv('TESLA Search Trend vs Price.csv')\n", "\n", "df_btc_search = pd.read_csv('Bitcoin Search Trend.csv')\n", "df_btc_price = pd.read_csv('Daily Bitcoin Price.csv')\n", "\n", "df_unemployment = pd.read_csv('UE Benefits Search vs UE Rate 2004-19.csv')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Data Exploration" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Tesla" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Challenge**:
\n", "" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Largest value for Tesla in Web Search: \n", "Smallest value for Tesla in Web Search: \n" ] } ], "source": [ "print(f'Largest value for Tesla in Web Search: ')\n", "print(f'Smallest value for Tesla in Web Search: ')" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Unemployment Data" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Largest value for \"Unemployemnt Benefits\" in Web Search: \n" ] } ], "source": [ "print('Largest value for \"Unemployemnt Benefits\" '\n", " f'in Web Search: ')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Bitcoin" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "largest BTC News Search: \n" ] } ], "source": [ "print(f'largest BTC News Search: ')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Data Cleaning" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Check for Missing Values" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Challenge**: Are there any missing values in any of the dataframes? If so, which row/rows have missing values? How many missing values are there?" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Missing values for Tesla?: \n", "Missing values for U/E?: \n", "Missing values for BTC Search?: \n" ] } ], "source": [ "print(f'Missing values for Tesla?: ')\n", "print(f'Missing values for U/E?: ')\n", "print(f'Missing values for BTC Search?: ')" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Missing values for BTC price?: \n" ] } ], "source": [ "print(f'Missing values for BTC price?: ')" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Number of missing values: \n" ] } ], "source": [ "print(f'Number of missing values: ')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Challenge**: Remove any missing values that you found. " ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Convert Strings to DateTime Objects" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Challenge**: Check the data type of the entries in the DataFrame MONTH or DATE columns. Convert any strings in to Datetime objects. Do this for all 4 DataFrames. Double check if your type conversion was successful." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Converting from Daily to Monthly Data\n", "\n", "[Pandas .resample() documentation](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.resample.html)
" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Data Visualisation" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Notebook Formatting & Style Helpers" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [], "source": [ "# Create locators for ticks on the time axis" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [], "source": [ "# Register date converters to avoid warning messages" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Tesla Stock Price v.s. Search Volume" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Challenge:** Plot the Tesla stock price against the Tesla search volume using a line chart and two different axes. Label one axis 'TSLA Stock Price' and the other 'Search Trend'. " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Challenge**: Add colours to style the chart. This will help differentiate the two lines and the axis labels. Try using one of the blue [colour names](https://matplotlib.org/3.1.1/gallery/color/named_colors.html) for the search volume and a HEX code for a red colour for the stock price. \n", "
\n", "
\n", "Hint: you can colour both the [axis labels](https://matplotlib.org/3.3.2/api/text_api.html#matplotlib.text.Text) and the [lines](https://matplotlib.org/3.2.1/api/_as_gen/matplotlib.lines.Line2D.html#matplotlib.lines.Line2D) on the chart using keyword arguments (kwargs). " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Challenge**: Make the chart larger and easier to read. \n", "1. Increase the figure size (e.g., to 14 by 8). \n", "2. Increase the font sizes for the labels and the ticks on the x-axis to 14. \n", "3. Rotate the text on the x-axis by 45 degrees. \n", "4. Make the lines on the chart thicker. \n", "5. Add a title that reads 'Tesla Web Search vs Price'\n", "6. Keep the chart looking sharp by changing the dots-per-inch or [DPI value](https://matplotlib.org/3.1.1/api/_as_gen/matplotlib.pyplot.figure.html). \n", "7. Set minimum and maximum values for the y and x axis. Hint: check out methods like [set_xlim()](https://matplotlib.org/3.1.1/api/_as_gen/matplotlib.axes.Axes.set_xlim.html). \n", "8. Finally use [plt.show()](https://matplotlib.org/3.2.1/api/_as_gen/matplotlib.pyplot.show.html) to display the chart below the cell instead of relying on the automatic notebook output." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "How to add tick formatting for dates on the x-axis. " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Bitcoin (BTC) Price v.s. Search Volume" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Challenge**: Create the same chart for the Bitcoin Prices vs. Search volumes.
\n", "1. Modify the chart title to read 'Bitcoin News Search vs Resampled Price'
\n", "2. Change the y-axis label to 'BTC Price'
\n", "3. Change the y- and x-axis limits to improve the appearance
\n", "4. Investigate the [linestyles](https://matplotlib.org/3.2.1/api/_as_gen/matplotlib.pyplot.plot.html ) to make the BTC price a dashed line
\n", "5. Investigate the [marker types](https://matplotlib.org/3.2.1/api/markers_api.html) to make the search datapoints little circles
\n", "6. Were big increases in searches for Bitcoin accompanied by big increases in the price?" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Unemployement Benefits Search vs. Actual Unemployment in the U.S." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Challenge** Plot the search for \"unemployment benefits\" against the unemployment rate. \n", "1. Change the title to: Monthly Search of \"Unemployment Benefits\" in the U.S. vs the U/E Rate
\n", "2. Change the y-axis label to: FRED U/E Rate
\n", "3. Change the axis limits
\n", "4. Add a grey [grid](https://matplotlib.org/3.2.1/api/_as_gen/matplotlib.pyplot.grid.html) to the chart to better see the years and the U/E rate values. Use dashes for the line style
\n", "5. Can you discern any seasonality in the searches? Is there a pattern? " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Challenge**: Calculate the 3-month or 6-month rolling average for the web searches. Plot the 6-month rolling average search data against the actual unemployment. What do you see in the chart? Which line moves first?\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Including 2020 in Unemployment Charts" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Challenge**: Read the data in the 'UE Benefits Search vs UE Rate 2004-20.csv' into a DataFrame. Convert the MONTH column to Pandas Datetime objects and then plot the chart. What do you see?" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.3" } }, "nbformat": 4, "nbformat_minor": 2 }