{ "cells": [ { "cell_type": "markdown", "id": "2d4b6ad3-ee6e-43ec-a0e6-544e92887e4e", "metadata": {}, "source": [ "# Exercises" ] }, { "cell_type": "markdown", "id": "59979bb7-98eb-423e-9893-6261dd9a43e1", "metadata": {}, "source": [ "## Exercise 3.1\n", "\n", "In this exercise your task is to open and explore a NOAA weather data file using Pandas. The data file name is 6153237444115dat.csv and it is located in the data folder (*add link*). An overview of the tasks in this exercise:\n", "\n", "- Import the Pandas module\n", "- Read the data using Pandas into a variable called data\n", "- Calculate a number of basic statistics from the data\n", "\n", "### Problem 1 - Read the file and clean it\n", "\n", "Import the pandas module and read the weather data into a variable called `data`. Print the first five rows of the data file.\n", "\n", "### Problem 2 - Basic characteristics of the data\n", "\n", "Based on the `data` DataFrame from Problem 1, answer to following questions:\n", "\n", "1. How many rows is there in the data?\n", "2. What are the column names?\n", "3. What are the datatypes of the columns?\n", "\n", "### Problem 3 - Descriptive statistics\n", "\n", "Based on the `data` DataFrame from Problem 1, answer to following questions:\n", "\n", "- What is the mean Fahrenheit temperature in the data (use the `TEMP` column)?\n", "- What is the standard deviation of the Maximum temperature (use the `MAX` column)?\n", "- How many unique stations exists in the data (use the `USAF` column)?" ] }, { "cell_type": "markdown", "id": "6cca621b-1faa-4e66-b649-e47f639c8866", "metadata": {}, "source": [ "## Exercise 3.2\n", "\n", "In this exercise, you will clean the data from our data file by removing no-data values, convert temperature values in Fahrenheit to Celsius, and split the data into separate datasets using the weather station identification code. We will start this problem by cleaning and converting our temperature data. An overview of the tasks in this exercise:\n", "\n", "- Create a new dataframe called `selected` that contains selected columns from the data file\n", "- Clean the new DataFrame by removing no-data values\n", "- Create a new column for temperatures converted from Fahrenheit to Celsius\n", "- Divide the data into separate DataFrames for the Helsinki Kumpula and Rovaniemi stations\n", "- Save the new DataFrames to CSV files\n", "\n", "### Problem 1 - Read the data and remove NaN values\n", "\n", "The first step for this problem is to read the data file 6153237444115dat.csv into a variable `data` using pandas and cleaning it a bit:\n", "\n", "- Select the columns `USAF, YR--MODAHRMN, TEMP, MAX, MIN` from the `data` DataFrame and assign them to a variable `selected`\n", "- Remove all rows from `selected` that have NoData in the column `TEMP` using the `dropna()` function\n", "\n", "### Problem 2 - Convert temperatures to Celsius\n", "\n", "Convert the temperature values from Fahrenheits to Celsius:\n", "\n", "- Create a new column to `selected` called `Celsius`.\n", "- Convert the Fahrenheit temperatures from `TEMP` using the conversion formula below and store the results in the new `Celsius` column:\n", " - TempCelsius = (TempFahrenheit - 32) / 1.8\n", "- Round the values in the `Celsius` column to have 0 decimals (do not create a new column, update the current one)\n", "- Convert the `Celsius` values into integers (do not create a new column, update the current one)\n", "\n", "### Problem 3 - Select data and save to disk\n", "\n", "Divide the data in `selected` into two separate DataFrames:\n", "\n", "- Select all rows from the selected DataFrame with the `USAF` code `29980` into a variable called `kumpula`.\n", "- Select all rows from the selected DataFrame with the `USAF` code `28450` into a variable called `rovaniemi`.\n", "- Save the `kumpula` DataFrame into a file `Kumpula_temps_May_Aug_2017.csv` in CSV format:\n", " - Separate the columns with commas (,)\n", " - Use only 2 decimals for the floating point numbers\n", "- Repeat the same procedures and save the `rovaniemi` DataFrame into a file `Rovaniemi_temps_May_Aug_2017.csv`." ] }, { "cell_type": "markdown", "id": "b7ff4901-2d10-4047-9dcb-ef9d781b7250", "metadata": {}, "source": [ "## Exercise 3.3\n", "\n", "In this Exercise, we will explore our temperature data by comparing spring temperatures between Kumpula and Rovaniemi. To do this we'll use some conditions to extract subsets of our data and then analyse these subsets using basic pandas functions. Notice that in this exercise, we will use data saved from the previous Exercise (2.2.6), hence you should finish that Exercise before this one. An overview of the tasks in this exercise:\n", "\n", "- Calculate the median temperatures for Kumpula and Rovaniemi for the summer of 2017\n", "- Select temperatures for May and June 2017 in separate DataFrames for each location\n", "- Calculate descriptive statistics for each month (May, June) and location (Kumpula, Rovaniemi)\n", "\n", "### Problem 1 - Read the data and calculate basic statistics\n", "\n", "Read in the CSV files generated in Exercise 2.2.6 to the variables `kumpula` and `rovaniemi` and answer to following questions:\n", "\n", "- What was the median Celsius temperature during the observed period in Helsinki Kumpula? Store the answer in a variable `kumpula_median`.\n", "- What was the median Celsius temperature during the observed period in Rovaniemi? Store the answer in a variable `rovaniemi_median`.\n", "\n", "### Problem 2 - Select data and compare temperatures between months\n", "\n", "The median temperatures above consider data from the entire summer (May-Aug), hence the differences might not be so clear. Let's now find out the mean temperatures from May and June 2017 in Kumpula and Rovaniemi:\n", "\n", "- From the `kumpula` and `rovaniemi` DataFrames, select the rows where values of the `YR--MODAHRMN` column are from May 2017. Assign these selected rows into the variables `kumpula_may` and `rovaniemi_may` \n", "- Repeat the procedure for the month of June and assign those values into variables to `kumpula_june` and `rovaniemi_june`\n", "- Calculate and print the mean, min and max Celsius temperatures for both places in May and June using the new subset dataframes (kumpula_may, rovaniemi_may, kumpula_june, and rovaniemi_june). Answer to following questions:\n", " - Does there seem to be a large difference in temperatures between the months?\n", " - Is Rovaniemi a much colder place than Kumpula?\n", "\n", "### Problem 3 - Parse daily temperatures by aggregating data \n", "\n", "In this problem, the aim is to aggregate the hourly temperature data for Kumpula and Rovaniemi weather stations to a daily level. Currently, there are at most three measurements per hour in the data, as you can see from the YR--MODAHRMN column:\n", "\n", "```\n", " USAF YR--MODAHRMN TEMP MAX MIN Celsius\n", "0 28450 201705010000 31.0 NaN NaN -1\n", "1 28450 201705010020 30.0 NaN NaN -1\n", "2 28450 201705010050 30.0 NaN NaN -1\n", "3 28450 201705010100 31.0 NaN NaN -1\n", "4 28450 201705010120 30.0 NaN NaN -1\n", "```\n", "\n", "In this problem you should:\n", "\n", "- Summarize the information for each day by aggregating (grouping) the DataFrame using the `groupby()` function.\n", "- The output should be a new DataFrame where you have calculated mean, max and min Celsius temperatures for each day separately based on hourly values.\n", "- Repeat the task for the two data sets you created in Problem 2 (May-August temperatures from Rovaniemi and Kumpula)." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.7" } }, "nbformat": 4, "nbformat_minor": 5 }