Dear Data Science

This course is about Data Science and Data Humanism and a blending of the two. The reference authors are data scientist Hadley Wickham and information designer Giorgia Lupi. I will try to follow the following teaching principles:

core-periphery layout: give a minimal core of tools and a large periphery of applications (possibly expanding the toolset)
multiple views: approach the same task from different perspectives
non-linearity: do not proceed on a linear path, but open multiple branches and jump forward and backward
interactivity: involve directly the students with questions and exercises
multiple tasks: not only frontal teaching, but also watching, listening, reading, making.

Play

Teasers
- watch The joy of stats by Hans Rosling
- watch The whole game by Hadley Wickham
- watch Data Humanism by Giorgia Lupi
Data Science
- Trailer
- Relational databases
  - learn Entity-Relationship model
  - learn Relational model
  - learn Relational algebra
  - use Relax with RelaX
  - use yEd graph editor
  - make Dry run
- A hasty tour inside R
  - use R
  - use RStudio
  - watch Learn R with Mike Marin
  - learn Inside R [html / Rmd]
  - watch A focus on data frames [Part 1 / Part 2]
  - learn Data frames or tibbles?
  - learn R Markdown
    - learn R Markdown
    - learn R Markdown formats
    - glance Markdown cheatsheet
    - glance R Markdown Cheatsheet
  - glance Git and Github
    - glance Git and Github
    - dig RStudio, Git and GitHub
  - make Dry run
  - glance Cheatsheet. Base R
  - glance Cheatsheet. RStudio
- The pipeline of data analytics
  - learn Little bunny Foo Foo
  - Import
    - watch Data import in base R
    - learn Data import in base R. Read chapter 4 in R Cookbook
    - make Dry run
    - learn Data import with readr
  - Tidy
    - glance Tidyverse
    - learn Tidy data with tidyr
    - make Dry run
    - glance Cheatsheet. Data import: readr, tibble, tidyr
  - Transform
    - learn Data transformation in base R. Also read chapters from 5.18 to 5.31 in R Cookbook
    - learn Data transformation with dplyr
    - learn Joins with dplyr
    - learn From the shell [shell, adult.data]
    - make Dry run
    - glance Cheatsheet. Data transformation: dplyr
  - Visualize
    - read The Great Wave off Kanagawa
    - learn Data visualization in base R. Read chapter 10 in R Cookbook
    - watch Plotting with Base R. Part 1 / Part 2 / Part 3
    - learn Data visualization with ggplot
    - learn Exploratory Data Analysis with dplyr and ggplot
    - learn Perfection is in the details
    - learn Animated plots [Rmd / html]
    - learn Interactive plots
      - learn Shiny
        
        helloShiny
        
        helloWidgets
        
        helloReactivity
      - glance HTML widgets
        
        Showcase
      - glance Dashboards
        
        K-means
        
        Storyboard
    - make Dry run
    - glance Generative art in R
    - glance Cheatsheet. Data visualization: ggplot2
  - Model
    - watch Linear regression in base R
    - learn Model basics
    - learn Model building
    - learn Many models
    - glance The corplot package
    - make Dry run
  - Big data
    - watch Christopher Barnatt
    - watch Tim Smith
    - watch How to take a picture of a black hole | Katie Bouman
    - learn Big Data
    - learn Writing efficient R code [Rmd / html]
    - learn Introduction to Spark in R using sparklyr [Rmd / html]
    - learn BigQuery [Rmd / html]
    - make Dry run
Data humanism
- watch How we can find ourselves in data
- glance Giogia Lupi
- read Data humanism, the revolution will be visualized
- use Processing
- learn A hasty tour inside Processing
- learn Customized data visualization in Processing
  - watch Data vs information visualization
  - learn Visualize hot artists
  - learn Force-directed network visualization
  - learn Visualize prime numbers
- use Arduino
- watch Wired on Arduino
- learn From Arduino to Processing and back
- learn Visualizing real-time data
- make Great gaps in the world of art auctions
- make Your Lucky Numbers
- make Pick 3 visualizations from La Lettura and dig into them
- make Dry run

Task-tag legend

learn: I teach, you listen (and hopefully learn).
make: I give you an assignment, you make it during the class. We discuss the solutions during the next class.
use: you use a software: download, install and run it for the first time. I give you a brief practical introduction to it.
watch: We watch a video together. By and large, the video acts as a teaser, introducing the next topic in an informal and attractive way.
glance: You give a brief and fast look at something, generally an informative website. I steer you towards the most important sections.
read: You read a story, typically at home. We discuss it together during the following class.
dig: You read a deepening of the current topic, normally at home. We talk about it during one of the next classes.
listen: The class is given by an invited speaker, an expert in the field.

Tools

R and RStudio
Processing

Books

Datasets

Data stories

In a data story (or data challenge) you tell a story with data. Find a dataset, pose questions, and try to solve them using an analysis notebook in R. Follow your curiosity and be creative.

Visualize flowers with R challenge [Tasks: visualize]
Generative art with R challenge [Tasks: visualize]
Which is the best team ever in Italian soccer? challenge [Tasks: transform, visualize]
Which among goals for and goals against contribute more to points in Italian soccer? challenge [Tasks: transform, model]
What are the qualities of diamonds? challenge [Tasks: transform, visualize, model]
What affects the number of daily flights? challenge [Tasks: transform, visualize, model]
Are iris flower sepal and petal sizes correlated? challenge [Tasks: visualize, model]
How life expectancy change over time for each country? challenge [Tasks: transform, visualize, model]
Is child mortality decreasing over time? challenge [Tasks: transform, visualize, model]
In there a first-mover advantage in chess? challenge [Tasks: transform, visualize, program]
What are the busy days in the market of crypto art? challenge [Tasks: transform, visualize]

Mid-course assignment

The mid-course assignment covers the full pipeline of Data Science. You’re asked to investigate the Italian Soccer League.

Exam

The exam consists of a written exam. The written part consists of a list of questions, either open questions or exercises, over all the covered syllabus. During the written exam students are allowed to use only sheets covering the syntax of languages (such as cheatsheets). The outcome of the written part is a mark from 0 to 30.

The student can also make a project, which is optional and gives the student a bonus from 0 to 3 points (to sum to the mark of the written part). The project consists of one significant data challenge chosen by the student. It is done individually and must use methods, languages and software tools seen during the course. The student will discuss the project the day of the written exam, in a maximum time of 15 minutes, using a presentation on a personal laptop (bring adapters). The presentation must focus on the used dataset, the data questions, the performed analyzes and the results obtained. Both the project and the presentation skills will be evaluated. Each student can discuss the project only once. If the written part fails, the bonus of the project is still valid.