Cancer Diagnosis

Cartoon of a penguin in a lab coat using a magnifying glass to compare a healthy cell, marked with a check, to a spiky cancerous cell, marked with an X.

The Data

In this lab, you will train and evaluate a classification algorithm to determine whether or not a fine needle aspiration biopsy is cancerous (malignant) or non-cancerous (benign). The data for this lab are a lightly processed version of the data found at the UC Irvine Machine Learning Repository. The cells data frame is available under the files tab in bCourses!

Questions

Question 1

What is the unit of observation in the data frame?

Question 2

We will be fitting models to output a diagnosis (“M” or “B”, for “malignant” and “benign”, respectively). Will the model estimate the probability of being benign or malignant by default? Hint: remember the concept of the reference level!

Question 3

part a

Write ggplot2 code to visualize the distributions of radius_mean for malignant vs. benign biopsies.

part b

Describe the relationship between the two distributions in at least one sentence.

Question 4

part a

Write ggplot2 code to visualize the association between radius_mean and area_mean.

part b

Modify your code to color the points on your visualization by diagnosis.

part c

Describe how the malignant and benign biopsies differ as seen on the visualization.

Question 5

Split the data set into a roughly 80-20 train-test set split (80 percent of the data to the training set, 20 percent of the data to the testing set).

Question 6

Using the training data, fit a simple logistic regression model that predicts the diagnosis using the mean of the texture index.

Question 7

Consider a new biopsy with a mean texture of 15.

part a

What does your model assign as the probability that the cancer is malignant?

part b

Using a probability threshold of 0.5, what outcome would your model predict for this biopsy (benign or malignant)?

Question 8

part a

Calculate and report two misclassification rates for your simple model: first on the training data and then on the test data.

part b

Calculate and report Precision and Recall for your simple model: first on the training data and then on the test data.

part c

Which framework for evaluation is preferable in this case in your opinion? MCR or Precision+Recall?

Question 9

part a

What can you change about your prediction rule to lower the number of false negatives?

part b

Make the change you identified in the previous question and calculate the new confusion matrix.

part c

Calculate the test misclassification rate, Precision and Recall, using your new classification rule. How did these metrics move?

Question 10

Now take your model and make it overfit.

part a

Add a polynomial term of degree \(d\) in the mean texture index variable and keep calculating training and test evaluation metrics for increasing values of \(d\) until the model overfits, i.e. until the training performance keeps improving but the test performance starts getting significantly worse.

part b

What value of \(d\) starts overfitting according to your calculation? Report the training/test metrics for a few values od \(d\), a bit before before, just at, and a bit after the point it starts to overfit.

Question 11

In many realms of medicine, classification algorithms can be more accurate than the most well-trained medical doctors. What is gained and what is lost by shifting to algorithmic diagnoses (when whatever diagnosis the model outputs is treated as official)? Answer in at least three sentences.

Last Question

Will you ensure that your submission to Gradescope…

  1. is a pdf generated from a qmd file
  2. your code is fully visible (not running off the page)
  3. follows formatting rules (here)
  4. and assigns each of the questions to all pages that show your work for that question?

(This one is easy! Just answer “yes” or “no”)