All work
ShippedCourse

RL for Skin Cancer Decision Support

Replicated and extended a Nature Medicine RL model for skin-lesion triage, where a missed melanoma costs far more than over-treating.

Role
Replication, environment simulation, baselines, reward-matrix experiments, visualizations and presentation
Team
3 people
When
Jan 2026 – May 2026
Context
NJIT Reinforcement Learning course
Result
~0.96 melanoma recall with the team's Dueling DQN (best epoch)
Stack
  • Python
  • TensorFlow
  • PyTorch
  • Gym
  • scikit-learn
ASYMMETRIC COST · ILLUSTRATIVEPREDICTEDACTUALTPFNFPTNCOSTLIESTEPISODESREWARD

Highlights

  1. 01Reproduced the paper's diagnosis results on HAM10000: plain DQN matched its melanoma recall (0.80 vs 0.795)
  2. 02Replicated lesion-level management and wrote the naive and threshold baselines, enabling a three-way comparison
  3. 03Simulated the decision environment and tweaked the expert reward matrix to compare how reward design changes the agent's behavior
  4. 04RL matched the hand-tuned threshold rule on melanoma safety with about 2.5x fewer unnecessary excisions
  5. 05Built the evaluation visualizations (confusion heatmaps, reward curves, paper-vs-ours overlays) and shaped the presentation storyline
  6. 06Team extension with a Dueling DQN and reward-matrix tuning reached about 0.96 melanoma recall in its best epoch
  7. 07Ported old TensorFlow 1 and Gym research code to current Colab and Kaggle environments

Write-up

Problem

Classifiers optimized for accuracy treat every error the same, but in skin-cancer triage a missed melanoma costs far more than an unnecessary excision. The paper turns this into a decision problem and uses an expert-designed reward matrix to teach an RL agent the asymmetric cost.

What I built

With a team of three, I reproduced the paper's diagnosis results on HAM10000 (~10,000 images, 7 classes), where our plain DQN matched the paper's melanoma recall (0.80 vs 0.795). I replicated the lesion-level management pipeline (dismiss, cryotherapy, excise) and wrote the naive and threshold-based baselines myself, so we could compare all three approaches. I also simulated the environment, tweaked the reward matrix for comparison runs, built the evaluation visualizations and shaped the presentation.

What I learned

Reward design shapes what the agent learns. In our runs, RL matched a hand-tuned threshold rule on melanoma safety with about 2.5x fewer unnecessary excisions. Our numbers followed the paper's trends but ran 15 to 30 points lower, mostly because we used an internal split instead of the paper's external ISIC 2018 test set.

Key decisions

Why a Dueling DQN

For the team's extension we used a Dueling DQN, which splits the network into two parts: one estimates how good a case is overall, the other how much better each action is than the alternatives. Separating the two tends to make learning steadier when several actions score similarly, which, together with reward-matrix tuning, likely helped it reach about 0.96 melanoma recall.

How a Dueling DQN worksA lesion image goes through shared layers. The network then splits into a value stream, which scores how good the case is overall, and an advantage stream, which scores how much better each action is than the others. The two are combined into one Q-value per action, and the agent picks the highest.Lesion imagethe state sShared layerslearn image featuresValue stream · V(s)how good is this case overall?one numberAdvantage stream · A(s, a)how much better is each action?one per actionCombineQ = V + A− mean(A)Q-valuespick the highest
Dueling DQN: the network learns "how good is this case" and "how much better is each action" separately, then adds them back into one score per action. Bars are illustrative.

More top projects