Case Study

From Notebook Output to an Experience People Can Use

The original implementation worked, but most of its value was hidden inside notebook cells, console output, and score logs. My goal for this version was not to disguise that work as a new product. I wanted to give it a professional front end that makes the behavior easier to explore while keeping the original implementation available.

The problem I wanted to solve

A completed machine-learning notebook can be difficult to evaluate quickly. A visitor has to understand the environment, locate the important settings, and interpret pages of code and output before seeing what the work accomplishes. That is too much friction for a portfolio piece.

I needed a browser-based version that could communicate the project on its own: what the agent sees, what it can do, how training changes over time, and how the displayed behavior relates to the source work.

01class DQNSolver:
02    def __init__(self, observation_space, action_space):
03        self.exploration_rate = EXPLORATION_MAX
04        self.memory = deque(maxlen=MEMORY_SIZE)
05
06        self.model = Sequential()
07        self.model.add(Dense(24, input_shape=(observation_space,), activation="relu"))
08        self.model.add(Dense(24, activation="relu"))
09        self.model.add(Dense(action_space, activation="linear"))
10        self.model.compile(loss="mse", optimizer=Adam(lr=LEARNING_RATE))

Separating the source from the showcase

I treated the Python notebook as the source implementation and the website as its presentation layer. The original notebook, scores directory, and score_logger.py helper remain available rather than being replaced by the JavaScript version. That separation lets visitors inspect the actual work while still having an immediate way to interact with its main ideas.

import random
import gym
import numpy as np
from collections import deque
from keras.models import Sequential
from keras.layers import Dense
from keras.optimizers import Adam

from scores.score_logger import ScoreLogger

Deciding what the interface needed to show

I centered the interface on the information that explains the agent’s behavior without forcing someone to read the code first. The four state values sit directly below the simulation. Training metrics show the episode count, current score, rolling average, best score, exploration rate, and replay-memory size. A score chart and event log provide the longer view.

I also exposed the original model configuration near the top of the page so the technical context is visible without interrupting the simulation.

Translating the behavior for the browser

GitHub Pages cannot execute the Python, Gym, and Keras workflow directly, so I recreated the CartPole physics and dense-layer behavior in vanilla JavaScript. I kept the same basic state-action-reward loop, experience replay, exploration decay, and score tracking, then connected those values to the interface in real time.

The manual-control option was a deliberate addition. Letting someone try to balance the pole with the arrow keys gives the agent’s task immediate context. The visitor can feel how quickly one correction creates the need for another, then switch back to the agent and watch the same problem through the training data.

Using the recorded results responsibly

The website does not present its JavaScript recreation as if it were running the original Keras model. The completed training results belong to the preserved notebook and are shown as supporting evidence. One gamma test reached a 195.38 average, and the recorded completed runs reached 197.25 and 199.43.

# Test stronger weighting of future rewards
ENV_NAME = "CartPole-v1"
GAMMA = 0.995  # changed from 0.95
LEARNING_RATE = 0.001
BATCH_SIZE = 20
EXPLORATION_DECAY = 0.995

In [2]: cartpole()
Run: 189, exploration: 0.01
Scores: min 16, avg 195.38

SOLVED in 89 runs, 189 total runs.

I kept the separate learning-rate test in the development record as well. It documents part of the testing process without implying that the larger 0.1 value became the final browser configuration.

A third experiment increased the minimum exploration rate from 0.01 to 0.02 while leaving the other baseline settings unchanged. The agent solved the environment at run 340, completed 440 episodes, reached a maximum score of 500, and maintained late-stage averages over 190. I included that result because it shows how continued exploration affected consistency and recovery during later training.

# Baseline used by the browser profile
LEARNING_RATE = 0.001

# Separate parameter test preserved in the development record
LEARNING_RATE = 0.1

Scores: avg 197.25, max 500
SOLVED in 99 runs, 199 total runs.

Scores: avg 199.43, max 500
SOLVED in 39 runs, 139 total runs.

The finished result

The final site works as more than a wrapper around old files. It gives the project a clear entry point, provides a live demonstration, explains the technical concepts separately, and keeps the original evidence accessible for anyone who wants to look deeper.

Skills represented

• Reinforcement-learning implementation and parameter testing.
• JavaScript translation of physics and neural-network behavior.
• Interactive data visualization and real-time interface updates.
• Responsive front-end design for a technical portfolio audience.
• Clear separation between source evidence, technical explanation, and public-facing demonstration.