Homework #3: Generative Adversarial Networks and Variational Autoencoders
EE 641: Fall 2026
ImportantAssignment Details
Assigned: 16 September
Due: Tuesday, 29 September at 23:59
Submission: Gradescope via GitHub repository
WarningRequirements
- PyTorch >= 2.0 must be installed
- Allowed libraries: PyTorch, NumPy, Pillow (PIL), matplotlib, librosa (for audio only), and Python standard library
- No other external libraries permitted (including no torchvision.models, pre-trained models, or GAN-specific libraries)
Overview
In this assignment you will implement and analyze two fundamental generative models: Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs). The first problem explores GAN training dynamics and mode collapse through font generation. The second problem implements a hierarchical VAE for drum pattern generation with style control.
Getting Started
Download the starter code: hw3-starter.zip
Extract and generate the datasets:
unzip hw3-starter.zip
cd hw3-starter
python setup_data.py --seed 641This creates a data/ directory with: - fonts/: Synthetic font dataset (28×28 grayscale letter images) - drums/: Drum pattern dataset (16×9 binary matrices)
Use seed 641 to ensure consistent results across submissions.
Problem 1: Font Generation GAN - Understanding Mode Collapse
Build a GAN that generates letter images and observe mode collapse firsthand. You will implement diagnostic tools and test different stabilization techniques.
Part A: Dataset and Data Loading
You will work with a font dataset containing grayscale images of letters A-Z in 10 different fonts.
The dataset is provided in the following structure: - Images: 28×28 grayscale, normalized to [0, 1] - Classes: 26 letters × 10 fonts = 260 unique letter-font combinations - Training: 200 samples per letter (mixed fonts) - Validation: 60 samples per letter
The starter code provides dataset.py with the data loader:
"""
Dataset loader for font generation task.
"""
import torch
from torch.utils.data import Dataset
from PIL import Image
import os
import json
import numpy as np
class FontDataset(Dataset):
def __init__(self, data_dir, split='train'):
"""
Initialize the font dataset.
Args:
data_dir: Path to font dataset directory
split: 'train' or 'val'
"""
self.data_dir = data_dir
self.split = split
# TODO: Load metadata from fonts_metadata.json
# Expected structure:
# {
# "train": [{"path": "A/font1_A.png", "letter": "A", "font": 1}, ...],
# "val": [...]
# }
metadata_path = os.path.join(data_dir, 'fonts_metadata.json')
with open(metadata_path, 'r') as f:
metadata = json.load(f)
self.samples = metadata[split]
self.letter_to_id = {chr(65+i): i for i in range(26)} # A=0, B=1, ..., Z=25
def __len__(self):
return len(self.samples)
def __getitem__(self, idx):
"""
Return a sample from the dataset.
Returns:
image: Tensor of shape [1, 28, 28]
letter_id: Integer 0-25 representing A-Z
"""
sample = self.samples[idx]
# TODO: Load and process image
# 1. Load image from sample['path']
# 2. Convert to grayscale if needed
# 3. Resize to 28x28 if needed
# 4. Normalize to [0, 1]
# 5. Convert to tensor
image_path = os.path.join(self.data_dir, sample['path'])
image = Image.open(image_path).convert('L') # Ensure grayscale
image = image.resize((28, 28), Image.LANCZOS)
# Convert to numpy and normalize
image_np = np.array(image).astype(np.float32) / 255.0
# Convert to tensor and add channel dimension
image_tensor = torch.from_numpy(image_np).unsqueeze(0)
# Get letter ID
letter_id = self.letter_to_id[sample['letter']]
return image_tensor, letter_idPart B: GAN Architecture
Implement models.py with Generator and Discriminator networks.
The starter code provides architecture skeletons that work well for 28×28 grayscale images:
"""
GAN models for font generation.
"""
import torch
import torch.nn as nn
class Generator(nn.Module):
def __init__(self, z_dim=100, conditional=False, num_classes=26):
"""
Generator network that produces 28×28 letter images.
Args:
z_dim: Dimension of latent vector z
conditional: If True, condition on letter class
num_classes: Number of letter classes (26)
"""
super().__init__()
self.z_dim = z_dim
self.conditional = conditional
# Calculate input dimension
input_dim = z_dim + (num_classes if conditional else 0)
# Architecture proven to work well for this task:
# Project and reshape: z → 7×7×128
self.project = nn.Sequential(
nn.Linear(input_dim, 128 * 7 * 7),
nn.BatchNorm1d(128 * 7 * 7),
nn.ReLU(True)
)
# Upsample: 7×7×128 → 14×14×64 → 28×28×1
self.main = nn.Sequential(
# TODO: Implement upsampling layers
# Use ConvTranspose2d with appropriate padding/stride
# Include BatchNorm2d and ReLU (except final layer)
# Final layer should use Tanh activation
)
def forward(self, z, class_label=None):
"""
Generate images from latent code.
Args:
z: Latent vectors [batch_size, z_dim]
class_label: One-hot encoded class labels [batch_size, num_classes]
Returns:
Generated images [batch_size, 1, 28, 28] in range [-1, 1]
"""
# TODO: Implement forward pass
# If conditional, concatenate z and class_label
# Project to spatial dimensions
# Apply upsampling network
pass
class Discriminator(nn.Module):
def __init__(self, conditional=False, num_classes=26):
"""
Discriminator network that classifies 28×28 images as real/fake.
"""
super().__init__()
self.conditional = conditional
# Proven architecture for 28×28 images:
self.features = nn.Sequential(
# TODO: Implement convolutional layers
# 28×28×1 → 14×14×64 → 7×7×128 → 3×3×256
# Use Conv2d with appropriate stride
# LeakyReLU(0.2) and Dropout2d(0.25)
)
# Calculate feature dimension after convolutions
feature_dim = 256 * 3 * 3 # Adjust based on your architecture
self.classifier = nn.Sequential(
nn.Linear(feature_dim + (num_classes if conditional else 0), 1),
nn.Sigmoid()
)
def forward(self, img, class_label=None):
"""
Classify images as real (1) or fake (0).
Returns:
Probability of being real [batch_size, 1]
"""
# TODO: Extract features, flatten, concatenate class if conditional
passPart C: Training Dynamics and Mode Collapse
Implement the training loop in training_dynamics.py to observe mode collapse:
"""
GAN training implementation with mode collapse analysis.
"""
import torch
import torch.nn as nn
import torch.optim as optim
import numpy as np
from collections import defaultdict
def train_gan(generator, discriminator, data_loader, num_epochs=100, device='cuda'):
"""
Standard GAN training implementation.
Uses vanilla GAN objective which typically exhibits mode collapse.
Args:
generator: Generator network
discriminator: Discriminator network
data_loader: DataLoader for training data
num_epochs: Number of training epochs
device: Device for computation
Returns:
dict: Training history and metrics
"""
# Initialize optimizers
g_optimizer = optim.Adam(generator.parameters(), lr=0.0002, betas=(0.5, 0.999))
d_optimizer = optim.Adam(discriminator.parameters(), lr=0.0002, betas=(0.5, 0.999))
# Loss function
criterion = nn.BCELoss()
# Training history
history = defaultdict(list)
for epoch in range(num_epochs):
for batch_idx, (real_images, labels) in enumerate(data_loader):
batch_size = real_images.size(0)
real_images = real_images.to(device)
# Labels for loss computation
real_labels = torch.ones(batch_size, 1).to(device)
fake_labels = torch.zeros(batch_size, 1).to(device)
# ========== Train Discriminator ==========
# TODO: Implement discriminator training step
# 1. Zero gradients
# 2. Forward pass on real images
# 3. Compute real loss
# 4. Generate fake images from random z
# 5. Forward pass on fake images (detached)
# 6. Compute fake loss
# 7. Backward and optimize
# ========== Train Generator ==========
# TODO: Implement generator training step
# 1. Zero gradients
# 2. Generate fake images
# 3. Forward pass through discriminator
# 4. Compute adversarial loss
# 5. Backward and optimize
# Log metrics
if batch_idx % 10 == 0:
history['d_loss'].append(d_loss.item())
history['g_loss'].append(g_loss.item())
history['epoch'].append(epoch + batch_idx/len(data_loader))
# Analyze mode collapse every 10 epochs
if epoch % 10 == 0:
mode_coverage = analyze_mode_coverage(generator, device)
history['mode_coverage'].append(mode_coverage)
print(f"Epoch {epoch}: Mode coverage = {mode_coverage:.2f}")
return history
def analyze_mode_coverage(generator, device, n_samples=1000):
"""
Measure mode coverage by counting unique letters in generated samples.
Args:
generator: Trained generator network
device: Device for computation
n_samples: Number of samples to generate
Returns:
float: Coverage score (unique letters / 26)
"""
# TODO: Generate n_samples images
# Use provided letter classifier to identify generated letters
# Count unique letters produced
# Return coverage score (0 to 1)
pass
def visualize_mode_collapse(history, save_path):
"""
Visualize mode collapse progression over training.
Args:
history: Training metrics dictionary
save_path: Output path for visualization
"""
# TODO: Plot mode coverage over time
# Show which letters survive and which disappear
passPart D: Implementing Fixes for Mode Collapse
The starter code provides three techniques to combat mode collapse. Choose ONE to implement in fixes.py:
"""
GAN stabilization techniques to combat mode collapse.
"""
import torch
import torch.nn as nn
import torch.nn.functional as F
import copy
def train_gan_with_fix(generator, discriminator, data_loader,
num_epochs=100, fix_type='feature_matching'):
"""
Train GAN with mode collapse mitigation techniques.
Args:
generator: Generator network
discriminator: Discriminator network
data_loader: DataLoader for training data
num_epochs: Number of training epochs
fix_type: Stabilization method ('feature_matching', 'unrolled', 'minibatch')
Returns:
dict: Training history with metrics
"""
if fix_type == 'feature_matching':
# Feature matching: Match statistics of intermediate layers
# instead of just final discriminator output
def feature_matching_loss(real_images, fake_images, discriminator):
"""
TODO: Implement feature matching loss
Extract intermediate features from discriminator
Match mean statistics: ||E[f(x)] - E[f(G(z))]||²
Use discriminator.features (before final classifier)
"""
pass
elif fix_type == 'unrolled':
# Unrolled GANs: Look ahead k discriminator updates
def unrolled_discriminator(discriminator, real_data, fake_data, k=5):
"""
TODO: Implement k-step unrolled discriminator
Create temporary discriminator copy
Update it k times
Compute generator loss through updated discriminator
"""
pass
elif fix_type == 'minibatch':
# Minibatch discrimination: Let discriminator see batch statistics
class MinibatchDiscrimination(nn.Module):
"""
TODO: Add minibatch discrimination layer to discriminator
Compute L2 distance between samples in batch
Concatenate statistics to discriminator features
"""
pass
# Training loop with chosen fix
# TODO: Implement modified training using selected technique
passPart E: Analysis and Experiments
Complete evaluate.py to analyze your trained models:
"""
Analysis and evaluation experiments for trained GAN models.
"""
import torch
import numpy as np
import matplotlib.pyplot as plt
def interpolation_experiment(generator, device):
"""
Interpolate between latent codes to generate smooth transitions.
TODO:
1. Find latent codes for specific letters (via optimization)
2. Interpolate between them
3. Visualize the path from A to Z
"""
pass
def style_consistency_experiment(conditional_generator, device):
"""
Test if conditional GAN maintains style across letters.
TODO:
1. Fix a latent code z
2. Generate all 26 letters with same z
3. Measure style consistency
"""
pass
def mode_recovery_experiment(generator_checkpoints):
"""
Analyze how mode collapse progresses and potentially recovers.
TODO:
1. Load checkpoints from different epochs
2. Measure mode coverage at each checkpoint
3. Identify when specific letters disappear/reappear
"""
passDeliverables
Your problem1/ directory must contain:
- All code files as specified above
results/training_log.jsonwith loss curves and mode coverage metricsresults/best_generator.pth- saved model weightsresults/mode_collapse_analysis.png- visualization of mode collapseresults/visualizations/containing:- Generated letter grids at epochs 10, 30, 50, 100
- Mode coverage histogram (which letters survive)
- Interpolation sequences
- Comparison of vanilla vs fixed GAN
Your report must include analysis of:
- Why certain letters (like O, A) survive mode collapse while others (Q, X, Z) disappear
- Quantitative comparison of mode coverage with and without your chosen fix
- Discussion of training dynamics: when does collapse begin?
- Evaluation of your chosen stabilization technique’s effectiveness
Problem 2: Hierarchical VAE for Music Generation
Build a Variational Autoencoder that learns to generate drum patterns with controllable style. You will implement hierarchical latent variables, handle discrete outputs, and prevent posterior collapse.
Part A: Dataset and Representation
You will work with a dataset of drum patterns represented as binary matrices.
The dataset contains: - Format: 16×9 binary matrices (16 timesteps, 9 drum instruments) - Instruments: Kick, Snare, Closed Hi-hat, Open Hi-hat, Tom1, Tom2, Crash, Ride, Clap - Styles: Rock, Jazz, Hip-hop, Electronic, Latin (200 patterns each) - Total: 1000 unique drum patterns from MIDI files
The starter code provides dataset.py with the data loader:
"""
Dataset loader for drum pattern generation task.
"""
import torch
from torch.utils.data import Dataset
import numpy as np
import json
import os
class DrumPatternDataset(Dataset):
def __init__(self, data_dir, split='train'):
"""
Initialize drum pattern dataset.
Args:
data_dir: Path to drum dataset directory
split: 'train' or 'val'
"""
self.data_dir = data_dir
self.split = split
# Load patterns from drum_patterns.npz
data_path = os.path.join(data_dir, 'drum_patterns.npz')
data = np.load(data_path)
# Expected structure:
# patterns: [N, 16, 9] binary arrays
# styles: [N] style labels (0-4)
# metadata: dict with instrument names, style names
# Split data into train/val
n_samples = len(data['patterns'])
n_train = int(0.8 * n_samples)
if split == 'train':
self.patterns = data['patterns'][:n_train]
self.styles = data['styles'][:n_train]
else:
self.patterns = data['patterns'][n_train:]
self.styles = data['styles'][n_train:]
# Load metadata
metadata_path = os.path.join(data_dir, 'drum_metadata.json')
with open(metadata_path, 'r') as f:
self.metadata = json.load(f)
self.instrument_names = self.metadata['instruments']
self.style_names = self.metadata['styles']
def __len__(self):
return len(self.patterns)
def __getitem__(self, idx):
"""
Return a drum pattern sample.
Returns:
pattern: Binary tensor of shape [16, 9]
style: Integer style label (0-4)
density: Float indicating pattern density (for analysis)
"""
pattern = self.patterns[idx]
style = self.styles[idx]
# Convert to tensor
pattern_tensor = torch.from_numpy(pattern).float()
# Compute density metric (fraction of active hits)
density = pattern.sum() / (16 * 9)
return pattern_tensor, style, density
def pattern_to_pianoroll(self, pattern):
"""
Convert pattern to visual piano roll representation.
Args:
pattern: Binary array [16, 9] or tensor
Returns:
pianoroll: Visual representation for plotting
"""
if torch.is_tensor(pattern):
pattern = pattern.cpu().numpy()
# Create visual representation with instrument labels
import matplotlib.pyplot as plt
import matplotlib.patches as patches
fig, ax = plt.subplots(figsize=(10, 6))
# Plot each active hit
for t in range(16):
for i in range(9):
if pattern[t, i] > 0.5:
rect = patches.Rectangle((t, i), 1, 1,
linewidth=1,
edgecolor='black',
facecolor='blue')
ax.add_patch(rect)
# Add grid
ax.set_xlim(0, 16)
ax.set_ylim(0, 9)
ax.set_xticks(range(17))
ax.set_yticks(range(10))
ax.set_yticklabels([''] + self.instrument_names)
ax.set_xlabel('Time Step')
ax.set_ylabel('Instrument')
ax.grid(True, alpha=0.3)
ax.invert_yaxis()
return figPart B: Hierarchical VAE Architecture
A hierarchical VAE uses multiple levels of latent variables to capture different aspects of the data. The provided architecture design separates high-level style from low-level variations:
Implement the VAE architecture in hierarchical_vae.py:
"""
Hierarchical VAE for drum pattern generation.
"""
import torch
import torch.nn as nn
import torch.nn.functional as F
class HierarchicalDrumVAE(nn.Module):
def __init__(self, z_high_dim=4, z_low_dim=12):
"""
Two-level VAE for drum patterns.
The architecture uses a hierarchy of latent variables where z_high
encodes style/genre information and z_low encodes pattern variations.
Args:
z_high_dim: Dimension of high-level latent (style)
z_low_dim: Dimension of low-level latent (variation)
"""
super().__init__()
self.z_high_dim = z_high_dim
self.z_low_dim = z_low_dim
# Encoder: pattern → z_low → z_high
# We use 1D convolutions treating the pattern as a sequence
self.encoder_low = nn.Sequential(
nn.Conv1d(9, 32, kernel_size=3, padding=1), # [16, 9] → [16, 32]
nn.ReLU(),
nn.Conv1d(32, 64, kernel_size=3, stride=2, padding=1), # → [8, 64]
nn.ReLU(),
nn.Conv1d(64, 128, kernel_size=3, stride=2, padding=1), # → [4, 128]
nn.ReLU(),
nn.Flatten() # → [512]
)
# Low-level latent parameters
self.fc_mu_low = nn.Linear(512, z_low_dim)
self.fc_logvar_low = nn.Linear(512, z_low_dim)
# Encoder from z_low to z_high
self.encoder_high = nn.Sequential(
nn.Linear(z_low_dim, 64),
nn.ReLU(),
nn.Linear(64, 32),
nn.ReLU()
)
# High-level latent parameters
self.fc_mu_high = nn.Linear(32, z_high_dim)
self.fc_logvar_high = nn.Linear(32, z_high_dim)
# Decoder: z_high → z_low → pattern
# TODO: Implement decoder architecture
# Mirror the encoder structure
# Use transposed convolutions for upsampling
def encode_hierarchy(self, x):
"""
Encode pattern to both latent levels.
Args:
x: Drum patterns [batch_size, 16, 9]
Returns:
mu_low, logvar_low: Parameters for q(z_low|x)
mu_high, logvar_high: Parameters for q(z_high|z_low)
"""
# Reshape for Conv1d: [batch, 16, 9] → [batch, 9, 16]
x = x.transpose(1, 2).float()
# TODO: Encode to z_low parameters
# TODO: Sample z_low using reparameterization
# TODO: Encode z_low to z_high parameters
pass
def reparameterize(self, mu, logvar):
"""
Reparameterization trick for sampling.
TODO: Implement
z = mu + eps * std where eps ~ N(0,1)
"""
pass
def decode_hierarchy(self, z_high, z_low=None, temperature=1.0):
"""
Decode from latent variables to pattern.
Args:
z_high: High-level latent code
z_low: Low-level latent code (if None, sample from prior)
temperature: Temperature for binary output (lower = sharper)
Returns:
pattern_logits: Logits for binary pattern [batch, 16, 9]
"""
# TODO: If z_low is None, sample from conditional prior p(z_low|z_high)
# TODO: Decode z_high and z_low to pattern logits
# TODO: Apply temperature scaling before sigmoid
pass
def forward(self, x, beta=1.0):
"""
Full forward pass with loss computation.
Args:
x: Input patterns [batch_size, 16, 9]
beta: KL weight for beta-VAE (use < 1 to prevent collapse)
Returns:
recon: Reconstructed patterns
mu_low, logvar_low, mu_high, logvar_high: Latent parameters
"""
# TODO: Encode, decode, compute losses
passPart C: Training Techniques for Discrete Data
Discrete outputs and posterior collapse are major challenges for VAEs. The starter code provides proven techniques:
Use the provided utilities in training_utils.py:
"""
Training implementations for hierarchical VAE with posterior collapse prevention.
"""
import torch
import torch.nn as nn
import numpy as np
from collections import defaultdict
def train_hierarchical_vae(model, data_loader, num_epochs=100, device='cuda'):
"""
Train hierarchical VAE with KL annealing and other tricks.
Implements several techniques to prevent posterior collapse:
1. KL annealing (gradual beta increase)
2. Free bits (minimum KL per dimension)
3. Temperature annealing for discrete outputs
"""
optimizer = torch.optim.Adam(model.parameters(), lr=0.001)
# KL annealing schedule
def kl_anneal_schedule(epoch):
"""
TODO: Implement KL annealing schedule
Start with beta ≈ 0, gradually increase to 1.0
Consider cyclical annealing for better results
"""
pass
# Free bits threshold
free_bits = 0.5 # Minimum nats per latent dimension
history = defaultdict(list)
for epoch in range(num_epochs):
beta = kl_anneal_schedule(epoch)
for batch_idx, patterns in enumerate(data_loader):
patterns = patterns.to(device)
# TODO: Implement training step
# 1. Forward pass through hierarchical VAE
# 2. Compute reconstruction loss
# 3. Compute KL divergences (both levels)
# 4. Apply free bits to prevent collapse
# 5. Total loss = recon_loss + beta * kl_loss
# 6. Backward and optimize
pass
return history
def sample_diverse_patterns(model, n_styles=5, n_variations=10, device='cuda'):
"""
Generate diverse drum patterns using the hierarchy.
TODO:
1. Sample n_styles from z_high prior
2. For each style, sample n_variations from conditional p(z_low|z_high)
3. Decode to patterns
4. Organize in grid showing style consistency
"""
pass
def analyze_posterior_collapse(model, data_loader, device='cuda'):
"""
Diagnose which latent dimensions are being used.
TODO:
1. Encode validation data
2. Compute KL divergence per dimension
3. Identify collapsed dimensions (KL ≈ 0)
4. Return utilization statistics
"""
passPart D: Training Implementation
The starter code provides train.py with the training loop structure:
import torch
import torch.nn as nn
import torch.optim as optim
from torch.utils.data import DataLoader
import json
def compute_hierarchical_elbo(recon_x, x, mu_low, logvar_low, mu_high, logvar_high, beta=1.0):
"""
Compute ELBO for hierarchical VAE.
ELBO = E[log p(x|z_low)] - beta * KL(q(z_low|x) || p(z_low|z_high))
- beta * KL(q(z_high|z_low) || p(z_high))
Args:
recon_x: Reconstructed pattern logits [batch, 16, 9]
x: Original patterns [batch, 16, 9]
mu_low, logvar_low: Low-level latent parameters
mu_high, logvar_high: High-level latent parameters
beta: KL weight
Returns:
loss: Total loss
recon_loss: Reconstruction component
kl_low: KL for low-level latent
kl_high: KL for high-level latent
"""
# Reconstruction loss (binary cross-entropy)
recon_loss = F.binary_cross_entropy_with_logits(
recon_x.view(-1), x.view(-1), reduction='sum'
)
# TODO: Implement KL divergences
# KL(q(z_high) || p(z_high)) where p(z_high) = N(0, I)
kl_high = -0.5 * torch.sum(1 + logvar_high - mu_high.pow(2) - logvar_high.exp())
# TODO: KL(q(z_low) || p(z_low|z_high))
# This is more complex - can simplify to standard KL for now
kl_low = -0.5 * torch.sum(1 + logvar_low - mu_low.pow(2) - logvar_low.exp())
return recon_loss + beta * (kl_low + kl_high), recon_loss, kl_low, kl_high
def train_epoch(model, data_loader, optimizer, epoch, device):
"""
Train for one epoch with annealing schedules.
"""
model.train()
total_loss = 0
# Get annealing parameters for this epoch
beta = kl_annealing_schedule(epoch, method='cyclical')
temperature = temperature_annealing_schedule(epoch)
for batch_idx, (patterns, styles, _) in enumerate(data_loader):
patterns = patterns.to(device)
# TODO: Forward pass
# TODO: Compute loss with current beta
# TODO: Backward and optimize
# Log progress
if batch_idx % 10 == 0:
print(f'Epoch {epoch}, Batch {batch_idx}: Loss = {loss.item():.4f}, '
f'Beta = {beta:.3f}, Temp = {temperature:.2f}')
return total_loss / len(data_loader)
def main():
# Configuration
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
batch_size = 32
num_epochs = 100
learning_rate = 0.001
# TODO: Initialize dataset, model, optimizer
# TODO: Training loop with logging
# TODO: Save checkpoints and final model
pass
if __name__ == '__main__':
main()Part E: Analysis and Music Generation
Complete analyze_latent.py to analyze the trained model and generate music:
"""
Latent space analysis tools for hierarchical VAE.
"""
import torch
import numpy as np
import matplotlib.pyplot as plt
from sklearn.manifold import TSNE
def visualize_latent_hierarchy(model, data_loader, device='cuda'):
"""
Visualize the two-level latent space structure.
TODO:
1. Encode all data to get z_high and z_low
2. Use t-SNE to visualize z_high (colored by genre)
3. For each z_high cluster, show z_low variations
4. Create hierarchical visualization
"""
pass
def interpolate_styles(model, pattern1, pattern2, n_steps=10, device='cuda'):
"""
Interpolate between two drum patterns at both latent levels.
TODO:
1. Encode both patterns to get latents
2. Interpolate z_high (style transition)
3. Interpolate z_low (variation transition)
4. Decode and visualize both paths
5. Compare smooth vs abrupt transitions
"""
pass
def measure_disentanglement(model, data_loader, device='cuda'):
"""
Measure how well the hierarchy disentangles style from variation.
TODO:
1. Group patterns by genre
2. Compute z_high variance within vs across genres
3. Compute z_low variance for same genre
4. Return disentanglement metrics
"""
pass
def controllable_generation(model, genre_labels, device='cuda'):
"""
Test controllable generation using the hierarchy.
TODO:
1. Learn genre embeddings in z_high space
2. Generate patterns with specified genre
3. Control complexity via z_low sampling temperature
4. Evaluate genre classification accuracy
"""
passPart F: Creative Experiments
Create a notebook experiments.ipynb with the following analyses:
- Genre Blending: Interpolate between jazz and rock patterns
- Complexity Control: Find latent dimensions that control pattern density
- Humanization: Add controlled variations to mechanical patterns
- Style Consistency: Generate full drum tracks with consistent style
Deliverables
Your problem2/ directory must contain:
- All code files as specified above
results/training_log.jsonwith loss curves and KL valuesresults/best_model.pth- saved model weightsresults/generated_patterns/containing:- 10 samples from each style
- Interpolation sequences
- Style transfer examples
results/latent_analysis/containing:- t-SNE visualization of latent space
- Disentanglement analysis
- Dimension interpretation results
results/audio_samples/with generated drum loops (optional but encouraged)
Your report must include analysis of:
- Evidence of posterior collapse and how annealing prevented it
- Interpretation of what each latent dimension learned to control
- Quality assessment: Do generated patterns sound musical?
- Comparison of different annealing strategies
- Success rate of style transfer while preserving rhythm
Submission Requirements
Your GitHub repository must follow this exact structure:
ee641-hw2-[username]/
├── problem1/
│ ├── models.py
│ ├── dataset.py
│ ├── models.py
│ ├── training_dynamics.py
│ ├── fixes.py
│ ├── train.py
│ ├── evaluate.py
│ └── results/
│ ├── training_log.json
│ ├── best_generator.pth
│ ├── mode_collapse_analysis.png
│ └── visualizations/
├── problem2/
│ ├── dataset.py
│ ├── hierarchical_vae.py
│ ├── training_utils.py
│ ├── train.py
│ ├── analyze_latent.py
│ └── results/
│ ├── training_log.json
│ ├── best_model.pth
│ ├── generated_patterns/
│ └── latent_analysis/
├── report.pdf
└── README.mdThe README.md in your repository root must contain:
- Your full name
- USC email address
- Instructions to run each problem if they differ from the standard commands
- Any implementation notes
TipTesting Your Submission
Before submitting:
- Your repository structure must match the requirement exactly
python train.pymust run without errors in each problem directory- All output files must be generated in the correct locations