What Would a Next-Gen Emulator Look Like?

What if an emulator wasn't just a compatibility layer, but a full development platform? This post outlines the architecture for a next-generation emulator that combines Apple Metal rendering, real-time AI opponents, massively parallel training infrastructure, and the developer tooling that fighting game communities actually want.

Most Requested Features Across Popular Emulators

Emulation communities consistently highlight several key features that enrich gameplay, improve latency, and aid development. Below is a compilation of highly requested features from platforms like RetroArch, MAME, FBNeo, Dolphin, PCSX2, etc.

AI-Powered Augmentations

Developer Tooling & Debuggers

In-Game Training & Practice Features

These are highly sought after in the fighting game community:

Latency Reduction & UX Enhancements

Online Play and Tournament Features

FBNeo Metal Build - Project Goals

Primary Objectives

Key Goals

1. Metal Rendering Backend on macOS: Replace or augment existing video renderer with Apple Metal for better performance and future-proofing on macOS (especially Apple Silicon)

2. Integrated AI Opponent Support: Introduce real-time AI-controlled opponents selectable via UI dropdown, allowing human players to spar with trained AI

3. Massively Parallel Training Mode: Enable running many emulator instances (100+ if possible) in parallel (headless, no GUI) for training machine learning models

4. Leverage Apple's ML Stack: Use Metal Performance Shaders (MPS), MPSGraph, and CoreML for GPU-accelerated neural network operations

5. Focus Titles: Ensure Marvel vs. Capcom and Street Fighter III: 3rd Strike are fully supported

Architecture Overview

Metal Renderer Module

AI Manager Module

Training Mode Architecture

Parallel Instance Execution

Game Environment Interface

class GameEnv:
    def __init__(self, player_side: int):
        self.player_side = player_side
        self.game_over = False
        BurnDrvInit(nBurnDrvActive)
    
    def reset(self):
        """Reset game to start new episode"""
        BurnDrvExit()
        BurnDrvInit(nBurnDrvActive)
        self.game_over = False
    
    def step(self, action: int) -> tuple[GameState, float, bool]:
        """Step game by one frame with given action"""
        self.apply_action(action)
        BurnDrvFrame()
        state = self.capture_state()
        reward = self.compute_reward(state)
        done = state.is_terminal
        return state, reward, done
    
    def apply_action(self, action: int):
        """Map action index to button presses"""
        InpClearPlayer(self.player_side)
        # Map action to inputs...

Apple ML Stack Integration

CoreML for Inference

MPS for Training

Build System

Makefile Structure

# Feature flags
USE_METAL := 1
USE_COREML := 1
USE_PYTORCH := 1
HEADLESS_MODE := 0
 
# Compiler flags
CXXFLAGS := -std=c++17 -ObjC++ -Wall -O2 -arch arm64
 
# Framework linking
ifdef USE_METAL
LDFLAGS += -framework Metal -framework MetalKit -framework Cocoa
endif
 
ifdef USE_COREML
LDFLAGS += -framework CoreML
endif
 
ifdef USE_PYTORCH
LDFLAGS += -L$(LIBTORCH_PATH)/lib -ltorch -lc10
endif

Build Targets

Neural Network Architecture

Policy Network

class PolicyNet(nn.Module):
    def __init__(self, input_channels=4, num_actions=10):
        super().__init__()
        self.net = nn.Sequential(
            nn.Conv2d(input_channels, 32, 5, stride=2),
            nn.ReLU(),
            nn.Conv2d(32, 64, 3, stride=2),
            nn.ReLU(),
            nn.Flatten(),
            nn.Linear(64 * 19 * 19, 128),
            nn.ReLU(),
            nn.Linear(128, num_actions)
        )
    
    def forward(self, x):
        return self.net(x)

Training Loop

def training_step(policy, optimizer, batch):
    states, actions, rewards, next_states, dones = batch
    
    # Compute Q-values
    pred_q = policy(states).gather(1, actions.unsqueeze(1)).squeeze()
    
    # Compute target Q
    with torch.no_grad():
        next_q = policy(next_states).max(1).values
        target_q = rewards + 0.99 * next_q * (1 - dones.float())
    
    # Loss and optimization
    loss = F.mse_loss(pred_q, target_q)
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
    
    return loss.item()

Headless Mode

Configuration

#ifdef HEADLESS_MODE
// Disable all video/audio initialization
#define NO_VIDEO_OUTPUT
#define NO_AUDIO_OUTPUT
 
// Run at maximum speed (no frame limiting)
#define UNCAPPED_FRAMERATE
 
// Enable training API
#define TRAINING_API_ENABLED
#endif

Command Line Usage

# Run training with 100 parallel instances
./fbneo_headless --train-ai \
    --instances=100 \
    --game=mvsc \
    --iterations=100000 \
    --model-output=./models/mvc_agent_v1.pt

Reward Function Design

Fighting Game Rewards

def compute_reward(state, prev_state, player_side):
    reward = 0.0
    
    # Health differential
    if player_side == 2:
        opp_health = state.p1_health
        ai_health = state.p2_health
        prev_opp = prev_state.p1_health
        prev_ai = prev_state.p2_health
    else:
        opp_health = state.p2_health
        ai_health = state.p1_health
        prev_opp = prev_state.p2_health
        prev_ai = prev_state.p1_health
    
    # Damage dealt/taken
    reward += (prev_opp - opp_health) * 1.0
    reward -= (prev_ai - ai_health) * 1.0
    
    # Terminal rewards
    if state.is_terminal:
        if opp_health <= 0:
            reward += 100.0  # Win
        elif ai_health <= 0:
            reward -= 100.0  # Loss
    
    return reward

Future Enhancements

AI Explainability Overlays

Visualize what the AI is "thinking" during matches:

• Highlight important screen regions
• Show predicted actions
• Display evaluation scores

AI vs AI Tournaments

Run accelerated matches between different AI models:

• Evaluate training progress
• Compare strategies
• Self-play improvement

Plugin Architecture

Real-World Validation: The MvC1 Phoenix Edition

Our MvC1 Phoenix Edition project demonstrates why these tools matter. Using existing emulator infrastructure (MAME debugger, Ghidra + GhidraMCP, Lua scripting), we:

• Decompiled 5 functions from a 1998 encrypted ROM binary, including all secret character load functions
• Mapped a 32-byte character entry structure at $065CCA with player runtime offsets (ID at +$53, health at +$60, unlock flags at $FF803C-$FF804C)
• Discovered 4 conditional load points for an unreleased character (ID 0x2E) hidden in the code, 3 of which share code paths with Shadow Lady
• Reverse engineered 3 different table formats for secret characters, each implemented at different development stages
• Built a 128MB-GFX / 16MB-audio padded ROM set (the CPS-2 file-format ceiling) via automated Python pipeline. Corrected July 2026: padding alone is not addressable -- the stock emulator masks GFX at 32MB and QSound at 8MB. The caps were genuinely broken later by a custom FBNeo core (extended d_cps2.cpp + mapper_mvsc64 + OBJ Y-bit12→code-bit18 hook) reaching a pixel-proven 64MB GFX + 16MB QSound, at the cost of standard Fightcade netplay compatibility
• Injected a 24-line diagnostic verification screen (ULTRA SETTINGS) into the game's F2 Test Menu using 68000 assembly, with hardware-polled input, interactive sound test, and interrupt-disabled clean display
• Patched the Z80 audio CPU ROM with hand-assembled Z80 machine code: dispatch table hook (type $0A → custom handler), QSound channel 15 programming, and a multi-bank streaming engine via ISR exit hook for full-length playback of injected tracks (sound codes $0900+)
• CPS-2 cross-game audio import: Imported 8MB of MSHVSSF QSound sample data directly at the ROM level (zero conversion -- same CPS-2 format). Also verified with XMSF + XCOTA (4MB + 4MB). Corrected July 2026: banks past 8MB wrap under the stock emulator; the imported region is genuinely playable only under the custom FBNeo core's 16MB QSound driver. QSound DSP's 24-bit addressing (16MB max) remains the hardware ceiling

All of this was done with existing tools -- imagine what's possible with purpose-built developer infrastructure that integrates AI analysis, real-time memory visualization, and automated function discovery.

Summary

This project aims to create a next-generation emulator platform that combines:

1. Modern Graphics: Metal rendering for optimal macOS performance
2. AI Integration: Real-time AI opponents using CoreML/PyTorch
3. Training Infrastructure: Massively parallel headless instances
4. Developer Tools: Full debugging and scripting support

The result will be both a top-tier emulator for players and a cutting-edge platform for AI research in retro gaming.