Browser

Browser-Based ML: WebGPU and WebAssembly

Browser ML Technologies
| Technology | Purpose | Performance |
|------------|---------|-------------|
| WebGPU | GPU compute in browser | Fast |
| WebGL | Graphics + limited compute | Medium |
| WebAssembly | Near-native CPU | Fast |
| JavaScript | Pure JS execution | Slow |

WebGPU for ML

// Check WebGPU support
if (!navigator.gpu) {
    console.log("WebGPU not supported");
    return;
}

// Get GPU adapter and device
const adapter = await navigator.gpu.requestAdapter();
const device = await adapter.requestDevice();

// Create compute pipeline for matrix multiply
const shaderModule = device.createShaderModule({
    code: `
        @compute @workgroup_size(8, 8)
        fn main(@builtin(global_invocation_id) global_id: vec3<u32>) {
            // Matrix multiplication kernel
        }
    `
});

Frameworks

Transformers.js

import { pipeline } from "@xenova/transformers";

// Load model (downloads to browser cache)
const classifier = await pipeline("sentiment-analysis");

// Run inference
const result = await classifier("I love this product!");
console.log(result);  // [{label: "POSITIVE", score: 0.99}]

ONNX Runtime Web

import * as ort from "onnxruntime-web";

// Load model
const session = await ort.InferenceSession.create("model.onnx");

// Prepare input
const tensor = new ort.Tensor("float32", inputData, [1, 3, 224, 224]);

// Run inference
const results = await session.run({ input: tensor });
console.log(results.output.data);

WebLLM

import { CreateMLCEngine } from "@mlc-ai/web-llm";

const engine = await CreateMLCEngine("Llama-2-7b-chat-hf-q4f16_1");

const response = await engine.chat.completions.create({
    messages: [{ role: "user", content: "Hello!" }],
    stream: true
});

Performance Comparison
| Backend | Relative Speed |
|---------|----------------|
| Native CUDA | 100% |
| WebGPU | 20-40% |
| WebGL | 10-20% |
| WASM SIMD | 5-15% |
| Pure JS | 1-5% |

Model Size Considerations
| Model | Size | Browser Suitable |
|-------|------|------------------|
| DistilBERT | 250MB | Yes |
| BERT base | 440MB | Yes |
| Llama 7B Q4 | 3.5GB | Challenging |
| GPT-2 | 500MB | Yes |

Best Practices
- Use WebGPU when available, fallback to WebGL/WASM
- Cache models in IndexedDB
- Show download progress
- Use streaming for large models
- Consider model splitting
- Test across browsers (Chrome, Firefox, Safari)