RustScope Engineering Update

From the Editor

We raised $5,000 from Superteam Solana to accelerate RustScope toward v1.0. This grant funds the critical-path workstreams outlined below — timeline correlation, hotspot capture reliability, CI/CD integration, continuous profiling, distributed tracing, and export interop.

Think of this as the public engineering journal for RustScope. Every section outlines a concrete problem, the target architecture (with DFDs), the implementation plan, and what success looks like.

Grant: Superteam Solana

This grant directly funds Phases 2 & 3 of the v1.0 roadmap. Milestones and progress will be posted here as we ship.

1. Timeline Correlation Engine

Problem: RustScope captures process metrics (CPU, memory, threads) and function hotspots, but they are not correlated in time. You see a memory spike happened, but you cannot click it to reveal which function caused it.

Goal: Build a time-correlated session analysis engine where every spike, plateau, and dip in system metrics maps to the exact functions dominating that window.

DFD — Correlation Pipeline

2. Hotspot Capture Overhaul

Problem: macOS sample has SIP restrictions and fragile symbol extraction. Linux perf works but symbolization fails for Rust binaries without debug symbols.

Target: >90% successful symbolization on both platforms for real-world Rust binaries.

DFD — Symbol Resolution Fallback Chain

Code

fn resolve_symbols_with_atos(pid: u32, addresses: &[u64]) -> HashMap<u64, String> {
    let mut cmd = Command::new("xcrun");
    cmd.arg("atos").arg("-p").arg(pid.to_string())
       .arg("-fullPath")
       .stdin(Stdio::piped()).stdout(Stdio::piped());
    let mut child = cmd.spawn()?;
    // feed addresses via stdin, parse "symbol + offset" output
}

pub struct SymbolDB {
    symbols: HashMap<u64, (String, String)>,
}
impl SymbolDB {
    pub fn from_self() -> Self {
        unsafe { libc::dl_iterate_phdr(Some(Self::callback), ...) }
    }
}

3. CI/CD Integration & Regression Gating

Problem: RustScope has a compare CLI mode but no turnkey CI/CD templates. Teams want performance regression gating with minimal config.

Goal: Production-ready CI templates that automatically profile before/after, compare, and report as PR comments or MR widgets.

DFD — CI/CD Pipeline Flow

Action Definition

# .github/actions/rustscope/action.yml
name: "RustScope Performance Check"
inputs:
  threshold:   { default: "10" }
  fail-on:     { default: "critical" }
runs:
  using: "docker"
  image: "ghcr.io/anurag/rustscope-action:latest"

New --ci-output Flag

rustscope --compare-baseline base.json --compare-current curr.json --ci-output github
rustscope --compare-baseline base.json --compare-current curr.json --ci-output gitlab
rustscope --compare-baseline base.json --compare-current curr.json --ci-output json

Statistical Significance (Welch's t-test)

pub fn compare_with_significance(
    baseline: &[f64],
    current: &[f64],
    threshold: f64,
) -> RegressionResult {
    let t_stat = welch_t_test(baseline, current);
    let p_value = students_t_cdf(t_stat, df);
    // Only flag if: pct_change > threshold AND p_value < 0.05
}

4. Continuous Profiling Ring Buffer

Problem: Current profiling is session-based (start → stop → analyze). For production debugging, we need continuous, low-overhead profiling that retains the last N minutes and lets you "go back in time."

Goal: Ring-buffer continuous profiler at <1% CPU overhead.

DFD — Ring Buffer Architecture

Lock-Free Ring Buffer

struct LockFreeRingBuffer {
    buffer: *mut SampleEntry,          // mmap'd
    capacity: usize,
    write_pos: AtomicU64,              // sampling thread only
    read_pos: AtomicU64,               // analysis thread only
}
impl LockFreeRingBuffer {
    fn push(&self, entry: &SampleEntry) -> bool {
        let wp = self.write_pos.load(Ordering::Relaxed);
        let rp = self.read_pos.load(Ordering::Acquire);
        if wp.wrapping_sub(rp) >= (self.capacity - 1) as u64 {
            return false; // buffer full, drop
        }
        unsafe { *self.buffer.add((wp % self.capacity as u64) as usize) = *entry; }
        self.write_pos.store(wp.wrapping_add(1), Ordering::Release);
        true
    }
}

CLI Usage

# 60-minute retention, 8MB buffer
rustscope --pid 12345 --continuous --retention-minutes 60 --buffer-mb 8

# Signal-based control
kill -USR1 <rustscope-pid>  # flush buffer → JSON
kill -USR2 <rustscope-pid>  # clear buffer

5. OpenTelemetry Distributed Tracing

Problem: RustScope profiles a single process. In microservices, you need to correlate profiler data with distributed traces.

Goal: Bridge RustScope output with OpenTelemetry so performance data is visible in the same context as traces.

DFD — OTel Integration Flow

Span Attributes

// Injected by #[profile] macro:
span.set_attribute(Attribute::new("rustscope.profile_id", profile_id));
span.set_attribute(Attribute::new("rustscope.function", function_name));

CLI Flags

# Export to local OTEL collector
rustscope --pid 12345 --otel --otel-endpoint http://localhost:4317

# Continuous + OTEL stream
rustscope --continuous --otel --otel-period-sec 60

New TraceTab.tsx in the visualizer — fetches traces from Jaeger API and renders inline flame graphs for spans that have rustscope.profile_id.

6. Export Interoperability

Problem: RustScope exports to pprof, Chrome Tracing, SpeedScope, and collapsed stacks — but round-trip fidelity is untested. Metadata (crate names, module paths, lock data) is lost.

Goal: 100% round-trip fidelity for all export formats.

Current State: What's Lost

Target: Metadata Preservation

// pprof: use labels
Label { key: "crate", str: func.crate_name }

// Chrome Tracing: use args
Event { args: json!({"crate": func.crate_name, "module": func.module_path}) }

// SpeedScope: use "other" metadata map
profile.other.insert("rustscope", json!({ "crate_rollups": ... }));

// Collapsed stacks: embed in name
"rustscope[my-crate][handlers]::process_order 150000000"

Round-Trip Test Suite

#[test]
fn pprof_roundtrip_fidelity() {
    let session = load_fixture("complex_session.json");
    let pprof_bytes = export_pprof(&session).unwrap();
    let imported = import_pprof(&pprof_bytes).unwrap();
    assert_eq!(session.functions.len(), imported.functions.len());
    assert_eq!(session.crate_rollups.len(), imported.crate_rollups.len());
}

Import Support (New)

rustscope --import-pprof session.pb.gz --export-json rustscope-format.json

7. v1.0 Roadmap & Execution Phases

Definition of Done (v1.0)

Success Metrics