RiskLab: Multi-Agent Risk Simulation Framework

RiskLab is a controlled multi-agent interaction framework for instantiating, probing, and measuring emergent social risks in LLM-based agent collectives. It accompanies the paper “Emergent Social Intelligence Risks of Multi-Agent Systems” (Yue Huang et al., 2025).

Note

RiskLab is designed as a config-driven framework. Define your experiment in a single YAML file — agents, topology, environment, protocol, and risk metrics — then run it with one command.

Key Features

🎯 13-Risk Taxonomy: Covers cooperative, competitive, and collective interaction risks — 4 implemented detectors, 9 extensible slots
🔧 Five-Tuple Architecture: Experiments are defined as (Topology, Environment, Protocol, Agent, Task) — each component independently configurable
📊 Config-Driven Execution: One YAML file defines the entire experiment — agents, connectivity, environment, protocol, and risk metrics
🔌 Extensible Design: Registry pattern for agents, environments, protocols, and risk detectors — add new components without modifying core code
📈 Built-in Analysis: Trajectory logging, risk detection with counterfactual analysis, and multi-seed experiment aggregation
🤖 Multi-Provider LLM Support: Supports OpenAI, Anthropic, and any OpenAI-compatible API with per-agent model assignment

Quick Example

Run a built-in experiment:

cd examples/R2
python run_r2.py --condition C1

As a library:

from risklab.experiments.config_loader import (
    load_experiment_config,
    build_experiment_from_config,
)
from risklab.experiments.runner import ExperimentRunner

config = load_experiment_config("examples/R2/configs/r2_C1_basic.yaml")
components = build_experiment_from_config(config)
runner = ExperimentRunner(**components)
results = runner.run()  # list[dict] — one per seed

Minimal YAML config:

experiment:
  id: my_experiment

llm_config_path: "llm_config.yaml"

topology:
  agents: ["A", "B", "C"]
  flow:
    cyclic: true
    stop_conditions:
      - type: max_rounds
        value: 5

environment:
  name: homogeneous_goods_market
  type: competitive
  parameters:
    marginal_cost: 10
    price_range: [10, 100]

protocol:
  type: market_turn_based

agents:
  - agent_id: A
    role: seller
    model: gpt-4o
    objective: selfish
  - agent_id: B
    role: seller
    model: gpt-4o
    objective: selfish
  - agent_id: C
    role: seller
    model: gpt-4o
    objective: selfish

risks:
  - name: tacit_collusion
    parameters:
      marginal_cost: 10

Why RiskLab?

Standard multi-agent benchmarks focus on task performance, but overlook emergent social dynamics:

❌ No framework to study inter-agent risk phenomena
❌ Risks like collusion, groupthink, and free-riding go undetected
❌ No reproducible methodology for risk experiments

RiskLab solves this by:

✅ Providing a controlled simulation environment for risk phenomena
✅ Implementing quantitative risk detectors with scoring and counterfactual analysis
✅ Offering a reproducible, config-driven experiment pipeline
✅ Supporting an extensible taxonomy — add new risks as plug-in detectors

Use Cases

Risk Research — Study emergent social risks in LLM agent groups
Safety Evaluation — Probe multi-agent systems for collusion, manipulation, or deadlock
Benchmark Development — Create reproducible risk benchmarks for multi-agent systems
Framework Extension — Add custom risk detectors for domain-specific phenomena

Documentation

Getting Started

Tutorials

Additional Resources

License

MIT License — see the LICENSE file for details.