Skip to content

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

2 Commits

Folders and files

Repository files navigation

Multi-Agent Aggregate

This project studies multi-agent aggregation and reliability modeling across multimodal behavioral prediction benchmarks.

Directory Layout

benchmarks/
  webpage_likability/          Web aesthetics screenshots and human ratings
  tweet_engagement/            Content Behavior Corpus / tweet-like engagement data
  lambda_ad_memorability/      Ad memorability metadata
  ad_ctr_prediction/           Placeholder for Ad CTR data if a public source is found
methods/
  webpage_likability/          Main benchmark: generation, aggregators, summaries
  content_behavior_corpus/     CBC generation and aggregation experiments
  lambda_ad_memorability/      LAMBDA generation and MVP evaluation
third_party/
  MM-TPP/                      Clean copy of the official MM-TPP implementation
results/
  README.md                    Reserved for cross-benchmark summaries
docs/
  papers/                      Relevant papers
  notes/                       Project notes and experiment interpretation
scripts/                       Stable entry scripts

Current Status

  • Each active benchmark keeps its own generation scripts, raw agent outputs, aggregation code, and summaries under methods/<benchmark>/.
  • webpage_likability is the current main line because its multi-agent oracle and learned aggregation results are strongest.
  • content_behavior_corpus and lambda_ad_memorability are exploratory transfer benchmarks.
  • Old DanmakuTPP and TAXI-PRO workspaces have been retired from this repo.

Method Layout

Within each mature benchmark directory:

methods/<benchmark>/
  scripts/                     Benchmark-specific generation and evaluation scripts
  data_outputs/                Cached raw agent outputs and small smoke/MVP outputs
  aggregators/                 Aggregation and reliability modeling methods
  summaries/                   Result tables and experiment notes

webpage_likability/aggregators/ is organized by method family:

01_oracle/                     Mean, best single agent, oracle upper bounds
02_math_aggregators/           Calibration, routers, cross-attention style aggregators
03_agent_modeling/             EM and reliability modeling methods
04_llm_finetuning/             VLM/LLM judge and LoRA experiments

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages