MetaEncoder - a facebook Collection - One Encoder for Retrieval and Decision Making
Reported by r/LocalLLaMA1 source ↓
A community post describes MetaEncoder, a Facebook model that scores candidate items against a natural-language task, with images and video available as additional information.
Key points
- The system scores candidates described in natural language against a task also expressed in natural language
- Candidate representations can be cached and compared with an inner product, according to the model description
- The post says the model was contrastively fine-tuned from Muse-Glimmer-30B
- MetaEncoder and an improved version are listed on Hugging Face; an evaluation repository is on GitHub
Details
The r/LocalLLaMA post describes MetaEncoder as a multimodal system for retrieval and decision-making. Users can state a task and candidates in free-form language; images and video can provide additional information. The model returns scores for the candidates according to the task.
The post says tasks and candidates are encoded separately, allowing candidate representations to be cached and compared through an inner product. It says this setup can work with approximate-nearest-neighbor indexing for large candidate sets. The post describes the model as contrastively fine-tuned from meta-models/Muse-Glimmer-30B.
The post lists a MetaEncoder model and an improved v1 version on Hugging Face, as well as a GitHub evaluation repository and a research paper. It says the original model card specifies an Apache-2.0 license, while the v1 card has no license section.
Written from coverage as of