---
title: "MM-RAG (Multimodal RAG)"
source: "https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)"
wiki: "systems-analysis.info/eng"
article: "MM-RAG_(Multimodal_RAG)"
language: "en"
categories:
  - "Category:English"
  - "Category:Large language models"
  - "Category:Prompt engineering"
  - "Category:Technology"
revision_id: 226
wiki_created_at: 2026-09-06T22:19:11Z
wiki_modified_at: 2026-09-06T22:19:11Z
downloaded_at: 2026-09-07T22:22:02Z
---

# MM-RAG (Multimodal RAG)

**MM-RAG** (*Multimodal Retrieval-Augmented Generation*) is an extension of the classic RAG paradigm in which LLMs use not only text but also visual data (images, diagrams, tables, charts) to generate answers. Multimodal retrieval allows for finding and linking evidence across different representations, reducing the risk of hallucinations by grounding the generation in external sources with precise references to page fragments and regions (*bounding boxes*)<sup>[\[1\]](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_note-lewis2020-1)[\[2\]](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_note-visrag2024-2)</sup>.

MM-RAG is particularly useful for documents where a significant portion of the meaning is conveyed in a non-textual form (page layout, diagrams, table structures). In such cases, classic text-based RAG often loses important contextual elements<sup>[\[3\]](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_note-docvqa2021-3)[\[4\]](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_note-textvqa2019-4)</sup>.

## Context and the Problem Being Solved

Classic RAG operates on text passages and is unaware of visual structures (element layout, figure captions, chart axes). MM-RAG addresses these gaps by extracting structured elements (text, tables, images with coordinates), indexing them into a vector space, and combining evidence from different modalities<sup>[\[5\]](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_note-layoutlm2020-5)[\[6\]](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_note-clip2021-6)</sup>.

## MM-RAG Architecture

The MM-RAG pipeline extends the classic RAG pipeline with stages for visual data processing and modality alignment: **ingestion → indexing → multimodal retrieval → fusion and reranking → generation with traceability**.

1.  **Ingestion and Preprocessing.** The pipeline takes PDFs, scans, or images as input. It performs OCR and **layout analysis** to identify regions such as paragraphs, headings, tables, images, and their coordinates. Common tools include models from the LayoutLM family and libraries like LayoutParser; validation and training often rely on datasets like PubLayNet and DocLayNet<sup>[\[7\]](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_note-publaynet2019-7)[\[8\]](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_note-doclaynet2022-8)[\[9\]](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_note-layoutparser2021-9)</sup>.
2.  **Region Segmentation.** Visual objects (diagrams, tables, illustrations, captions) are extracted. For increased robustness, OCR-free models (e.g., Donut) or combined OCR+VLM pipelines are used<sup>[\[10\]](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_note-donut2021-10)</sup>.
3.  **Indexing (Vector Index).** Text chunks and visual elements (images or their descriptions) are converted into vector representations and stored in a vector database. For the shared *text↔image* space, models like CLIP or SigLIP are used; for production environments, multimodal/multi-vector indexes (one object, multiple vectors) are convenient<sup>[\[6\]](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_note-clip2021-6)[\[11\]](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_note-siglip2023-11)[\[12\]](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_note-milvus-mv-12)</sup>.
4.  **Multimodal Retrieval and Reranking.** A combination of text and visual search is performed. The candidates (paragraphs, tables, images/regions) are combined and reranked by a more heavyweight model (a cross-encoder or LLM reranker) to improve precision<sup>[\[13\]](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_note-cohere-rerank-13)</sup>.
5.  **Context Assembly and Generation.** The selected fragments are fed into an LLM/VLM. If the model is multimodal (e.g., GPT-4V/4o), images can be passed directly; if it's a text-only LLM, images are converted into detailed descriptions beforehand<sup>[\[14\]](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_note-gpt4v-14)[\[15\]](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_note-gpt4o-15)</sup>.
6.  **Traceability and Citation.** The generated answer is accompanied by clickable citations linked not only to the document/page but also to the specific region (coordinates). This enhances *grounding* and user trust<sup>[\[2\]](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_note-visrag2024-2)</sup>.

## Quality Evaluation and Metrics

The effectiveness of MM‑RAG is evaluated at the extraction, retrieval, and generation levels.

- **Visual Data Extraction Quality.** OCR accuracy (WER/CER), layout analysis quality (mAP/Precision/Recall) on datasets like DocLayNet/PubLayNet<sup>[\[8\]](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_note-doclaynet2022-8)[\[7\]](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_note-publaynet2019-7)</sup>.
- **Retrieval Quality.** Standard IR metrics: Recall@K, Precision@K, MRR; for multimodality, these are measured separately for each modality and for the combined results.
- **Answer Quality (End‑to‑End).** Automated metrics for *faithfulness*/*groundedness* and human evaluation. In practice, frameworks like RAGAS, TruLens, and DeepEval are used<sup>[\[16\]](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_note-ragas-16)</sup>.
- **Benchmarks.**
  - **DocVQA**: questions about document images<sup>[\[3\]](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_note-docvqa2021-3)</sup>.
  - **TextVQA**: questions that require reading text within images<sup>[\[4\]](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_note-textvqa2019-4)</sup>.
  - **InfographicVQA**: questions about infographics<sup>[\[17\]](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_note-infographicvqa2021-17)</sup>.
  - **ChartQA**: questions about charts that require logical reasoning<sup>[\[18\]](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_note-chartqa2022-18)</sup>.
  - **MMDocRAG**: a benchmark for multimodal RAG for DocQA (multi-page documents, cross-modal evidence chains)<sup>[\[19\]](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_note-mmdocrag2025-19)</sup>.

## Comparative Table of Components

| Component              | Implementation Options                                           | Pros                                                                                                  | Cons / Risks                                                               | When to Choose                                                                               |
|------------------------|------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------|----------------------------------------------------------------------------------------------|
| OCR                    | Tesseract / PaddleOCR / Cloud APIs                               | Local options offer privacy and control; cloud APIs provide high out-of-the-box accuracy.             | Errors on complex layouts; APIs involve costs and compliance requirements. | For private data, use local OCR; for maximum accuracy, use a cloud service (if permissible). |
| Layout Analysis        | Rule-based / ML model (LayoutLM, LayoutParser)                   | Rules are simple for uniform templates; ML is robust to variety.                                      | Rules break on new layouts; ML requires resources/data.                    | For uniform forms, use rules; for a diverse corpus, use ML.                                  |
| Vectorization (Images) | CLIP / SigLIP / OCR-free descriptions (Donut/Pix2Struct)         | Shared latent space for *text↔image* (CLIP/SigLIP); OCR-free removes dependency on OCR.               | CLIP does not read text within images; descriptions can distort meaning.   | CLIP/SigLIP for basic multimodal search; OCR-free for low-quality scans.                     |
| Result Fusion          | Sort by score / Modality-based quotas / LLM-reranker             | A reranker significantly improves context selection accuracy.                                         | Increased latency and cost.                                                | For high-precision scenarios; simpler methods for PoCs.                                      |
| Storage/Index          | Single vector / Multi-vector (text+image) / Hybrid (BM25+vector) | Multi-vector covers different representations of one object; hybrid search helps with keywords/codes. | Increased schema and update complexity.                                    | Production systems with mixed data and strict SLAs.                                          |

Comparison of key components and approaches in MM‑RAG

## Practical Notes

- **Hybrid search** (BM25 + vector) is the de facto standard for improving recall and precision on specific terms/codes<sup>[\[20\]](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_note-weaviate-hybrid-20)</sup>.
- **Reranking** with a cross-encoder or LLM saves tokens by discarding irrelevant candidates before generation<sup>[\[13\]](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_note-cohere-rerank-13)</sup>.
- **Modern VLM retrievers** (e.g., ColPali) show advantages on visually rich documents by directly indexing page images<sup>[\[21\]](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_note-colpali2024-21)</sup>.

## See also

- [Retrieval-Augmented Generation](https://systems-analysis.info/eng/Retrieval-augmented_generation_(RAG) "Retrieval-augmented generation (RAG)")
- [Vector database](https://systems-analysis.info/eng/Vector_database "Vector database")
- [Embedding](https://systems-analysis.info/eng/Embedding_(NLP) "Embedding (NLP)")
- [GraphRAG](https://systems-analysis.info/eng/GraphRAG "GraphRAG")

## Literature

- Lewis, P. et al. (2020). *Retrieval‑Augmented Generation for Knowledge‑Intensive NLP Tasks*. NeurIPS. <a href="https://arxiv.org/abs/2005.11401" class="external text" rel="nofollow">arXiv:2005.11401</a>.
- Gao, L. et al. (2023). *Precise Zero‑Shot Dense Retrieval without Relevance Labels (HyDE)*. ACL 2023. <a href="https://arxiv.org/abs/2212.10496" class="external text" rel="nofollow">arXiv:2212.10496</a>.
- Mei, L., Mo, S., Yang, Z., Chen, C. (2025). *A Survey of Multimodal Retrieval‑Augmented Generation*. <a href="https://arxiv.org/abs/2504.08748" class="external text" rel="nofollow">arXiv:2504.08748</a>.
- Abootorabi, M.M. et al. (2025). *Ask in Any Modality: A Comprehensive Survey on Multimodal Retrieval‑Augmented Generation*. Findings of ACL 2025. <a href="https://aclanthology.org/2025.findings-acl.861/" class="external text" rel="nofollow">ACL Anthology</a>.
- Yu, S. et al. (2024). *VisRAG: Vision‑based Retrieval‑augmented Generation on Multi‑modality Documents*. <a href="https://arxiv.org/abs/2410.10594" class="external text" rel="nofollow">arXiv:2410.10594</a>.
- Cho, J. et al. (2024). *M3DocRAG: Multi‑modal Retrieval is What You Need for Multi‑document QA*. <a href="https://arxiv.org/abs/2411.04952" class="external text" rel="nofollow">arXiv:2411.04952</a>.
- Tanaka, R. et al. (2025). *VDocRAG: Retrieval‑Augmented Generation over Visually‑Rich Documents*. CVPR 2025. <a href="https://arxiv.org/abs/2504.09795" class="external text" rel="nofollow">arXiv:2504.09795</a> • <a href="https://openaccess.thecvf.com/content/CVPR2025/papers/Tanaka_VDocRAG_Retrieval-Augmented_Generation_over_Visually-Rich_Documents_CVPR_2025_paper.pdf" class="external text" rel="nofollow">CVF Open Access</a>.
- Dong, K. et al. (2025). *MMDocRAG: Benchmarking Retrieval‑Augmented Multimodal Generation for Document Question Answering*. <a href="https://arxiv.org/abs/2505.16470" class="external text" rel="nofollow">arXiv:2505.16470</a>.
- Wasserman, N. et al. (2025). *REAL‑MM‑RAG: A Real‑World Multi‑Modal Retrieval Benchmark*. <a href="https://arxiv.org/abs/2502.12342" class="external text" rel="nofollow">arXiv:2502.12342</a>.
- Faysse, M. et al. (2024). *ColPali: Efficient Document Retrieval with Vision Language Models*. <a href="https://arxiv.org/abs/2407.01449" class="external text" rel="nofollow">arXiv:2407.01449</a>.
- Radford, A. et al. (2021). *Learning Transferable Visual Models From Natural Language Supervision (CLIP)*. ICML 2021. <a href="https://arxiv.org/abs/2103.00020" class="external text" rel="nofollow">arXiv:2103.00020</a>.
- Tschannen, M. et al. (2025). *SigLIP 2: Multilingual Vision‑Language Encoders with Improved Semantic Understanding, Localization, and Dense Features*. <a href="https://arxiv.org/abs/2502.14786" class="external text" rel="nofollow">arXiv:2502.14786</a>.
- Xu, Y. et al. (2020). *LayoutLM: Pre‑training of Text and Layout for Document Image Understanding*. KDD 2020. <a href="https://dl.acm.org/doi/10.1145/3394486.3403172" class="external text" rel="nofollow">DOI</a> • <a href="https://arxiv.org/abs/1912.13318" class="external text" rel="nofollow">arXiv:1912.13318</a>.
- Huang, Y. et al. (2022). *LayoutLMv3: Pre‑training for Document AI with Unified Text and Image Masking*. <a href="https://arxiv.org/abs/2204.08387" class="external text" rel="nofollow">arXiv:2204.08387</a>.
- Zhong, X., Tang, J., Jimeno‑Yepes, A.J. (2019). *PubLayNet: Largest Dataset Ever for Document Layout Analysis*. <a href="https://arxiv.org/abs/1908.07836" class="external text" rel="nofollow">arXiv:1908.07836</a>.
- Pfitzmann, B. et al. (2022). *DocLayNet: A Large Human‑Annotated Dataset for Document‑Layout Analysis*. <a href="https://arxiv.org/abs/2206.01062" class="external text" rel="nofollow">arXiv:2206.01062</a>.
- Kim, G. et al. (2021). *OCR‑free Document Understanding Transformer (Donut)*. <a href="https://arxiv.org/abs/2111.15664" class="external text" rel="nofollow">arXiv:2111.15664</a>.
- Shen, Z. et al. (2021). *LayoutParser: A Unified Toolkit for Deep Learning Based Document Image Analysis*. <a href="https://arxiv.org/abs/2103.15348" class="external text" rel="nofollow">arXiv:2103.15348</a>.
- Singh, A. et al. (2019). *Towards VQA Models That Can Read (TextVQA)*. CVPR 2019. <a href="https://arxiv.org/abs/1904.08920" class="external text" rel="nofollow">arXiv:1904.08920</a>.
- Mathew, M. et al. (2021/2022). *DocVQA / InfographicVQA: Datasets for VQA on Document Images and Infographics*. WACV 2021 / WACV 2022. <a href="https://openaccess.thecvf.com/content/WACV2021/html/Mathew_DocVQA_A_Dataset_for_VQA_on_Document_Images_WACV_2021_paper.html" class="external text" rel="nofollow">CVF</a> • <a href="https://arxiv.org/abs/2104.12756" class="external text" rel="nofollow">arXiv:2104.12756</a>.
- Masry, A. et al. (2022). *ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning*. Findings of ACL 2022. <a href="https://aclanthology.org/2022.findings-acl.177/" class="external text" rel="nofollow">ACL</a> • <a href="https://arxiv.org/abs/2203.10244" class="external text" rel="nofollow">arXiv:2203.10244</a>.
- Liu, F. et al. (2022). *DePlot: One‑shot Visual Language Reasoning by Plot‑to‑Table Translation*. <a href="https://arxiv.org/abs/2212.10505" class="external text" rel="nofollow">arXiv:2212.10505</a>.
- Wang, P. et al. (2024). *Qwen2‑VL: Enhancing Vision‑Language Model’s Capabilities in OCR and Chart QA*. <a href="https://arxiv.org/abs/2409.12191" class="external text" rel="nofollow">arXiv:2409.12191</a>.
- Es, S. et al. (2023). *RAGAS: Automated Evaluation of Retrieval‑Augmented Generation*. <a href="https://arxiv.org/abs/2309.15217" class="external text" rel="nofollow">arXiv:2309.15217</a>.

## References

1.  <span id="cite_note-lewis2020-1">[↑](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_ref-lewis2020_1-0) Lewis, P., Perez, E., et al. (2020). *Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks*. NeurIPS. <a href="https://arxiv.org/abs/2005.11401" class="external text" rel="nofollow">arXiv:2005.11401</a>.</span>
2.  <span id="cite_note-visrag2024-2">↑ <sup>[2.0](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_ref-visrag2024_2-0)</sup> <sup>[2.1](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_ref-visrag2024_2-1)</sup> Yu, S. et al. (2024). *VisRAG: Vision-based Retrieval-Augmented Generation on Multi-modality Documents*. <a href="https://arxiv.org/abs/2407.06437" class="external text" rel="nofollow">arXiv:2407.06437</a>.</span>
3.  <span id="cite_note-docvqa2021-3">↑ <sup>[3.0](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_ref-docvqa2021_3-0)</sup> <sup>[3.1](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_ref-docvqa2021_3-1)</sup> Mathew, M. et al. (2021). *DocVQA: A Dataset for VQA on Document Images*. WACV. <a href="https://arxiv.org/abs/2007.00398" class="external text" rel="nofollow">arXiv:2007.00398</a>.</span>
4.  <span id="cite_note-textvqa2019-4">↑ <sup>[4.0](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_ref-textvqa2019_4-0)</sup> <sup>[4.1](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_ref-textvqa2019_4-1)</sup> Singh, A. et al. (2019). *TextVQA: Towards VQA Models That Can Read*. CVPR. <a href="https://arxiv.org/abs/1904.08920" class="external text" rel="nofollow">arXiv:1904.08920</a>.</span>
5.  <span id="cite_note-layoutlm2020-5">[↑](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_ref-layoutlm2020_5-0) Xu, Y. et al. (2020). *LayoutLM: Pre-training of Text and Layout for Document Image Understanding*. KDD. <a href="https://dl.acm.org/doi/10.1145/3394486.3403172" class="external text" rel="nofollow">DOI</a>.</span>
6.  <span id="cite_note-clip2021-6">↑ <sup>[6.0](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_ref-clip2021_6-0)</sup> <sup>[6.1](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_ref-clip2021_6-1)</sup> Radford, A. et al. (2021). *Learning Transferable Visual Models From Natural Language Supervision (CLIP)*. ICML. <a href="https://arxiv.org/abs/2103.00020" class="external text" rel="nofollow">arXiv:2103.00020</a>.</span>
7.  <span id="cite_note-publaynet2019-7">↑ <sup>[7.0](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_ref-publaynet2019_7-0)</sup> <sup>[7.1](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_ref-publaynet2019_7-1)</sup> Zhong, X., Tang, J., Yepes, A. J. (2019). *PubLayNet: Largest Dataset for Document Layout Analysis*. ICDAR. <a href="https://arxiv.org/abs/1908.07836" class="external text" rel="nofollow">arXiv:1908.07836</a>.</span>
8.  <span id="cite_note-doclaynet2022-8">↑ <sup>[8.0](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_ref-doclaynet2022_8-0)</sup> <sup>[8.1](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_ref-doclaynet2022_8-1)</sup> Pfitzmann, B. et al. (2022). *DocLayNet: A Large Human‑Annotated Dataset for Document‑Layout Analysis*. KDD. <a href="https://dl.acm.org/doi/10.1145/3534678.3539043" class="external text" rel="nofollow">DOI</a> / <a href="https://arxiv.org/abs/2206.01062" class="external text" rel="nofollow">arXiv:2206.01062</a>.</span>
9.  <span id="cite_note-layoutparser2021-9">[↑](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_ref-layoutparser2021_9-0) Shen, Z. et al. (2021). *LayoutParser: A Unified Toolkit for DL‑based Document Image Analysis*. <a href="https://arxiv.org/abs/2103.15348" class="external text" rel="nofollow">arXiv:2103.15348</a>.</span>
10. <span id="cite_note-donut2021-10">[↑](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_ref-donut2021_10-0) Kim, G. et al. (2021). *Donut: OCR‑free Document Understanding Transformer*. <a href="https://arxiv.org/abs/2111.15664" class="external text" rel="nofollow">arXiv:2111.15664</a>.</span>
11. <span id="cite_note-siglip2023-11">[↑](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_ref-siglip2023_11-0) Zhai, X. et al. (2023). *Sigmoid Loss for Language‑Image Pre‑Training (SigLIP)*. ICCV. <a href="https://arxiv.org/abs/2303.15343" class="external text" rel="nofollow">arXiv:2303.15343</a>.</span>
12. <span id="cite_note-milvus-mv-12">[↑](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_ref-milvus-mv_12-0) Milvus Docs. *Multi‑Vector Hybrid Search*. <a href="https://milvus.io/docs/multi-vector-search.md" class="external text" rel="nofollow">milvus.io/docs/multi-vector-search.md</a>.</span>
13. <span id="cite_note-cohere-rerank-13">↑ <sup>[13.0](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_ref-cohere-rerank_13-0)</sup> <sup>[13.1](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_ref-cohere-rerank_13-1)</sup> Cohere Docs. *Rerank API*. <a href="https://docs.cohere.com/reference/rerank" class="external text" rel="nofollow">docs.cohere.com/reference/rerank</a>.</span>
14. <span id="cite_note-gpt4v-14">[↑](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_ref-gpt4v_14-0) OpenAI. *GPT‑4V(ision) System Card*. (2023). <a href="https://cdn.openai.com/papers/GPTV_System_Card.pdf" class="external text" rel="nofollow">PDF</a>.</span>
15. <span id="cite_note-gpt4o-15">[↑](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_ref-gpt4o_15-0) OpenAI. *Hello GPT‑4o*. (2024). <a href="https://openai.com/index/hello-gpt-4o/" class="external text" rel="nofollow">openai.com/index/hello-gpt-4o/</a>.</span>
16. <span id="cite_note-ragas-16">[↑](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_ref-ragas_16-0) Es, S. et al. (2023). *RAGAS: Automated Evaluation of Retrieval Augmented Generation*. <a href="https://arxiv.org/abs/2309.15217" class="external text" rel="nofollow">arXiv:2309.15217</a>.</span>
17. <span id="cite_note-infographicvqa2021-17">[↑](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_ref-infographicvqa2021_17-0) Mathew, M. et al. (2021). *InfographicVQA: Understanding Infographics via Question Answering*. ICDAR. <a href="https://arxiv.org/abs/2104.12756" class="external text" rel="nofollow">arXiv:2104.12756</a>.</span>
18. <span id="cite_note-chartqa2022-18">[↑](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_ref-chartqa2022_18-0) Masry, A. et al. (2022). *ChartQA: A Benchmark for Question Answering about Charts*. ACL (Findings). <a href="https://arxiv.org/abs/2103.16435" class="external text" rel="nofollow">arXiv:2103.16435</a>.</span>
19. <span id="cite_note-mmdocrag2025-19">[↑](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_ref-mmdocrag2025_19-0) Dong, K. et al. (2025). *Benchmarking Retrieval‑Augmented Multimodal Generation for Document QA (MMDocRAG)*. <a href="https://arxiv.org/abs/2505.16470" class="external text" rel="nofollow">arXiv:2505.16470</a>.</span>
20. <span id="cite_note-weaviate-hybrid-20">[↑](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_ref-weaviate-hybrid_20-0) Weaviate Docs. *Hybrid search*. <a href="https://docs.weaviate.io/weaviate/concepts/search/hybrid-search" class="external text" rel="nofollow">docs.weaviate.io/.../hybrid-search</a>.</span>
21. <span id="cite_note-colpali2024-21">[↑](https://systems-analysis.info/eng/MM-RAG_(Multimodal_RAG)#cite_ref-colpali2024_21-0) Faysse, M. et al. (2024). *ColPali: Efficient Document Retrieval with Vision‑Language Models*. <a href="https://arxiv.org/abs/2407.01449" class="external text" rel="nofollow">arXiv:2407.01449</a>.</span>