Skip to content

Issues: open-compass/VLMEvalKit

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

30 Open 152 Closed
30 Open 152 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Is A-OKVQA support discontinued?
#518 opened Oct 12, 2024 by MOSHIIUR
Are images compressed in tsv files?
#517 opened Oct 12, 2024 by ywwynm
Support for Falcon2 VLM
#515 opened Oct 11, 2024 by Venkatesh-Balavadani
Integration of MuirBench
#514 opened Oct 11, 2024 by nicokossmann
评测LLaVa系列模型时有bug
#510 opened Oct 10, 2024 by KingMV
2
无法评测llava1.5-7b
#505 opened Oct 8, 2024 by itsqyh
2
DocVQA_TEST和InfoVQA_TEST无法评测
#504 opened Oct 8, 2024 by helloworld01001
1
在MCQ任务中，一道题目具有多张图片，我应该如何构建框架需要的tsv数据集？
#499 opened Oct 4, 2024 by Nefefilibata
1
textonly benchmarks
#497 opened Sep 30, 2024 by JcWang20
@PhoenixZ810
1
[Help Wanted] the alignment with official accuracy in llama3.2-vision help wanted Extra attention is needed
#493 opened Sep 29, 2024 by droidXrobot
7
Reproducing QWen2VL Results on Video Benchmarks with VLMEvalKit
#484 opened Sep 23, 2024 by aniki-ly
@FangXinyu-0913
2
【提问】关于语音模型的评测 help wanted Extra attention is needed
#473 opened Sep 19, 2024 by xx0412
1
Input length of input_ids is 0, but max_length is set to -2009.
#455 opened Sep 10, 2024 by wangli68
ChatAPI - ERROR - HTTPSConnectionPool(host='api.openai.com', port=443): Max retries exceeded with url: /v1/chat/completions (Caused by ProxyError('Unable to connect to proxy', NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f481233f640>: Failed to establish a new connection: [Errno 111] Connection refused')))
#406 opened Aug 25, 2024 by yansuoyuli
3
CMMMU数据集支持 help wanted Extra attention is needed
#398 opened Aug 21, 2024 by DietDietDiet
3
关于llava-v1.5-7b复现
#391 opened Aug 19, 2024 by helloworld01001
11
关于幻觉benchmark的支持 help wanted Extra attention is needed
#384 opened Aug 15, 2024 by shufangxun
1
7B模型在4*A100 80GB上发生OOM
#382 opened Aug 13, 2024 by Rocky77JHxu
4
KeyError: 'answer'
#378 opened Aug 11, 2024 by lucasjinreal
19
Inconsistent max_new_tokens
#356 opened Aug 3, 2024 by amitbcp
2
[Help Wanted] Supporting the chat_inner API for existing VLMs. help wanted Extra attention is needed
#323 opened Jul 27, 2024 by kennymckormick
2
Audio modality evaluation help wanted Extra attention is needed
#313 opened Jul 25, 2024 by Simplesss
1
[Request] Consider integrating the following list? help wanted Extra attention is needed
#257 opened Jul 5, 2024 by sarapieri
2
grounding数据支持
#219 opened Jun 12, 2024 by sxlyiyiyi
3
best to have docker image help wanted Extra attention is needed
#190 opened May 3, 2024 by max-yue
2
ProTip! Find all open issues with in progress development work with linked:pr.