Dear Research Team/Authors,
Hello! I am a learner of the MMSite model, and I sincerely appreciate your open-sourcing the code and dataset. I encountered some issues while reproducing the experiments with the align model and hope to seek your guidance.
I followed the instructions in the paper and code repository strictly:
Used the align model provided in the link as the base model
Did not modify any configuration parameters in the source code
Trained with esm_1b and PubMedBERT-abs on the 10% threshold dataset (also from the link)
However, my test result shows an F1 score of only about 0.79, significantly lower than the 0.825 reported in the paper. Notably, when I directly tested with the fuse model from the link, I achieved an F1 score of 0.823, which suggests that the functional descriptions generated by prot2text are likely correct.
I have double-checked the training process and parameter settings but still cannot identify the cause of the discrepancy. Could you please help analyze potential reasons for this difference? Are there any training details or preprocessing steps not elaborated in the paper?
Any suggestions or insights would be extremely helpful to me. Thank you again for your time and contributions!
Best regards,
尊敬的研究团队 / 作者:
您好!我是 MMSite 模型的学习者,超级感谢您开源了代码和数据集。我在复现 align 模型的实验时遇到了一些问题,希望能得到您的指导。
我严格按照论文和代码仓库的说明进行操作:
使用链接中提供的 align 模型作为基础
未修改任何源代码的 config 参数
在 10% 阈值的数据集(同样来自链接)上使用 esm_1b 和 PubMedBERT-abs 进行训练
不过,我的测试结果 F1 值仅为 0.79 左右,明显低于论文中报告的 0.825。但当我直接使用链接中的 fuse 模型进行测试时,能够复现 0.823 的 F1 值,这说明我使用 prot2text 生成的功能描述应该是正确的。
我已经反复检查了训练过程和参数设置,但依旧无法找到问题所在。请问您能否帮我分析一下可能导致这种差异的缘由?是否有一些论文中未详细说明的训练细节或预处理步骤?
任何提议或提示对我都将超级有协助,再次感谢您的时间和工作!
此致
敬礼

















暂无评论内容