CoderEval: A Benchmark of Pragmatic Code Generation with Generative Pre-trained Models

By Hao Yu, Bo Shen, Dezhi Ran, Jiaxin Zhang, Qi Zhang, Yuchi Ma, Guangtai Liang, Ying Li, Qianxiang Wang, Tao Xie