Table 8 Results of MarianCG model on the CoNaLa dataset
From: MarianCG: a code generation transformer model inspired by machine translation
Rank | Model | BLEU score | Exact match accuracy | Year |
|---|---|---|---|---|
1 | MarianCG (Ours) | 34.43 | 10.2 | 2022 |
2 | TranX + BERT w/ mined [28] | 34.2 | 5.8 | 2022 |
3 | BERT + TAE [26] | 33.41 | - | 2021 |
4 | External Knowledge With API + Reranking [24] | 32.26 | - | 2020 |
5 | External Knowledge With API [24] | 30.69 | - | 2020 |
6 | BART W/ Mined [27] | 30.55 | - | 2021 |
7 | Reranker [21] | 30.11 | 2.8 | 2019 |
8 | BART Base [27] | 26.24 | - | 2021 |
9 | TranX [20] | 24.3 | - | 2018 |