Table 6 Results of the first experiment on CoNaLa
From: MarianCG: a code generation transformer model inspired by machine translation
Model | BLEU score | Exact match accuracy | Year |
|---|---|---|---|
TranX + BERT w/ mined [28] | 34.2 | 5.8 | 2022 |
BERT + TAE [26] | 33.41 | - | 2021 |
External Knowledge With API + Reranking [24] | 32.26 | - | 2020 |
MarianCG (Ours) | 30.92 | 6.2 | 2022 |
External Knowledge With API [24] | 30.69 | - | 2020 |
BART W/ Mined [27] | 30.55 | - | 2021 |
Reranker [21] | 30.11 | 2.8 | 2019 |
BART Base [27] | 26.24 | - | 2021 |
TranX [20] | 24.3 | - | 2018 |