lseif@sopuli.xyz to Programmer Humor@lemmy.ml · 2 years agoIf AI can now speak Italian, it can certainly replace us...sopuli.xyzimagemessage-square9linkfedilinkarrow-up18arrow-down10
arrow-up18arrow-down1imageIf AI can now speak Italian, it can certainly replace us...sopuli.xyzlseif@sopuli.xyz to Programmer Humor@lemmy.ml · 2 years agomessage-square9linkfedilink
minus-squarestingpie@lemmy.worldlinkfedilinkarrow-up1·2 years agoCan’t find the exact source–I’m on mobile right now–but the code for the gpt-2 encoder uses a utf-8 to unicode look up table to shrink the vocab size. https://github.com/openai/gpt-2/blob/master/src/encoder.py
Can’t find the exact source–I’m on mobile right now–but the code for the gpt-2 encoder uses a utf-8 to unicode look up table to shrink the vocab size. https://github.com/openai/gpt-2/blob/master/src/encoder.py