| Age | Commit message (Expand) | Author |
|---|---|---|
| 2024-07-27 | Merge mainline llama.cpp (#3) | Kawrakow |
| 2024-06-21 | llama : optimize long word tokenization with WPM (#8034) | Georgi Gerganov |
| 2024-06-18 | tokenizer : BPE fixes (#7530) | jaime-m-p |
| 2024-06-16 | unicode : avoid char32_t (#7957) | Georgi Gerganov |
| 2024-05-18 | Unicode codepoint flags for custom regexs (#7245) | jaime-m-p |
| 2024-05-09 | llama3 custom regex split (#6965) | jaime-m-p |
| 2024-05-04 | tests : add test-tokenizer-0.sh + fix some tokenizers (#7036) | Georgi Gerganov |
| 2024-04-29 | llama : fix BPE pre-tokenization (#6920) | Georgi Gerganov |
| 2024-03-26 | wpm : portable unicode tolower (#6305) | Jared Van Bortel |
| 2024-03-11 | llama : refactor unicode stuff (#5992) | Georgi Gerganov |
 |
index : ik_llama.cpp.git | |
| Unnamed repository; edit this file 'description' to name the repository. |
| summaryrefslogtreecommitdiff |
path: root/unicode.cpp