From beea6e1b16e783a0886e78dec01002a8c00db24d Mon Sep 17 00:00:00 2001 From: Jan Boon Date: Mon, 8 Apr 2024 20:43:30 +0800 Subject: llama : save and restore kv cache for single seq id (#6341) * llama : save and restore kv cache for single seq id * remove trailing whitespace * respond error in case there's no space in the kv cache * add kv seq save restore to test case * add --slot-save-path arg to enable save restore and restrict save location * Returning 0 for some cases, instead of asserting. * cleanup error cases * rename sequence state functions * rename state get set functions * add previous function names back in with DEPRECATED notice * update doc * adjust endpoints to preferred style * fix restoring zero cell count * handle seq rm return value * unused param * keep in the size check * fix return types * add server test case for slot save restore * cleanup * add cake * cleanup style * add special * removing a whole sequence never fails * move sequence state file functionality from server to llama to match session api and add version tags * catch exceptions on save as well * error log messages * check types for stricter restore * update server doc * readme : update API changes date * strict filename validation * move include, reject bom as well * also reject empty filename * reject whitespace and trailing dot --------- Co-authored-by: Martin Evans Co-authored-by: Georgi Gerganov --- common/common.h | 2 ++ 1 file changed, 2 insertions(+) (limited to 'common/common.h') diff --git a/common/common.h b/common/common.h index 99ee90bc..4635e05d 100644 --- a/common/common.h +++ b/common/common.h @@ -179,6 +179,8 @@ std::string gpt_random_prompt(std::mt19937 & rng); void process_escapes(std::string& input); +bool validate_file_name(const std::string & filename); + // // String utils // -- cgit v1.2.3