Support worker (GPU & CPU) node registration, run gguf format models, respond to inference requests from front-end users.
Sbert：Sbert is a framwork that supports calculating similarity scores for pairs of texts and predicting scores for pairs of sentences.
Similarity score：The value range is between -1 and 1. The closer the value is to 1, the more similar the results are.
Predict score：Predict score：The value range is between -11 and 11. The closer the value is to 11, the better the result. A higher value means the inference quality is better.
Step 1：select model
prompt
please input Inference prompt word
Step 2：compare quality
On CPU
On GPU
Times
~ S
~ S
Compare Inference Quality score by Sbert
Predict score: The value range is between -11 and 11. The closer the value is to 11, the better the result. A higher value means the inference quality is better.
~
~
Step 3：compare consistency
Similarity score: The value range is between -1 and 1. The closer the value is to 1, the more similar the results are.
Your score0
Negative
strong correlation
Weak
correlation
Positive
strong correlation