Home Sign Up Sign In
Auston
V2EX  ›  问与答

有了解 fast tokenizer 的老哥吗？怎么将 baichuan2-13b 的 tokenizer 转成 fast tokenizer?

  •   
  •   Auston · Apr 20, 2024 · 1403 views
    This topic created in 865 days ago, the information mentioned may be changed or developed.
    目前在用 vllm 做 baichuan2-13b-chat 模型的推理框架，想进一步加速推理速度，怎么将 slow tokenizer 转成 fast tokenizer 呢？
  • Fast
  • tokenizer
  • 转换
    1 replies
    Auston
        1
    Auston  
    OP
       Apr 23, 2024
    大佬们，能给个思路吗
    About   ·   Help   ·   Advertise   ·   Blog   ·   API   ·   FAQ   ·   Privacy   ·   Solana   ·   1138 Online   Highest 6679   ·     Select Language
    创意工作者们的社区
    World is powered by solitude
    VERSION: 3.9.8.5 · 21ms · UTC 23:08 · PVG 07:08 · LAX 16:08 · JFK 19:08
    ♥ Do have faith in what you're doing.