The Go Programming Language
› http://golang.org/
› Go Playground
Go Projects
› Revel Web Framework
vway

Gse v0.40.0 发布了, Go 高性能分词, 增加更多常用 API

  •  
  •   vway · Mar 13, 2019 · 3798 views
    This topic created in 2808 days ago, the information mentioned may be changed or developed.

    Go 语言高效分词, 支持英文、中文、日文等

    词典用双数组 trie ( Double-Array Trie )实现, 分词器算法为基于词频的最短路径加动态规划, DAG 和 HMM (Viterbi) 算法分词, 新增 API 基本和结巴分词保持一致.

    支持普通、搜索引擎、全模式、精确模式和 HMM 模式多种分词模式,支持用户词典、词性标注,可运行 JSON RPC 服务。

    项目地址: https://github.com/go-ego/gse

    Snake River

    Add

    • [NEW] Add more common APIs and more examples
    • [NEW] Export seg.Find() function
    • [NEW] Add AddToken and AddTokenForce func
    • [NEW] Update dictionary code, add LoadNoFreq and MinTokenFreq parameter configuration
    • [NEW] Update examples code, add more examples and documents
    • [NEW] Optimize benchmark test tools code
    • [NEW] Add seg.Find() test
    • [NEW] Add AddToken and AddTokenForce test code
    • [NEW] Add crf.go file directory

    Update

    • [NEW] Add more hmm is bool judge
    • [NEW] Update test dictionary and example code
    • [NEW] Update segmenter test code
    • [NEW] Add clear some internal api name and update godoc
    • [NEW] Add go1.12.x remove go1.10.x
    • [NEW] Add AddToken and AddTokenForce example code
    • [NEW] Add custom dictionary example to example.go
    • [NEW] Update godoc
    • [NEW] Update CHANGELOG.md

    Fix

    • [FIX] Optimize code wait todo load model
    • [FIX] Fixed some README.md benchmark
    • [FIX] Fixed link break
    • [FIX] Fixed goroutines benchmark code open path

    See Commits for more details, after Jan 23.

    2 replies  •  2019-03-19 21:32:41 +08:00
    limard
    limard
    Mar 18, 2019
    1
    star 一个
    vway
    vway
    OP
    Mar 19, 2019
    2
    @limard 欢迎贡献代码和反馈
    About   ·   Help   ·   Advertise   ·   Blog   ·   API   ·   FAQ   ·   Privacy   ·   Solana   ·   2198 Online   Highest 6679   ·     Select Language
    创意工作者们的社区
    World is powered by solitude
    VERSION: 3.9.8.5 · 29ms · UTC 14:17 · PVG 22:17 · LAX 06:17 · JFK 09:17
    ♥ Do have faith in what you're doing.