pymmseg-cpp
pymmseg-cpp is a Python port of the rmmseg-cpp project. rmmseg-cpp is a MMSEG Chinese word segmenting algorithm implemented in C++ with a Ruby interface.
Download the binary release on the right sidebar and copy the pymmseg directory to your Python's path (e.g. /usr/lib/python2.5/site-packages/). Here's an example of usage:
from pymmseg import mmseg
mmseg.dict_load_defaults()
text = # ...
algor = mmseg.Algorithm(text)
for tok in algor:
print '%s [%d..%d]' % (tok.text, tok.start, tok.end)
Or you can download the source tarball or check out the latest code from the git repo hosted at github. Then you'll need to build the mmseg-cpp module yourself: goto the mmseg-cpp subdirectory and run the build.py script. It will build the native module for you.
For more information, refer to the README file.
很多同學(xué)都會(huì)出現(xiàn)亂碼的問題。可能是mmseg支持的是utf8，
windows的本地默認(rèn)編碼是cp936，也就是gbk編碼，所以在控制臺(tái)直接打印utf-8的字符串當(dāng)然是亂碼了。 解決方法：
在控制臺(tái)打印的地方用一個(gè)轉(zhuǎn)碼就ok了，打印的時(shí)候這么寫：
print myname.decode('UTF-8').encode('GBK')
from pymmseg import mmseg
mmseg.dict_load_defaults()
text = # ...
algor = mmseg.Algorithm(text)
for tok in algor:
print '%s [%d..%d]' % (tok.text.decode('UTF-8').encode('GBK') , tok.start, tok.end)
posted on 2011-05-03 13:27
漂漂 閱讀(1159)
評(píng)論(0) 編輯 收藏 引用