Skip to content

Commit 9b53fd4

Browse files
Create multiprocess.md
1 parent 3c0d815 commit 9b53fd4

File tree

1 file changed

+19
-0
lines changed

1 file changed

+19
-0
lines changed

multiprocess.md

+19
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
2+
#### 多进程分词
3+
4+
当将以上代码示例置于文件中运行时,如涉及多进程功能,请务必使用`if __name__ == '__main__'`保护全局语句,如:
5+
mp.py文件
6+
```python3
7+
import pkuseg
8+
9+
if __name__ == '__main__':
10+
pkuseg.test('input.txt', 'output.txt', nthread=20)
11+
pkuseg.train('msr_training.utf8', 'msr_test_gold.utf8', './models', nthread=20)
12+
```
13+
运行
14+
```
15+
python3 mp.py
16+
```
17+
详见[无法使用多进程分词和训练功能,提示RuntimeError和BrokenPipeError](https://github.com/lancopku/pkuseg-python/wiki#3-无法使用多进程分词和训练功能提示runtimeerror和brokenpipeerror)
18+
19+
**在Windows平台上,请当文件足够大时再使用多进程分词功能**,详见[关于多进程速度问题](https://github.com/lancopku/pkuseg-python/wiki#9-关于多进程速度问题)

0 commit comments

Comments
 (0)