Skip to content

Commit 430638d

Browse files
committed
update cn readme
1 parent fa23801 commit 430638d

File tree

4 files changed

+74
-4
lines changed

4 files changed

+74
-4
lines changed

LICENSE

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
The MIT License (MIT)
22

3-
Copyright (c) 2015
3+
Copyright (c) by Windfarer
44

55
Permission is hereby granted, free of charge, to any person obtaining a copy
66
of this software and associated documentation files (the "Software"), to deal

README.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,8 @@
11
# py-mysql-elasticsearch-sync
22
Simple and fast MySQL to Elasticsearch sync tool, written in Python.
33

4+
[中文文档](https://github.com/zhongbiaodev/py-mysql-elasticsearch-sync/blob/master/README_CN.md)
5+
46
## Introduction
57
This tool helps you to initialize MySQL dump table to Elasticsearch by parsing mysqldump, then incremental sync MySQL table to Elasticsearch by processing MySQL Binlog.
68
Also, during the binlog syncing, this tool will save the binlog sync position, so that it is easy to recover after this tool being shutdown for any reason.
@@ -9,7 +11,7 @@ Also, during the binlog syncing, this tool will save the binlog sync position, s
911
By following these steps.
1012

1113
##### 1. ibxml2 and libxslt
12-
Also, this tool depends on python lxml package, so that you should install the lxml's dependecies correctly, the libxml2 and libxslt are required.
14+
This tool depends on python lxml package, so that you should install the lxml's dependecies correctly, the libxml2 and libxslt are required.
1315

1416
For example, in CentOS:
1517

@@ -25,7 +27,7 @@ sudo apt-get install libxml2-dev libxslt-dev python-dev
2527

2628
See [lxml Installation](http://lxml.de/installation.html) for more infomation.
2729
##### 2. mysqldump
28-
And then, mysqldump is required.(and enable binlog)
30+
And then, mysqldump is required in the machine where this tool will be run on it.(and the mysql server must enable binlog)
2931

3032

3133
##### 3. this tool
@@ -58,7 +60,7 @@ es-sync path/to/your/config.yaml --fromfile
5860
to start sync, when xml sync is over, it will also start binlog sync.
5961

6062
## Deployment
61-
We provide an upstart script to help you deploy this tool,since we use virtualenv for requirements isolation, you must edit it for your own condition, besides, you can deploy it in your own way.
63+
We provide an upstart script to help you deploy this tool, you can edit it for your own condition, besides, you can deploy it in your own way.
6264

6365
## TODO
6466
- [ ] MultiIndex Supporting

README_CN.md

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
# py-mysql-elasticsearch-sync
2+
一个从MySQL向Elasticsearch同步数据的工具,使用Python实现。
3+
4+
## 简介
5+
在第一次初始化数据时,本工具解析mysqldump导出的数据,并导入ES中,在后续增量更新中,解析binlog的数据,对ES中的数据进行同步。在binlog同步阶段,支持断点恢复,因此无需担心意外中断的问题。
6+
7+
## 安装
8+
9+
##### 1. ibxml2 和 libxslt
10+
本工具基于lxml库,因此需要安装它的依赖的libxml2和libxslt
11+
12+
在CentOS中:
13+
14+
```
15+
sudo yum install libxml2 libxml2-devel libxslt libxslt-devel
16+
```
17+
18+
在Debian/Ubuntu中:
19+
20+
```
21+
sudo apt-get install libxml2-dev libxslt-dev python-dev
22+
```
23+
24+
查看[lxml Installation](http://lxml.de/installation.html)来获取更多相关信息
25+
26+
##### 2. mysqldump
27+
在运行本工具的机器上需要有mysqldump,并且mysql服务器需要开启binlog功能。
28+
29+
30+
##### 3. 本工具
31+
安装本工具
32+
33+
```
34+
pip install py-mysql-elasticsearch-sync
35+
```
36+
37+
## 配置
38+
你可以通过修改[配置文件示例](https://github.com/zhongbiaodev/py-mysql-elasticsearch-sync/blob/master/src/sample.yaml)来编写自己的配置文件
39+
40+
## 运行
41+
运行命令
42+
43+
```
44+
es-sync path/to/your/config.yaml
45+
```
46+
工具将开始执行mysqldump并解析流进行同步,当dump结束后,将启动binlog同步
47+
48+
最近一次binlog同步位置记录在一个文件中,这个文件的路径在config文件中配置过。
49+
50+
你可以删除记录文件来从头进行binlog同步,或者修改文件里的内容,来从特定位置开始同步。
51+
52+
53+
你也可以把自己从mysql导出的xml文件同步进ES中(在mysqldump的命令中加上参数```-X```即可导出xml)
54+
55+
然后执行
56+
57+
```
58+
es-sync path/to/your/config.yaml --fromfile
59+
```
60+
启动从xml导入,当从xml导入完毕后,它会开始同步binlog
61+
62+
## 服务管理
63+
我们写了一个[upstart脚本]来管理本工具的运行,你也可以用你自己的方式进行部署运行
64+
65+
## TODO
66+
- [ ] 多索引支持
67+
- [ ] 多表支持

src/__init__.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,7 @@ def encode_in_py2(s):
3636
DEFAULT_BULKSIZE = 100
3737
DEFAULT_BINLOG_BULKSIZE = 1
3838

39+
3940
class ElasticSync(object):
4041
table_structure = {}
4142
log_file = None

0 commit comments

Comments
 (0)