因为工程需要,所以引入一个图数据库(graph database),摸了一圈,找到了titan,据说很牛B的样子,值得尝试一下。
titan不自带存储，所以使用cassandra作为key-value持久化库,引入elasticsearch做搜索.因为打算远程连接和操作,所以引入rexster做服务端。这样一个基于titan的图数据库服务就搞定了. 客户端使用python,于是找了个bulbs作为客户端。
官方文档不多,bulbs还大多都是neo4j的,但各种看看基本总算能hold住。
from bulbs.titan import Graph,Config,TITAN_URI config = Config( TITAN_URI , "your username" , "your password") g = Graph( config )
这样就能得到g,其中TITAN_URI是本地的URI,print下可以发现这貌似就是个string
>>> print TITAN_URI http://localhost:8182/graphs/graph
若是连接远程服务，其实直接自己拼URI就可以。
照着做了几下，没啥问题，但很快，问题出现了... 本来，在得到前文所述的g后,我们应该可以随意使用其IO,比如如下
>>> g.V [<Vertex: http://localhost:8182/graphs/graph/vertices/160004>, <Vertex: http://localhost:8182/graphs/graph/vertices/160016>, <Vertex: http://localhost:8182/graphs/graph/vertices/160012>, <Vertex: http://localhost:8182/graphs/graph/vertices/160008>]
在创建了几个点后,我们数据g.V,我们应该得到一个vertices的list,而实际上,我们也可以得到这个值 但奇怪的是,这个结果开始可以用，但过了一段时间后，我们再输入g.V，却不再返回结果，而是被挂在了waiting response的路上,若我们这时候control+ c,可以得到如下结果
POST url: http://localhost:8182/graphs/graph/tp/gremlin
POST body: {"params": null, "script": "g.getVertices()"}
POST url: http://localhost:8182/graphs/graph/tp/gremlin
POST body: {"params": null, "script": "g.getVertices()"}
^CTraceback (most recent call last):
File "lib/ttclient/ttclient.py", line 145, in <module>
t.show_all();
File "lib/ttclient/ttclient.py", line 106, in show_all
vl = t.get_vertex_list()
File "lib/ttclient/ttclient.py", line 46, in get_vertex_list
return list(self.g.V)
File "/usr/local/lib/python2.7/site-packages/bulbs/base/graph.py", line 69, in V
resp = self.client.get_all_vertices()
File "/usr/local/lib/python2.7/site-packages/bulbs/rexster/client.py", line 399, in get_all_vertices
return self.gremlin(script, params)
File "/usr/local/lib/python2.7/site-packages/bulbs/rexster/client.py", line 356, in gremlin
return self.request.post(gremlin_path, params)
File "/usr/local/lib/python2.7/site-packages/bulbs/rest.py", line 131, in post
return self.request(POST, path, params)
File "/usr/local/lib/python2.7/site-packages/bulbs/rest.py", line 184, in request
http_resp = self.http.request(uri, method, body, headers)
File "/usr/local/lib/python2.7/site-packages/httplib2/__init__.py", line 1593, in request
(response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
File "/usr/local/lib/python2.7/site-packages/httplib2/__init__.py", line 1335, in _request
(response, content) = self._conn_request(conn, request_uri, method, body, headers)
File "/usr/local/lib/python2.7/site-packages/httplib2/__init__.py", line 1291, in _conn_request
response = conn.getresponse()
File "/usr/local/Cellar/python/2.7.7_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 1067, in getresponse
response.begin()
File "/usr/local/Cellar/python/2.7.7_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 409, in begin
version, status, reason = self._read_status()
File "/usr/local/Cellar/python/2.7.7_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 365, in _read_status
line = self.fp.readline(_MAXLINE + 1)
File "/usr/local/Cellar/python/2.7.7_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 476, in readline
data = self._sock.recv(self._rbufsize)
KeyboardInterrupt
从被interrupt的位置看,它是被卡在了socket recv的路上了,如果重启rexster service,我们就能正常使用一段时间,然后动不动就卡住不动了,这还怎么一起玩耍啊我摔...
google各种翻,果然有所收获 -- 这几乎是必然的,你遇到问题的话，一般总有人先于你遇到，你需要的就是自己找找前人的讨论。
很快找到这个issue: 作者描述道:
Hi, @xedin -- seems identical to the Rexster-Console issue. Do this curl http://localhost:8182/graph/tinkergraph/vertices/1 ...do that 3 times. Freezes on Mac OSX Java 7 and RedHat Java 1.6.0 This happens with Extensions and native REST API.
有人立刻提出了一个解决方法: iptables 的问题。当然妥妥的不是，我在localhost debug都会出问题...iptables早设置为不阻挡local的
Note that:
curl localhost:8182/graphs/tinkergraph/vertices/[1-100]
works! But if you do this:
curl localhost:8182/graphs/tinkergraph/vertices/1
curl localhost:8182/graphs/tinkergraph/vertices/2
curl localhost:8182/graphs/tinkergraph/vertices/3
curl localhost:8182/graphs/tinkergraph/vertices/4
LOCKED UP!
Marko.
然后一个staff提出了个commit 内容很简单:
- <io-strategy>worker</io-strategy> + <io-strategy>leader-follower</io-strategy>
但我检查了下我的config,早就被fix成leader-follower了.... 然后下面各种尝试debug各种调exception，但是貌似不幸的是,貌似还是没啥好的解决,只有各种同问题者的补充.
stephen mallette直接强制关掉了这个issue,但可惜回复并不能禁止，依然有人发回复.. 他愤怒的说道
I hate to play a blame game with this issue, but the fact is that Rexster just uses the infrastructure supplied by Grizzly so it is quite likely that the problem is in there. When I think about this issue in your particular case, I'm not sure it's a "problem" exactly as the solution is to simply increase the thread-pool sizes...so perhaps that's just mis-configuration from Grizzly's perspective. I think this issue is a "bug" when even changing the thread-pool sizes/strategy does nothing to change the issue. This is the case on at least one OS (can't recall which one...CentOS maybe). In TinkerPop3 we will be leaving Grizzly behind in favor of Netty. I'm already happier for that change. The Netty team has been very helpful, their development process is very open and I personally find the code easier to follow when I need to dig into it.
翻译就是: 你们说的问题都不是我们的，我们没有错，另外，前面那些破fix都是没用的，别烦我，滚！
看时间已经好多个月了,看来貌似还是没解决? 不过,这位说的"does nothing to change the issue"的方法:
<max-post-size>2097152</max-post-size>
<max-header-size>8192</max-header-size>
<upload-timeout-millis>30000</upload-timeout-millis>
<thread-pool>
<worker>
<core-size>32</core-size>
<max-size>32</max-size>
</worker>
<kernal>
<core-size>16</core-size>
<max-size>16</max-size>
</kernal>
</thread-pool>
<io-strategy>leader-follower</io-strategy>
大致说线程池worker两倍于kernal,并解释说
Grizzly's http keep-alive use the socket 30 seconds, chrome/firfox will open conncurrent sockets,if concurrent sockets GT grizzly thread pool size,then grizzly hung up. This bug maybe when client close the socket, the server socket send buffer NOT empty, then need APP close socket, grizzly LEAK this. much CLOSE_WAIT sockets remian in grizzly server. Just set BIGGER TP size before replace grizzly.
不明觉厉,但的确有人说"it works",试着用这方法fix了下,然后后来貌似一直不出错误了，当然，也可能只是压力太小了囧
后注，目前为止，感觉还是 dgraph 做得比较出彩，各位不妨试试。
2 Comments
gavin · May 22, 2015 at 10:31
求助TITAN集群安装 不胜感激
yu · May 22, 2015 at 20:28
@gavin 你好,后来我们放弃治疗了..
当时版本应该是0.4.4
我当时最后在conf下配出rexster-cassandra-es.xml和titan-cassandra-es.properties两个文件,然后在titan目录下启动如下:
相关的几个文件参考此处: https://gist.github.com/yuikns/8b7fa17fc17cb6990da6
后来我好像并没有接触这个系统,也不知道是否能在现在的环境下继续用,真是抱歉.
——
为了能让rexster正常使用,rexster-cassandra-es.xml中目录为: rexster/http/base-uri 的值可能需要修改下.具体是,若为rexster为本机,则不需要任何修改,若为远程的某个服务器,则需要将其修改为对应ip.
若启动成功,你应该可以访问 http://server_ip:8182 访问到它提供的一个web界面.