Python extension for libmemcache, the C API to memcached. cmemcache API is very similar to python-memcache, except for some return codes which follow libmemcache conventions.
cmemcache is about 2 times faster then python-memcache with short key names (8 characters), faster with larger key names.
This extension was created after doing some timings on a simple session caching scheme where I noticed that python-memcache was 'only' three times as fast as PostgreSQL. I expected it to be faster than that, since PostgreSQL does a lot more than memcached. It was suggested that it was perhaps python-memcache, which for instance gets the initial part of the message from memcached byte for byte. After discovering libmemcache it seemed pretty straight forward to create an extension on top of that.
cmemcache has 2 clients: StringClient and Client. StringClient is the extension that only supports python strings for values. The api was copied from python-memcache, except that it excepts strings only. Client is a module that implements caching of arbitrary python objects using Pickle on top of the StringClient. This code is a copy/paste from python-memcache. Most of the test code in test.py is run on the cmemcache Client and the python-memcache Client to make sure that they are interchangeable. Although I have not tested this but it should be possible to mix cmemcache and python-memcache clients in a running system as well, since they use the same constants for encoding object types.
The speed difference would be less if the memcached protocol would be changed to precede the reply header with its size. A client could then read the header size, read the full header (in one read), parse header to get the data size, and read data (this is already done in one read).
cmemcache:
libmemcache:
set_servers with the wrong port number causes a segfault (in libmemcache). See commented out testing code in test.py.
mc_err_filter_add() broken. This results in warnings on add() and replace().
This is fixed by applying the patch from the download section.
libmemcache-1.4.0.rc2 is not compatible with memcached 1.2.1, this results in get_stats returning no stats.
start memcached, create Client, kill memcached, start memcached, do a get() and python process exits, with messages like:
[ERROR@1170236923.979369] mcm_buf_read():361: read(2) failed: Operation now in progress: server unexpectedly closed connection
libmemcache does an exit on severe errors. It is possible to install an error handler that could change the severity to not exit and then the python extension could throw a python exception. However, I do not know if libmemcache will recover correctly.
I do not have time at the moment to try and fix this.
Reported by Mark and Philip, 31/01/2007.
Versions:
Copyright (C) 2006-2008 Gijsbert de Haan. This code is distributed under the GNU General Public License.
Gijsbert de Haan <gijsbert.de.haan@gmail.com>