pychbase 0.1.8

Pychbase

这是一个python c包装器，用于maprdb和hbase，使用libhbase C API。

pychbase是根据happybase api建模的，但它不使用 thrift，是maprdb的理想选择。

pychbase在python 2.7和mapr 5.1上测试。

ld_library_路径

要编译和导入pychbase，您的LD_LIBRARY_PATH必须 将带有libjvm.so的目录放在上面，通常放在：

• $java_home/lib/amd64/服务器
• $java home/jre/lib/amd64/服务器

如果您在mapr中使用这个，那么还必须有/opt/mapr/libon 您的LD_LIBRARY_PATH

例如，

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$JAVA_HOME/lib/amd64/server:/opt/mapr/lib

安装

export PYCHBASE_LIBJVM_DIR=/usr/lib/jvm/jre-1.7.0/lib/amd64/server
virtualenv pychbase
cd pychbase
source bin/activate
pip install pychbase

# Or build it from source
git clone https://github.com/mkmoisen/pychbase.git
cd pychbase
python setup.py install

运行测试

tests目录中的config.py文件有两个常量， ZOOKEEPER和TABLE_NAME，如果运行 未经修改的测试

创建一个tests/local_config.py文件，如下所示：

ZOOKEEPERS = 'localhost:7222'
TABLE_NAME = 'testpychbase'

要运行测试，请确保位于tests目录中，否则 您将面临导入问题：

cd tests
python tests.py

当前nosetests在不遇到导入问题的情况下无法工作。

用法

我试图尽可能地模仿伟大的happybase api 可能的。

确保设置ld_library_path环境变量：

# MapR
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$JAVA_HOME/jre/lib/amd64/server:/opt/mapr/lib

# Non MapR
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib/jvm/jre-1.7.0/lib/amd64/server::/home/matthew/libhbase/target/libhbase-1.0-SNAPSHOT/lib/native
export HBASE_LIB_DIR=/home/matthew/libhbase/target/libhbase-1.0-SNAPSHOT/lib/
# I've only gotten it to work on CDH4. If you are on CDH5 you'll need to mess around with the classpath some more

导入：

from pychbase import Connection, Table, Batch

创建连接：

# On MapR, you don't need to specify the CLDBS/zookeepers:
connection = Connection()

# On Non-MapR environments, you'll need to specify the zookeepers:
connection = Connection('zookeeper-one:2181",zookeeper-two:2181",zookeeper-three:2181"')

创建和删除表格：

# Create a table named 'test_table' with a single column family named 'f' with no additional attributes
connection.create_table('test_table', {'f': {}})
connection.delete_table('test_table')

要获得手术台：

table = connection.table('test_table')

放置、删除和从表中获取：

# Put a row with the current timestamp
table.put('rowkey1', {'f:foo': 'bar', 'f:hai': 'bai'})
data = table.row('rowkey1')
assert data == {'f:foo': 'bar', 'f:hai': 'bai'}

# Get a row and only include a single column
data = table.row('rowkey1', columns=('f:foo',))
assert data == {'f:foo': 'bar'}

# Delete the row
table.delete('rowkey1')
data = table.row('rowkey1')
assert data == {}

# Put a row with a given timestamp
table.put('rowkey1', {'f:foo': 'bar'}, timestamp=1000)
table.put('rowkey1', {'f:foo': 'BAR'}, timestamp=10000)

# Get a row with a given timestamp and include its timestamp
data = table.row('rowkey1', timestamp=1000, include_timestamp=True)
assert data == {'f:foo': ('bar', 1000)}

扫描：

# Full table scan:
for row, data in table.scan():
    pass

# Scan with a start and stop:
for row, data in table.scan('foo', 'bar'):
    pass

# Scan with a row prefix:
for row, data in table.scan(row_prefix='baz'): # E.g., start='baz', stop='baz~'
    pass

# Scan with a filter: # Check out tests.py on how to use all of the filters
for row, data in table.scan(filter="SingleColumnValueFilter('f', 'foo', =, 'binary:foo')":
    pass

# Scan a table but return only row keys, no rows:
table.put('foo', {'f:foo': 'foo'}
table.put('foo1', {'f:foo': 'foo'}
table.put('foo2', {'f:foo': 'foo'}

rows = list(table.scan(only_rowkeys=True))
assert rows == ['foo', 'foo1', 'foo2']

要计算表中的行数：

# Full table count:
count = table.count()

# Count all rows with a start and stop
count = table.count('foo', 'bar')

# Count all rows whose row key starts with a row prefix:
count = table.count(row_prefix='baz') # E.g., start='baz', stop='baz~'

# Count all rows with a filter:
count = table.count(filter="SingleColumnValueFilter('f', 'foo', =, 'binary:foo')")

批量放置：

batch = table.batch()
datas = [
    ('foo', 'a', 'b'),
    ('foo1', 'a1', 'b1'),
    ('foo2', 'a2', 'b2'),
]

for data in datas:
    batch.put(data[0], {'f:foo': data[1], 'f:bar': data[2]})

errors = batch.send()

assert errors == 0

批量删除：

batch = table.batch()
rows = ['foo', 'foo1', 'foo2']
for row in rows:
    batch.delete(row)

errors = batch.send()

assert errors == 0

注意，batch.send()返回发生的错误数， 如果有的话。由客户机来忽略此问题或引发异常。

batch = table.batch()
batch.put('foo', {'f:foo', 'bar'})
batch.put('foo', 'invalid')

errors = batch.send()
assert errors == 1

另一个helper方法是table.delete_prefix(row_prefix)， 删除所有以前缀开头的行。

table.put('foo', {'f:foo', 'foo'}
table.put('foo1', {'f:foo', 'foo'}
table.put('foo2', {'f:foo', 'foo'}
table.put('foo3', {'f:foo', 'foo'}
table.put('bar', {'f:bar', 'bar'}

number_deleted = table.delete_prefix('foo')
assert number_deleted == 3

assert table.count() == 2

注意，尝试批处理/放置未转义的空终止符将导致 在他们被剥夺。尝试将行键与未转义的 空终止符将引发TypeException。用户有责任 在尝试批处理/放置数据之前转义空终止符。

table.put('foo', {'f:foo\0bar': 'baz\0bak'})
data = table.row('foo')
assert data == {'f:foo': 'baz'}

table.put('bar', {'f:foo\\0bar': 'baz\00bak'})
data = table.row('foo')
assert data == {'f:foo\\0bar': 'baz\00bak'}

HappyBase兼容性

这个库的一个目标是保持与 很高兴。

查看__init__.py了解happybase i的哪些特性 尚未实施。

在将来，如果用户 使用未实现的功能。

非MAPR安装和环境变量指南

我没有在cloudera上测试过pychbase。我拿不到 由于libhbase的类路径问题，正在处理cdh5，并且 当我能用CDH4启动和运行时，一些测试 正在失败。在我看来，libhbase不完全兼容 在mapr外面。

export PYCHBASE_IS_MAPR=FALSE
export PYCHBASE_LIBJVM_DIR=/usr/lib/jvm/jre-1.7.0/lib/amd64/server
export PYCHBASE_INCLUDE_DIR=/home/matthew/libhbase/target/libhbase-1.0-SNAPSHOT/include
export PYCHBASE_LIBRARY_DIR=/home/matthew/libhbase/target/libhbase-1.0-SNAPSHOT/lib/native
virtualenv pychbase
cd pychbase
source bin/activate
pip install pychbase

# Or build it from source
git clone https://github.com/mkmoisen/pychbase.git
cd pychbase
python setup.py install

请注意，必须在 正确安装pychbase的顺序：

• Pychbase_是映射器
• pychbase库libjvm目录
• Pychbase_include_dir
• pychbase库目录

export PYCHBASE_IS_MAPR=FALSE

export PYCHBASE_LIBJVM_DIR=/usr/lib/jvm/jre-1.7.0/lib/amd64/server

export PYCHBASE_INCLUDE_DIR=/home/matthew/libhbase/target/libhbase-1.0-SNAPSHOT/include

export PYCHBASE_LIBRARY_DIR=/home/matthew/libhbase/target/libhbase-1.0-SNAPSHOT/lib/native

作者

Matthew Moisen

许可证

麻省理工学院