一个模拟amazon cloudsearch进行本地测试的rest服务。
nozama-cloudsearch的Python项目详细描述
nozama-cloudsearch
内容
用于本地测试的amazon云搜索服务的轻量级实现 目的。这是用于服务的功能/验收测试 使用云搜索。
nozama服务还实现了自己的rest api,允许您在 你通常无法在Amazon CloudSearch上找到数据。
使用nozama的一个方便的好处是它提供了一种从amazon迁移的方法。 CloudSearch到ElasticSearch。
“官方”码头集装箱:
GitHub存储库:
Why?
我想测试一个硬编码的平台,它只使用cloudsearch。在那里 我不可能改变这个密码。我也找不到其他的 由于预算限制而导致的实例。我四处寻找替代品,发现 没有一个我能在centos工作。
Quickstart
Docker
如果您的系统上已经有Docker和Docker组合,则可以执行以下操作:
# download the docker compose configuration: curl -O https://raw.githubusercontent.com/oisinmulvihill/nozama-cloudsearch/master/nozama-cloudsearch.yaml # Run in the background: docker-compose -f nozama-cloudsearch.yaml up -d # Check everything is up and running docker-compose -f nozama-cloudsearch.yaml ps curl http://localhost:15808/ping {"status": "ok", "name": "nozama-cloudsearch", "version": "2.0.2"}# To shutdown and stop all parts: docker-compose -f nozama-cloudsearch.yaml down
这将下载它所依赖的mongo和elasticsearch容器。下载完所有服务后,就可以开始使用rest api了。
Pypi.org
Mongo和ElasticSearch需要安装并在系统上运行。默认设置将使用名为“nozama cloudsearch”的数据库。有关详细信息,请参阅development.ini配置文件。
# create a quick environment to install into: mkvirtualenv -p python 3 nozama # activate en workon nozama # Install from pypi: pip install nozama-cloudsearch # download the development configuration: curl -O https://raw.githubusercontent.com/oisinmulvihill/nozama-cloudsearch/master/development.ini # Run the service: pserve development.ini Starting server in PID 6845. serving on 0.0.0.0:15808 view at http://127.0.0.1:15808
成功!
Quick API Usage Example
如果您有一个正在运行的服务,可以使用 命令行。
# Assumes: serving on 0.0.0.0:15808 view at http://127.0.0.1:15808 # A quick check of the version and that the service is running: curl http://localhost:15808/ping {"status": "ok", "name": "nozama-cloudsearch", "version": "2.0.2"}# Now check what documents are present / removed: curl http://localhost:15808/dev/documents {"documents_removed": [], "documents": []}# Add a document using the batch upload SDF: curl -X POST -H "Content-Type: application/json" http://localhost:15808/2013-08-22/documents/batch -d '[{"lang": "en", "fields": {"name": "bob"}, "version": 1376497963, "type": "add", "id": 1246}]'{"status": "ok", "adds": 1, "deletes": 0, "error": "", "warning": ""}# Check the document is there: curl http://localhost:15808/dev/documents {"documents": [{"_id": "1246", "lang": "en", "fields": {"name": "bob"}, "version": "1376497963", "id": "1246"}], "documents_removed": []}# Try searching for the document: curl http://localhost:15808/2013-08-22/search?q=bob {"rank": "-text_relevance", "match-expr": "(label 'bob')", "hits": {"found": 1, "start": 0, "hit": [{"id": "1246", "fields": {"name": "bob"}}]}, "info": {"rid": "47e87151546d5a349d7bf9b60eee0ebdf74783422a2e08cad0b9348e3ee3ef04eb198715bbe4e353", "time-ms": 5, "cpu-time-ms": 0}} curl http://localhost:15808/2013-08-22/search?q=somethingnotpresent {"rank": "-text_relevance", "match-expr": "(label 'somethingnotpresent')", "hits": {"found": 0, "start": 0, "hit": []}, "info": {"rid": "869d2b07c1e47a55ab1cb4cd615953333e52d886112e916ed7fa447355f5a518b1c16bbcbf40cb7e", "time-ms": 5, "cpu-time-ms": 0}}# Remove the document in another batch update: curl -X POST -H "Content-Type: application/json" http://localhost:15808/2013-08-22/documents/batch -d '[{"version": 1376497963, "type": "delete", "id": 1246}]'{"status": "ok", "adds": 0, "deletes": 1, "error": "", "warning": ""}# Check what was removed: curl http://localhost:15808/dev/documents {"documents": [], "documents_removed": [{"_id": "1246", "lang": "en", "fields": {"name": "bob"}, "version": "1376497963", "id": "1246"}]}# Empty out all stored content: curl -X DELETE http://localhost:15808/dev/documents {"status": "ok", "message": "Documents Removed OK.", "error": "", "traceback": ""}# Check there should now be nothing there: curl http://localhost:15808/dev/documents {"documents": [], "documents_removed": []}
Development
我在mac osx上开发和维护项目。我已经用brew安装了docker composer、docker、virtualenvwrappers和python3。我用“制造”来帮助发展。
# create a quick environment to install into: mkvirtualenv --clear -p python3 nozama # (activate if needed) workon nozama # Install the project dependancies make install # Start the project dependancies ElasticSearch and Mongo make up # Run the API locally: make up
Contributing
如果可能的话,提交一个包含测试的拉取请求。我会审查,测试,通常会批准。所有测试都必须通过。我现在和Python赛跑。然后我将增加版本,添加属性,然后释放到https://hub.docker.com/r/oisinmulvihill/nozama-elasticsearch和pypi.org,如果一切正常的话。
Release Process
帮助oisin记住发布过程:
# clean env for release: mkvirtualenv --clear -p python3.7 nozama # setup and run all tests: # # make sure mongo and elasticsearch are running: make up # run all unit and acceptance tests in a completely isolated environment. make docker_test # Build and release to test.pypi.org first: make test_pypi_release # If all is good time to release to pypi.org make release_to_pypi # Now release the new docker container make docker_release
尝试pip install和docker pull以获取新的包和容器。
Versions
2.0.3
修正了加载程序初始化文件的错误。在我放弃名称空间包装之后,我忘记了这么做。
2.0.2
从api中去掉了旧的sphinx文档。http://localhost:15808/docs/现在只显示到github rest api示例的链接。去掉客户端代码和其他未使用文件中未使用的代码。
2.0.0 -> 2.0.1
在注意到很多人仍然在使用它之后更新了这个项目。我已经对它进行了更新,以反映我目前对构建restapi的想法,以及它们是如何打包、开发和发布的。
更改:
- rest api保持不变,但是现在可以进行搜索。
- 迁移到Python3。
- 将项目重构为一个python包,使其更易于处理和贡献。
- 现在使用docker compose帮助开发管理mongo和elasticsearch依赖关系。
- 作为发布过程的一部分,我现在生成了“office”nozama cloudsearch容器。
- 取消了与python的依赖关系,并移到使用需求文件来满足生产和测试需求。