animus-omni 0.0.2

工作原理

Animus Omni由部署在互联网上的传感器网络供电。这些传感器没有业务价值，但有一套全面的日志记录规则。这些日志在加载到通过animus api提供的数据库之前进行聚合和分析。omni reduce分析您的日志文件，并将元数据传递给我们的api。api根据元数据返回一个过滤器，然后应用到文件中。结果是减少了日志文件的噪音。

安装

从源存储库：

$ python setup.py install

或者通过pypi:

$ pip install animus-omni

配置

此命令将要求您提供电子邮件地址，该地址将为您注册一个速率受限的帐户，以便在alpha期间免费使用：

$ omni-reduce --configure

用法

omni reduce工具的命令行用法：

usage: omni-reduce [-h] [--type {auth,http,generic}] [--noise]
                   [--out-file OUTFILE] [--stats] [--dry-run] [--port PORTS]
                   [--configure]
                   [filename]

positional arguments:
  filename              Filename of log file to reduce

optional arguments:
  -h, --help            show this help message and exit
  --type {auth,http,generic}, -t {auth,http,generic}
                        Log type to analyze
  --noise, -n           Print the noise from the file rather than reducing it
  --out-file OUTFILE, -o OUTFILE
                        Output file for the result
  --stats, -s           Print statistics to STDERR from the reduction
                        operation
  --dry-run, -d         Don't output the reduced log file, only print possible
                        reduction statistics to STDERR
  --port PORTS, -p PORTS
                        Port and protocol used by generic mode. Can be used
                        multiple times. Should be of the form "80:TCP" or
                        "53:UDP"
  --configure           Configure Omni Reduce.

示例

将简化的验证日志输出到屏幕：

$ omni-reduce /var/log/auth.log
[Results not shown]

将简化的身份验证日志输出到文件并将聚合统计信息打印到屏幕：

$ omni-reduce --output ~/auth.log.reduced -s /var/log/auth.log
489 lines were analyzed in this log file.
356 lines were determined to be noise by Animus.
133 lines were not determined to be noise by Animus.
The input file was reduced to 27.2% of it's original size.

将简化的http访问日志输出到文件：

$ omni-reduce -t http --output ~/access.log.reduced /etc/log/access.log

来自http访问日志的输出行，animus认为这些行是bots、crawler或其他互联网噪音：

$ cat /etc/log/access.log | omni-reduce -t http -n
[Results not shown]

按Animus在TCP端口80上看到的流量显示减少访问日志的统计信息，并且不在屏幕上显示结果：

$ omni-reduce -t generic -p 80:tcp --dry-run test/data/access.log.txt

隐私声明

为了减少日志文件中的噪音，我们需要从这些文件中收集元数据。这包括IP地址、用户名、用户代理字符串、引用程序和请求uri。我们使用此元数据来增强API的结果。如果您的日志文件中有敏感数据，或者不希望与我们共享此数据，请通过info@animus.io与我们联系，以获取有关私有on premesis解决方案的信息。