acrilog是一个在多处理环境中为我们提供多处理习惯用法的python库。

acrilog的Python项目详细描述


Overview

acrilog is a python library encapsulating multiprocessing logging into practical use.

acrilog started as Acrisel’s internal utility for programmers.

It included:
  1. Time and size rotating handler.
  2. Multiprocessing logging queue server

The library makes it easier to add logging in a multiprocessing environment where processes are split among multiple Python source codes.

We decided to contribute this library to Python community as a token of appreciation to what this community enables us.

We hope that you will find this library useful and helpful as we find it.

If you have comments or insights, please don’t hesitate to contact us at support@acrisel.com

TimedSizedRotatingHandler

Use TimedSizedRotatingHandler is combining TimedRotatingFileHandler with RotatingFileHandler. Usage as handler with logging is as defined in Python’s logging how-to

example

importlogging# create loggerlogger=logging.getLogger('simple_example')logger.setLevel(logging.DEBUG)# create console handler and set level to debugch=logging.TimedRotatingFileHandler()ch.setLevel(logging.DEBUG)# create formatterformatter=logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')# add formatter to chch.setFormatter(formatter)# add ch to loggerlogger.addHandler(ch)# 'application' codelogger.debug('debug message')logger.info('info message')logger.warn('warn message')logger.error('error message')logger.critical('critical message')

MpLogger and LevelBasedFormatter

Multiprocessor logger using QueueListener and QueueHandler It uses TimedSizedRotatingHandler as its logging handler

It also uses acris provided LevelBasedFormatter which facilitate message formats based on record level. LevelBasedFormatter inherent from logging.Formatter and can be used as such in customized logging handlers.

example

在主进程中
importtimeimportrandomimportloggingfromacrisimportMpLoggerimportosimportmultiprocessingasmpdefsubproc(limit=1,logger_info=None):logger=MpLogger.get_logger(logger_info,name="acrilog.subproc",)foriinrange(limit):sleep_time=3/random.randint(1,10)time.sleep(sleep_time)logger.info("proc [%s]: %s/%s - sleep %4.4ssec"%(os.getpid(),i,limit,sleep_time))level_formats={logging.DEBUG:"[ %(asctime)s ][ %(levelname)s ][ %(message)s ][ %(module)s.%(funcName)s(%(lineno)d) ]",'default':"[ %(asctime)s ][ %(levelname)s ][ %(message)s ]",}mplogger=MpLogger(logging_level=logging.DEBUG,level_formats=level_formats,datefmt='%Y-%m-%d,%H:%M:%S.%f')mplogger.start(name='main_process')logger=MpLogger.get_logger(mplogger.logger_info())logger.debug("starting sub processes")procs=list()forlimitin[1,1]:proc=mp.Process(target=subproc,args=(limit,mplogger.logger_info(),))procs.append(proc)proc.start()forprocinprocs:ifproc:proc.join()logger.debug("sub processes completed")mplogger.stop()

Example output

[2016-12-19,11:39:44.953189][DEBUG][startingsubprocesses][mplogger.<module>(45)][2016-12-19,11:39:45.258794][INFO][proc[932]:0/1-sleep0.3sec][2016-12-19,11:39:45.707914][INFO][proc[931]:0/1-sleep0.75sec][2016-12-19,11:39:45.710487][DEBUG][subprocessescompleted][mplogger.<module>(56)]

Clarification of parameters

name

name标识记录器的基名称。注意,此参数在mplogger init方法及其start方法中都可用。

当设置了consolidate时,mplogger init的name参数用于合并记录器。如果调用start()方法时未提供主进程的私有记录器,则它也用于主进程的私有记录器。

proecess_key

process_key定义一个或多个日志记录字段,该字段将成为日志文件名的一部分。如果使用它,记录器将为每个记录的进程密钥提供一个文件。如果设置了consolidate,这将是对合并日志的补充。

默认情况下,mplogger使用name作为进程键。如果提供了其他内容,例如processname,则它将作为后缀连接到name

file_prefix and file_suffix

允许通过设置file_prefixfile_suffix中的一个(或两个)来区分不同运行的日志集。通常,使用pid和粒度datetime作为前缀或后缀会创建一组唯一的日志。

file_mode

file_mode让程序定义如何打开日志。默认情况下,日志以追加模式打开。母鸡,历史被收集起来,并在一夜之间按大小归档。

consolidate

consolidate设置后,将从所有处理日志创建合并日志。 如果设置了consolidated,并且在没有name的情况下调用start(),则将合并到主进程中。

kwargs

kwargs是将传递给filehandler的命名参数。这包括:

file_mode='a', for RotatingFileHandler
maxBytes=0, for RotatingFileHandler
backupCount=0, for RotatingFileHandler and TimedRotatingFileHandler
encoding='ascii', for RotatingFileHandler and TimedRotatingFileHandler
delay=False, for TimedRotatingFileHandler
when='h', for TimedRotatingFileHandler
interval=1, TimedRotatingFileHandler
utc=False, TimedRotatingFileHandler
atTime=None, for TimedRotatingFileHandler

Change History

v0.9

  1. added ability to pass logger_info to subprocess,
  2. exposed encoding parameter

v1.0

  1. replaced force_global with consolidate to genrerate consolidated log
  2. add name argument to MpLogger.start(). This will return logger with that name for the main process.
  3. MpLogger.__init__() name argument will be used for consolidated log.

v1.1

  1. add file_prefix and file_suffix as MpLogger parameters.
  2. fix bug when logdir is None.

v2.0

  1. added NwLogger starting a server logger with NwLoggerClientHandler for remote processes.

Next Steps

  1. Cluster support using TCP/IP
  2. Logging monitor and alert

欢迎加入QQ群-->: 979659372 Python中文网_新手群

推荐PyPI第三方库


热门话题
IntelliJ中的java默认Maven项目结构不一致   java我希望链接(在帖子和页面上)在一些访问者加载时被自动点击   java如何使用单独的方法隐藏JButton并在新类中调用   java KStream leftJoin KStream具有相同的密钥   java图像在垂直滚动窗格视图端口中消失   java从指定的起始点开始以n的增量填充数组   java JLabel和JTextField不在swing表单中应用   java springboot mongo如果没有映像,请使用现有映像   类似C++映射的java容器   java如何在没有Valgrind错误的情况下调用JNI_CreateJavaVM?   java如何在安卓中运行后台服务   java onPostExecute不运行