快速记录内部气流

Question

我目前正在大量使用对速度敏感的脚本。 我试图记录高达每秒100克的大量信息

默认日志库正在工作，但速度太慢。 我当前的设置：

import logging
logger = logging.getLogger("airflow.task")
logger.info('.. start logging information')

我想使用faslogging库（https://github.com/brmmm3/fastlogging），它至少快5倍：

from fastlogging import LogInit
logger = LogInit(pathName="example1.log")
logger.info(".. start logging information fast")

我的问题是，我仍然希望使用airflow提供的所有优秀格式以及S3的自动备份。 要做到这一点，我需要使用一个类似于getLogger（“airflow.task”）的函数，而不必给LogInit指定路径名

from fastlogging import GetLogger

if __name__ == "__main__":
    logger = GetLogger("airflow.task")
    logger.debug("This is a debug message.")
    logger.rotate()
    logger.fatal("This is a fatal message.")
    logger.shutdown()
Warning: airflow.task: LogInit should be called first!

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-2-49874f7d5ab0> in <module>
      2 
      3 if __name__ == "__main__":
----> 4     logger = GetLogger()
      5     logger.debug("This is a debug message.")
      6     logger.rotate()

fastlogging/fastlogging.pyx in fastlogging.fastlogging.GetLogger()

fastlogging/fastlogging.pyx in fastlogging.fastlogging.Logger.__init__()

/opt/tljh/user/lib/python3.7/posixpath.py in dirname(p)
    154 def dirname(p):
    155     """Returns the directory component of a pathname"""
--> 156     p = os.fspath(p)
    157     sep = _get_sep(p)
    158     i = p.rfind(sep) + 1

TypeError: expected str, bytes or os.PathLike object, not NoneType

此外，不带文件名的LogInit：

from fastlogging import LogInit
logger = LogInit()
logger.info(".. start logging information fast")
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-3-d749b56426d4> in <module>
      1 from fastlogging import LogInit
----> 2 logger = LogInit()
      3 logger.info(".. start logging information fast")

fastlogging/fastlogging.pyx in fastlogging.fastlogging.LogInit()

fastlogging/fastlogging.pyx in fastlogging.fastlogging.GetLogger()

fastlogging/fastlogging.pyx in fastlogging.fastlogging.Logger.__init__()

/opt/tljh/user/lib/python3.7/posixpath.py in dirname(p)
    154 def dirname(p):
    155     """Returns the directory component of a pathname"""
--> 156     p = os.fspath(p)
    157     sep = _get_sep(p)
    158     i = p.rfind(sep) + 1

TypeError: expected str, bytes or os.PathLike object, not NoneType

有没有人试过或者知道怎么做？我没有足够的资源来分配部分https://www.astronomer.io/guides/logging

任何帮助都将不胜感激。谢谢