这是开发工具的子模块包
deepair-dev-utils的Python项目详细描述
深层开发工具
该软件包由专门用于数据操作和深空环境中的处理的开发人员实用程序组成。
包装结构
deepair_dev_utils
是的。
——概述
-init.py
——工具.py
-init.py
——装载机
-init.py
——tools.py
2个目录，5个文件
依赖关系
注意：运行此包需要以下python3包：
- 努比
- scipy
- 熊猫
- sklearn
- 全面质量管理
函数声明
下面是可用于DeepAir-dev的包中函数的签名。
一般.py
下面是可以通过将此模块导入为from deepair_dev_utils.general.tools import <function_name>来访问的函数。
log：
def log(message):
'''
prints message on console
input :
message : msg to print (string)
'''
get_data：
def get_data(path):
'''
Single file loader function
input :
path : abs path to load from (string)
'''
daterange：
def daterange(s_date, e_date):
'''
To return a list of all the dates from
start date to end date (excluding end date)
input :
s_date : start date (datetime)
e_date : end date (datetime)
returns :
list of dates
'''
jsonReader：
def jsonReader(path):
'''
JSON File Reader (from absolute path).
Args:
path : absolute path of json file (string)
Return:
data : loaded JSON
'''
jsonWriter：
def jsonWriter(data, path):
'''
JSON File Writer (to absolute path).
Args:
data : data to write (JSON/DICT/STRING)
path : absolute path of json file (string)
'''
ddmmyyyy2datetime：
def ddmmyyyy2datetime(start_date):
'''
Convert dd-mm-yyyy to std data time format.
Args:
start_date : date with dd-mm-yyyy (string)
Return:
date : converted format
'''
下面是可以通过将此模块导入为from deepair_dev_utils.general.decorators import <decorator_name>来访问的装饰器。
function_logger：
def function_logger(orig_func):
'''
Create a file with function.log (if possible)
otherwise with unknown_function.log and record
the arguments passed for the function
example:
@function_logger
def target_function(...):
...
'''
function_timer：
def function_timer(orig_func):
'''
Displays runtime on console
example:
@function_timer
def target_function(...):
...
'''
装载机
此子包包含用于将数据加载为Handler的工具。
处理程序
下面是可以通过将此模块导入为from deepair_dev_utils.loader.tools import Handler来访问的函数。
然后创建一个访问功能的对象。示例obj = Handler()，然后obj.<function_name>
__init__：
def __init__(self, verbose=True):
'''
Handlder (class) constructor.
inputs:
verbose: Indicator for log and progress bar (bool)
'''
loader：
def loader(self, dir_path, start_date, end_date,
prefix='', postfix='', ext='.csv'):
'''
Primary loader function to load the data from start date to
end date in concatinated (single dataframe) format.
inputs:
dir_path : absolute path to the directory path (series)
start_date : load start date in dd-mm-yyyy format (string)
end_date : load end date in dd-mm-yyyy format (string)
prefix : file prefix [if necessary] (string)
postfix : file postfix [if necessary] (string)
ext : file extension [default is .csv] (string)
return:
df: loaded concatenated dataframe (pandas df)
'''
loader_v2：
def loader_v2(self, dir_path, start_date, end_date,
prefix='', postfix='', ext='.csv'):
'''
(VERSION 2)
Primary loader function to load the data from start date to
end date in concatinated (single dataframe) format.
inputs:
dir_path : absolute path to the directory path (series)
start_date : load start date in yyyy-mm-dd format (string)
end_date : load end date in yyyy-mm-dd format (string)
prefix : file prefix [if necessary] (string)
postfix : file postfix [if necessary] (string)
ext : file extension [default is .csv] (string)
return:
df: loaded concatenated dataframe (pandas df)
'''
single_loader：
def single_loader(self, dir_path, start_date, end_date,
prefix='', postfix='', ext='.csv'):
'''
Single loader function to load the data from start date to
end date in individual datewise (each dataframe is of one date)
format.
inputs:
dir_path : absolute path to the directory path (series)
start_date : load start date in dd-mm-yyyy format (string)
end_date : load end date in dd-mm-yyyy format (string)
prefix : file prefix [if necessary] (string)
postfix : file postfix [if necessary] (string)
ext : file extension [default is .csv] (string)
return:
data: list of data frames datewise (list)
'''
batch_loader：
def batch_loader(self, dir_path, start_date, end_date,
batch_size=1, prefix='', postfix='', ext='.csv'):
'''
Batch loader function to load the data from start date to
end date in batches (each dataframe is in the form of batch datewise)
format.
inputs:
dir_path : absolute path to the directory path (series)
start_date : load start date in dd-mm-yyyy format (string)
end_date : load end date in dd-mm-yyyy format (string)
batch_size : batch size (int)
prefix : file prefix [if necessary] (string)
postfix : file postfix [if necessary] (string)
ext : file extension [default is .csv] (string)
return:
data: list of data frames datewise (list)
'''
_load_action：
def _load_action(self, df):
'''
@abstractmethod
User defined Bottle neck pipeline within load.
NOTE -> Default job of this function is pass i.e. do nothing
inputs:
df: Dataframe to apply this method on (pandas df)
return:
df: Modified dataframe (pandas df)
'''
欢迎加入QQ群-->： 979659372
