acris是一个python编程模式库，在acrisel，我们在python项目中使用它，并选择为python社区做出贡献。
acris的Python项目详细描述
Overview
acris is a python library providing useful programming patterns and tools.
acris started as Acrisel’s internal idioms and utilities for programmers.
- It included:
- programming idioms that are repeatedly used by programmers.
- utilities that helps programmers and administrators manage their environments
We decided to contribute this library to Python community as a token of appreciation to what this community enables us.
We hope that you will find this library useful and helpful as we find it.
If you have comments or insights, please don’t hesitate to contact us at support@acrisel.com
Programming Idoms
threaded
Note: inherent from acrilib decorator for methods that can be executed as a thread. RetriveAsycValue callable class used in the example below provide means to access results. One can provide their own callable to pass results.
示例
fromacrisimportthreaded,RetriveAsycValuefromtimeimportsleepclassThreadedExample(object):@threadeddefproc(self,id_,num,stall):s=numwhilenum>0:print("%s: %s"%(id_,s))num-=1s+=stallsleep(stall)print("%s: %s"%(id_,s))returns
示例输出
print("starting workers")te1=ThreadedExample().proc('TE1',3,1)te2=ThreadedExample().proc('TE2',3,1)print("collecting results")te1_callback=RetriveAsycValue('te1')te1.addCallback(te1_callback)te2_callback=RetriveAsycValue('te2')te2.addCallback(te2_callback)print('joining t1')te1.join()print('joined t1')print('%s callback result: %s'%(te1_callback.name,te1_callback.result))result=te1.syncResult()print('te1 syncResult : %s'%result)result=te2.syncResult()print('te2 syncResult : %s'%result)print('%s callback result: %s'%(te2_callback.name,te2_callback.result))will produce:
startingworkersTE1:3TE2:3collectingresultsjoiningt1TE1:4TE2:4TE1:5TE2:5TE1:6TE2:6joinedt1te1callbackresult:6te1syncResult:6te2syncResult:6te2callbackresult:6
Singleton and NamedSingleton
Note: inherent from acrilib meta class that creates singleton footprint of classes inheriting from it.
单例
fromacrisimportSingletonclassSequence(Singleton):step_id=0def__call__(self):step_id=self.step_idself.step_id+=1returnstep_id
fromacrisimportSingletonclassSequence(Singleton):step_id=0def__call__(self):step_id=self.step_idself.step_id+=1returnstep_id
示例输出
A=Sequence()print('A',A())print('A',A())B=Sequence()print('B',B())will produce:
A0A1B2
命名singleton示例
fromacrisimportSingletonclassSequence(NamedSingleton):step_id=0def__init__(self,name=''):self.name=namedef__call__(self,):step_id=self.step_idself.step_id+=1returnstep_id
fromacrisimportSingletonclassSequence(NamedSingleton):step_id=0def__init__(self,name=''):self.name=namedef__call__(self,):step_id=self.step_idself.step_id+=1returnstep_id
示例输出
A=Sequence('A')print(A.name,A())print(A.name,A())B=Sequence('B')print(B.name,B())will produce:
A0A1B0
Sequence
Note: inherent from acrilib meta class to produce sequences. Sequence allows creating different sequences using name tags.
示例
fromacrisimportSequenceA=Sequence('A')print('A',A())print('A',A())B=Sequence('B')print('B',B())A=Sequence('A')print('A',A())print('A',A())B=Sequence('B')print('B',B())
示例输出
A0A1B0A2A3B1
TimedSizedRotatingHandler
use acrilog instead.
MpLogger and LevelBasedFormatter
use acrilog instead.
Decorators
Note: inherent from acrilib Useful decorators for production and debug.
追踪法
logs entry and exit of function or method.
fromacrisimporttraced_methodtraced=traced_method(print,print_args=True,print_result=True)classOper(object):def__init__(self,value):self.value=valuedef__repr__(self):returnstr(self.value)@traceddefmul(self,value):self.value*=valuereturnself@traceddefadd(self,value):self.value+=valuereturnselfo=Oper(3)print(o.add(2).mul(5).add(7).mul(8))
would result with the following output:
[add][entering][args:(2)][kwargs:{}][trace_methods.py.Oper(39)][add][exiting][timespan:0:00:00.000056][result:5][trace_methods.py.Oper(39)][mul][entering][args:(5)][kwargs:{}][trace_methods.py.Oper(34)][mul][exiting][timespan:0:00:00.000010][result:25][trace_methods.py.Oper(34)][add][entering][args:(7)][kwargs:{}][trace_methods.py.Oper(39)][add][exiting][timespan:0:00:00.000007][result:32][trace_methods.py.Oper(39)][mul][entering][args:(8)][kwargs:{}][trace_methods.py.Oper(34)][mul][exiting][timespan:0:00:00.000008][result:256][trace_methods.py.Oper(34)]256
logs entry and exit of function or method.
fromacrisimporttraced_methodtraced=traced_method(print,print_args=True,print_result=True)classOper(object):def__init__(self,value):self.value=valuedef__repr__(self):returnstr(self.value)@traceddefmul(self,value):self.value*=valuereturnself@traceddefadd(self,value):self.value+=valuereturnselfo=Oper(3)print(o.add(2).mul(5).add(7).mul(8))
would result with the following output:
[add][entering][args:(2)][kwargs:{}][trace_methods.py.Oper(39)][add][exiting][timespan:0:00:00.000056][result:5][trace_methods.py.Oper(39)][mul][entering][args:(5)][kwargs:{}][trace_methods.py.Oper(34)][mul][exiting][timespan:0:00:00.000010][result:25][trace_methods.py.Oper(34)][add][entering][args:(7)][kwargs:{}][trace_methods.py.Oper(39)][add][exiting][timespan:0:00:00.000007][result:32][trace_methods.py.Oper(39)][mul][entering][args:(8)][kwargs:{}][trace_methods.py.Oper(34)][mul][exiting][timespan:0:00:00.000008][result:256][trace_methods.py.Oper(34)]256
Data Types
Note: inherent from acrilib varies derivative of Python data types
合并链信息技术
Similar to ChainedDict, but merged the keys and is actually derivative of dict.
a={1:11,2:22}b={3:33,4:44}c={1:55,4:66}d=MergedChainedDict(c,b,a)print(d)
Will output:
{1:55,2:22,3:33,4:66}
Similar to ChainedDict, but merged the keys and is actually derivative of dict.
a={1:11,2:22}b={3:33,4:44}c={1:55,4:66}d=MergedChainedDict(c,b,a)print(d)
Will output:
{1:55,2:22,3:33,4:66}
ResourcePool
Resource pool provides program with interface to manager resource pools. This is used as means to funnel processing.
ResourcePoolRequestor object can be used to request resource set resides in multiple pools.
ResourcePoolRequestors object manages multiple requests for multiple resources.
同步示例
importtimefromacrisimportresource_poolasrpfromacrisimportThreadedimportqueuefromdatetimeimportdatetimeclassMyResource1(rp.Resource):passclassMyResource2(rp.Resource):passrp1=rp.ResourcePool('RP1',resource_cls=MyResource1,policy={'resource_limit':2,}).load()rp2=rp.ResourcePool('RP2',resource_cls=MyResource2,policy={'resource_limit':1,}).load()@Threaded()defworker_awaiting(name,rp):print('[ %s ] %s getting resource'%(str(datetime.now()),name))r=rp.get()print('[ %s ] %s doing work (%s)'%(str(datetime.now()),name,repr(r)))time.sleep(4)print('[ %s ] %s returning %s'%(str(datetime.now()),name,repr(r)))rp.put(*r)r1=worker_awaiting('>>> w11-direct',rp1)r2=worker_awaiting('>>> w21-direct',rp2)r3=worker_awaiting('>>> w22-direct',rp2)r4=worker_awaiting('>>> w12-direct',rp1)
同步示例输出
[2016-12-1113:06:14.659569]>>>w11-directgettingresource[2016-12-1113:06:14.659640]>>>w11-directdoingwork([Resource(name:MyResource1)])[2016-12-1113:06:14.659801]>>>w21-directgettingresource[2016-12-1113:06:14.659834]>>>w21-directdoingwork([Resource(name:MyResource2)])[2016-12-1113:06:14.659973]>>>w22-directgettingresource[2016-12-1113:06:14.660190]>>>w12-directgettingresource[2016-12-1113:06:14.660260]>>>w12-directdoingwork([Resource(name:MyResource1)])[2016-12-1113:06:18.662362]>>>w11-directreturning[Resource(name:MyResource1)][2016-12-1113:06:18.662653]>>>w21-directreturning[Resource(name:MyResource2)][2016-12-1113:06:18.662826]>>>w12-directreturning[Resource(name:MyResource1)][2016-12-1113:06:18.662998]>>>w22-directdoingwork([Resource(name:MyResource2)])[2016-12-1113:06:22.667149]>>>w22-directreturning[Resource(name:MyResource2)]
[2016-12-1113:06:14.659569]>>>w11-directgettingresource[2016-12-1113:06:14.659640]>>>w11-directdoingwork([Resource(name:MyResource1)])[2016-12-1113:06:14.659801]>>>w21-directgettingresource[2016-12-1113:06:14.659834]>>>w21-directdoingwork([Resource(name:MyResource2)])[2016-12-1113:06:14.659973]>>>w22-directgettingresource[2016-12-1113:06:14.660190]>>>w12-directgettingresource[2016-12-1113:06:14.660260]>>>w12-directdoingwork([Resource(name:MyResource1)])[2016-12-1113:06:18.662362]>>>w11-directreturning[Resource(name:MyResource1)][2016-12-1113:06:18.662653]>>>w21-directreturning[Resource(name:MyResource2)][2016-12-1113:06:18.662826]>>>w12-directreturning[Resource(name:MyResource1)][2016-12-1113:06:18.662998]>>>w22-directdoingwork([Resource(name:MyResource2)])[2016-12-1113:06:22.667149]>>>w22-directreturning[Resource(name:MyResource2)]
异步示例
importtimefromacrisimportresource_poolasrpfromacrisimportThreadedimportqueuefromdatetimeimportdatetimeclassMyResource1(rp.Resource):passclassMyResource2(rp.Resource):passrp1=rp.ResourcePool('RP1',resource_cls=MyResource1,policy={'resource_limit':2,}).load()rp2=rp.ResourcePool('RP2',resource_cls=MyResource2,policy={'resource_limit':1,}).load()classCallback(object):def__init__(self,notify_queue):self.q=notify_queuedef__call__(self,resources=None):self.q.put(resources)@Threaded()defworker_callback(name,rp):print('[ %s ] %s getting resource'%(str(datetime.now()),name))notify_queue=queue.Queue()r=rp.get(callback=Callback(notify_queue))ifnotr:print('[ %s ] %s doing work before resource available'%(str(datetime.now()),name,))print('[ %s ] %s waiting for resources'%(str(datetime.now()),name,))ticket=notify_queue.get()r=rp.get(ticket=ticket)print('[ %s ] %s doing work (%s)'%(str(datetime.now()),name,repr(r)))time.sleep(2)print('[ %s ] %s returning (%s)'%(str(datetime.now()),name,repr(r)))rp.put(*r)r1=worker_callback('>>> w11-callback',rp1)r2=worker_callback('>>> w21-callback',rp2)r3=worker_callback('>>> w22-callback',rp2)r4=worker_callback('>>> w12-callback',rp1)
异步示例输出
[2016-12-1113:08:24.410447]>>>w11-callbackgettingresource[2016-12-1113:08:24.410539]>>>w11-callbackdoingwork([Resource(name:MyResource1)])[2016-12-1113:08:24.410682]>>>w21-callbackgettingresource[2016-12-1113:08:24.410762]>>>w21-callbackdoingwork([Resource(name:MyResource2)])[2016-12-1113:08:24.410945]>>>w22-callbackgettingresource[2016-12-1113:08:24.411227]>>>w22-callbackdoingworkbeforeresourceavailable[2016-12-1113:08:24.411273]>>>w12-callbackgettingresource[2016-12-1113:08:24.411334]>>>w22-callbackwaitingforresources[2016-12-1113:08:24.411452]>>>w12-callbackdoingwork([Resource(name:MyResource1)])[2016-12-1113:08:26.411901]>>>w11-callbackreturning([Resource(name:MyResource1)])[2016-12-1113:08:26.412200]>>>w21-callbackreturning([Resource(name:MyResource2)])[2016-12-1113:08:26.412505]>>>w22-callbackdoingwork([Resource(name:MyResource2)])[2016-12-1113:08:26.416130]>>>w12-callbackreturning([Resource(name:MyResource1)])[2016-12-1113:08:28.416001]>>>w22-callbackreturning([Resource(name:MyResource2)])
请求者示例
importtimefromacrisimportresource_poolasrpfromacrisimportThreadedimportqueuefromdatetimeimportdatetimeclassMyResource1(rp.Resource):passclassMyResource2(rp.Resource):passrp1=rp.ResourcePool('RP1',resource_cls=MyResource1,policy={'resource_limit':2,}).load()rp2=rp.ResourcePool('RP2',resource_cls=MyResource2,policy={'resource_limit':2,}).load()classCallback(object):def__init__(self,notify_queue):self.q=notify_queuedef__call__(self,ready=False):self.q.put(ready)@Threaded()defworker_callback(name,rps):print('[ %s ] %s getting resource'%(str(datetime.now()),name))notify_queue=queue.Queue()callback=Callback(notify_queue,name=name)request=rp.Requestor(request=rps,callback=callback)ifrequest.is_reserved():resources=request.get()else:print('[ %s ] %s doing work before resource available'%(str(datetime.now()),name,))print('[ %s ] %s waiting for resources'%(str(datetime.now()),name,))notify_queue.get()resources=request.get()print('[ %s ] %s doing work (%s)'%(str(datetime.now()),name,repr(resources)))time.sleep(2)print('[ %s ] %s returning (%s)'%(str(datetime.now()),name,repr(resources)))request.put(*resources)r1=worker_callback('>>> w11-callback',[(rp1,1),])r2=worker_callback('>>> w21-callback',[(rp1,1),(rp2,1)])r3=worker_callback('>>> w22-callback',[(rp1,1),(rp2,1)])r4=worker_callback('>>> w12-callback',[(rp1,1),])
importtimefromacrisimportresource_poolasrpfromacrisimportThreadedimportqueuefromdatetimeimportdatetimeclassMyResource1(rp.Resource):passclassMyResource2(rp.Resource):passrp1=rp.ResourcePool('RP1',resource_cls=MyResource1,policy={'resource_limit':2,}).load()rp2=rp.ResourcePool('RP2',resource_cls=MyResource2,policy={'resource_limit':2,}).load()classCallback(object):def__init__(self,notify_queue):self.q=notify_queuedef__call__(self,ready=False):self.q.put(ready)@Threaded()defworker_callback(name,rps):print('[ %s ] %s getting resource'%(str(datetime.now()),name))notify_queue=queue.Queue()callback=Callback(notify_queue,name=name)request=rp.Requestor(request=rps,callback=callback)ifrequest.is_reserved():resources=request.get()else:print('[ %s ] %s doing work before resource available'%(str(datetime.now()),name,))print('[ %s ] %s waiting for resources'%(str(datetime.now()),name,))notify_queue.get()resources=request.get()print('[ %s ] %s doing work (%s)'%(str(datetime.now()),name,repr(resources)))time.sleep(2)print('[ %s ] %s returning (%s)'%(str(datetime.now()),name,repr(resources)))request.put(*resources)r1=worker_callback('>>> w11-callback',[(rp1,1),])r2=worker_callback('>>> w21-callback',[(rp1,1),(rp2,1)])r3=worker_callback('>>> w22-callback',[(rp1,1),(rp2,1)])r4=worker_callback('>>> w12-callback',[(rp1,1),])
请求程序示例输出
[2016-12-1306:27:54.924629]>>>w11-callbackgettingresource[2016-12-1306:27:54.925094]>>>w21-callbackgettingresource[2016-12-1306:27:54.925453]>>>w22-callbackgettingresource[2016-12-1306:27:54.926188]>>>w12-callbackgettingresource[2016-12-1306:27:54.932922]>>>w11-callbackdoingwork([Resource(name:MyResource1)])[2016-12-1306:27:54.933709]>>>w12-callbackdoingwork([Resource(name:MyResource1)])[2016-12-1306:27:54.938425]>>>w22-callbackdoingworkbeforeresourceavailable[2016-12-1306:27:54.938548]>>>w22-callbackwaitingforresources[2016-12-1306:27:54.939256]>>>w21-callbackdoingworkbeforeresourceavailable[2016-12-1306:27:54.939267]>>>w21-callbackwaitingforresources[2016-12-1306:27:56.936881]>>>w11-callbackreturning([Resource(name:MyResource1)])[2016-12-1306:27:56.937543]>>>w12-callbackreturning([Resource(name:MyResource1)])[2016-12-1306:27:56.947615]>>>w22-callbackdoingwork([Resource(name:MyResource2),Resource(name:MyResource1)])[2016-12-1306:27:56.948587]>>>w21-callbackdoingwork([Resource(name:MyResource2),Resource(name:MyResource1)])[2016-12-1306:27:58.949812]>>>w22-callbackreturning([Resource(name:MyResource2),Resource(name:MyResource1)])[2016-12-1306:27:58.950064]>>>w21-callbackreturning([Resource(name:MyResource2),Resource(name:MyResource1)])
[2016-12-1306:27:54.924629]>>>w11-callbackgettingresource[2016-12-1306:27:54.925094]>>>w21-callbackgettingresource[2016-12-1306:27:54.925453]>>>w22-callbackgettingresource[2016-12-1306:27:54.926188]>>>w12-callbackgettingresource[2016-12-1306:27:54.932922]>>>w11-callbackdoingwork([Resource(name:MyResource1)])[2016-12-1306:27:54.933709]>>>w12-callbackdoingwork([Resource(name:MyResource1)])[2016-12-1306:27:54.938425]>>>w22-callbackdoingworkbeforeresourceavailable[2016-12-1306:27:54.938548]>>>w22-callbackwaitingforresources[2016-12-1306:27:54.939256]>>>w21-callbackdoingworkbeforeresourceavailable[2016-12-1306:27:54.939267]>>>w21-callbackwaitingforresources[2016-12-1306:27:56.936881]>>>w11-callbackreturning([Resource(name:MyResource1)])[2016-12-1306:27:56.937543]>>>w12-callbackreturning([Resource(name:MyResource1)])[2016-12-1306:27:56.947615]>>>w22-callbackdoingwork([Resource(name:MyResource2),Resource(name:MyResource1)])[2016-12-1306:27:56.948587]>>>w21-callbackdoingwork([Resource(name:MyResource2),Resource(name:MyResource1)])[2016-12-1306:27:58.949812]>>>w22-callbackreturning([Resource(name:MyResource2),Resource(name:MyResource1)])[2016-12-1306:27:58.950064]>>>w21-callbackreturning([Resource(name:MyResource2),Resource(name:MyResource1)])
Virtual ResourcePool
Like ResourcePool, VResourcePool manages resources. The main difference between the two is that ResourcePool manages physical resource objects. VResourcePool manages virtual resources (VResource) that only represent physical resources. VResources can not be activated or deactivated.
One unique property VResourcePool enables is that request could be returned by quantity.
虚拟请求者示例
importtimefromacrisimportvirtual_resource_poolasrpfromacris.threadedimportThreadedfromacris.mploggerimportcreate_stream_handlerimportqueuefromdatetimeimportdatetimeclassMyResource1(rp.Resource):passclassMyResource2(rp.Resource):passrp1=rp.ResourcePool('RP1',resource_cls=MyResource1,policy={'resource_limit':2,}).load()rp2=rp.ResourcePool('RP2',resource_cls=MyResource2,policy={'resource_limit':1,}).load()classCallback(object):def__init__(self,notify_queue,name=''):self.q=notify_queueself.name=namedef__call__(self,received=False):self.q.put(received)requestors=rp.Requestors()@Threaded()defworker_callback(name,rps):print('[ %s ] %s getting resource'%(str(datetime.now()),name))notify_queue=queue.Queue()callback=Callback(notify_queue,name=name)request_id=requestors.reserve(request=rps,callback=callback)ifnotrequestors.is_reserved(request_id):print('[ %s ] %s doing work before resource available'%(str(datetime.now()),name,))notify_queue.get()resources=requestors.get(request_id)print('[ %s ] %s doing work (%s)'%(str(datetime.now()),name,repr(resources)))time.sleep(1)print('[ %s ] %s returning (%s)'%(str(datetime.now()),name,repr(resources)))requestors.put_requested(rps)r2=worker_callback('>>> w21-callback',[(rp1,1),(rp2,1)])r1=worker_callback('>>> w11-callback',[(rp1,1),])r3=worker_callback('>>> w22-callback',[(rp1,1),(rp2,1)])r4=worker_callback('>>> w12-callback',[(rp1,1),])
importtimefromacrisimportvirtual_resource_poolasrpfromacris.threadedimportThreadedfromacris.mploggerimportcreate_stream_handlerimportqueuefromdatetimeimportdatetimeclassMyResource1(rp.Resource):passclassMyResource2(rp.Resource):passrp1=rp.ResourcePool('RP1',resource_cls=MyResource1,policy={'resource_limit':2,}).load()rp2=rp.ResourcePool('RP2',resource_cls=MyResource2,policy={'resource_limit':1,}).load()classCallback(object):def__init__(self,notify_queue,name=''):self.q=notify_queueself.name=namedef__call__(self,received=False):self.q.put(received)requestors=rp.Requestors()@Threaded()defworker_callback(name,rps):print('[ %s ] %s getting resource'%(str(datetime.now()),name))notify_queue=queue.Queue()callback=Callback(notify_queue,name=name)request_id=requestors.reserve(request=rps,callback=callback)ifnotrequestors.is_reserved(request_id):print('[ %s ] %s doing work before resource available'%(str(datetime.now()),name,))notify_queue.get()resources=requestors.get(request_id)print('[ %s ] %s doing work (%s)'%(str(datetime.now()),name,repr(resources)))time.sleep(1)print('[ %s ] %s returning (%s)'%(str(datetime.now()),name,repr(resources)))requestors.put_requested(rps)r2=worker_callback('>>> w21-callback',[(rp1,1),(rp2,1)])r1=worker_callback('>>> w11-callback',[(rp1,1),])r3=worker_callback('>>> w22-callback',[(rp1,1),(rp2,1)])r4=worker_callback('>>> w12-callback',[(rp1,1),])
虚拟请求程序示例输出
[2016-12-1614:27:53.224110]>>>w21-callbackgettingresource[2016-12-1614:27:53.224750]>>>w11-callbackgettingresource[2016-12-1614:27:53.225567]>>>w22-callbackgettingresource[2016-12-1614:27:53.226220]>>>w12-callbackgettingresource[2016-12-1614:27:53.237146]>>>w11-callbackdoingwork([Resource(name:MyResource1)])[2016-12-1614:27:53.238361]>>>w12-callbackdoingworkbeforeresourceavailable[2016-12-1614:27:53.241046]>>>w21-callbackdoingworkbeforeresourceavailable[2016-12-1614:27:53.242350]>>>w22-callbackdoingwork([Resource(name:MyResource1),Resource(name:MyResource2)])[2016-12-1614:27:54.238443]>>>w11-callbackreturning([Resource(name:MyResource1)])[2016-12-1614:27:54.246868]>>>w22-callbackreturning([Resource(name:MyResource1),Resource(name:MyResource2)])[2016-12-1614:27:54.257040]>>>w12-callbackdoingwork([Resource(name:MyResource1)])[2016-12-1614:27:54.259858]>>>w21-callbackdoingwork([Resource(name:MyResource1),Resource(name:MyResource2)])[2016-12-1614:27:55.258659]>>>w12-callbackreturning([Resource(name:MyResource1)])[2016-12-1614:27:55.262741]>>>w21-callbackreturning([Resource(name:MyResource1),Resource(name:MyResource2)])
[2016-12-1614:27:53.224110]>>>w21-callbackgettingresource[2016-12-1614:27:53.224750]>>>w11-callbackgettingresource[2016-12-1614:27:53.225567]>>>w22-callbackgettingresource[2016-12-1614:27:53.226220]>>>w12-callbackgettingresource[2016-12-1614:27:53.237146]>>>w11-callbackdoingwork([Resource(name:MyResource1)])[2016-12-1614:27:53.238361]>>>w12-callbackdoingworkbeforeresourceavailable[2016-12-1614:27:53.241046]>>>w21-callbackdoingworkbeforeresourceavailable[2016-12-1614:27:53.242350]>>>w22-callbackdoingwork([Resource(name:MyResource1),Resource(name:MyResource2)])[2016-12-1614:27:54.238443]>>>w11-callbackreturning([Resource(name:MyResource1)])[2016-12-1614:27:54.246868]>>>w22-callbackreturning([Resource(name:MyResource1),Resource(name:MyResource2)])[2016-12-1614:27:54.257040]>>>w12-callbackdoingwork([Resource(name:MyResource1)])[2016-12-1614:27:54.259858]>>>w21-callbackdoingwork([Resource(name:MyResource1),Resource(name:MyResource2)])[2016-12-1614:27:55.258659]>>>w12-callbackreturning([Resource(name:MyResource1)])[2016-12-1614:27:55.262741]>>>w21-callbackreturning([Resource(name:MyResource1),Resource(name:MyResource2)])
Mediator
Note: inherent from acrilib Class interface to generator allowing query of has_next()
示例
fromacrisimportMediatordefyrange(n):i=0whilei<n:yieldii+=1n=10m=Mediator(yrange(n))foriinrange(n):print(i,m.has_next(3),next(m))print(i,m.has_next(),next(m))
示例输出
0True01True12True23True34True45True56True67True78False89False9Traceback(mostrecentcalllast):File"/private/var/acrisel/sand/acris/acris/acris/example/mediator.py",line19,in<module>print(i,m.has_next(),next(m))File"/private/var/acrisel/sand/acris/acris/acris/acris/mediator.py",line38,in__next__value=next(self.generator)StopIteration
Utilities
commdir.py
usage:commdir.py[-h][--dir1DIR1][--dir2DIR2][--quiet][--out[REPORT]][--follow][--detailed][--sync-cmd][--merge][--total][--ignore[PATTERN[PATTERN...]]]Reportsdifferencesindirectorystructureandcontent.commdir.pywillexitwith0ifdirectoriesfoundthesame.otherwise,itwillexitwith1.optionalarguments:-h,--helpshowthishelpmessageandexit--dir1DIR1sourcefolderforthecomparison--dir2DIR2targetfolderforthecomparison--quietavoidwritinganyreportout,default:False--out[REPORT]filetowritereportto,default:stdout--followfollowlinkswhenwalkingfolders,default:False--detailedprovidedetailedfileleveldiff,default:False--sync-cmdprovidecommandsthatwouldaligndirsandfiles,default:False--mergewhensync-cmd,sethowdiffcommandswouldberesolved,default:dir1isbase.--totaloutputssummary.--ignore[PATTERN[PATTERN...]]patterntoignoreexample:pythoncommdir.py--dir1my_folder--dir2other_folder--ignore__pycache__.*DS_Storecommdir.py also provides access to its underlined function commdir:
commdir(dir1,dir2,ignore=[],detailed=False,followlinks=False,quiet=False,bool_result=True)compares two directory structures and their files.
commdir walks through two directories, dir1 and dir2. While walking, it aggregates information on the difference between the two structures and their content.
If bool_result is True, commdir will return True if difference was found. When False, it would return a DiffContent namedtuple with the following fields:
- diff (boolean)
- folders_only_in_dir1 (list)
- folders_only_in_dir2 (list)
- files_only_in_dir1 (list)
- files_only_in_dir2 (list)
- diff_files (list)
- diff_detail (list)
- 参数：
- dir1，dir2：要比较的两个目录结构。 忽略：要忽略的正则表达式字符串列表，当忽略目录时，其所有子文件夹也将被忽略。 详细：如果设置，将生成详细的文件级比较。 followlinks：如果设置，则将遵循符号链接。 安静：如果设置，信息将不会打印到stdio。 bool_result：指示函数如何响应调用方（true:boolean或false:diffcontent）
commdir示例输出
----------------------------foldersonlyinother_folder----------------------------static/admin/fontsstatic/admin/js/vendorstatic/admin/js/vendor/jquerystatic/admin/js/vendor/xregexp-----------------------filesonlyinmy_folder-----------------------docs/._example.rstdocs/._user_guide.rst--------------------------filesonlyinother_folder--------------------------static/admin/css/fonts.cssstatic/admin/fonts/LICENSE.txtstatic/admin/fonts/README.txtffstatic/admin/img/LICENSEstatic/admin/js/vendor/jquery/jquery.jsstatic/admin/js/vendor/jquery/jquery.min.jsstatic/admin/js/vendor/xregexp/xregexp.min.js----------------filesdifferent:----------------.pydevprojectui/settings/prod.pyui/wsgi.pypersonalenv.xml--------Summary:--------Foldersonlyinmy_folder:0Filesonlyinmy_folder:2Foldersonlyinother_folder:4Filesonlyinother_folder:7Filesdifferent:4
----------------------------foldersonlyinother_folder----------------------------static/admin/fontsstatic/admin/js/vendorstatic/admin/js/vendor/jquerystatic/admin/js/vendor/xregexp-----------------------filesonlyinmy_folder-----------------------docs/._example.rstdocs/._user_guide.rst--------------------------filesonlyinother_folder--------------------------static/admin/css/fonts.cssstatic/admin/fonts/LICENSE.txtstatic/admin/fonts/README.txtffstatic/admin/img/LICENSEstatic/admin/js/vendor/jquery/jquery.jsstatic/admin/js/vendor/jquery/jquery.min.jsstatic/admin/js/vendor/xregexp/xregexp.min.js----------------filesdifferent:----------------.pydevprojectui/settings/prod.pyui/wsgi.pypersonalenv.xml--------Summary:--------Foldersonlyinmy_folder:0Filesonlyinmy_folder:2Foldersonlyinother_folder:4Filesonlyinother_folder:7Filesdifferent:4
bee.py
utility to run commands on multiple hosts and collect responses.
usage:bee.py[-h]-cCOMMAND[-pPARALLEL]-tHOST[-uUSERNAME][--sudo-userUSERNAME][--keep-log]Sendssshcommandtomultipledestinations.optionalarguments:-h,--helpshowthishelpmessageandexit-cCOMMAND,--commandCOMMANDcommandtoexecuteoversshchannel-pPARALLEL,--parallelPARALLELnumberofparallelsessiontoopen-tHOST,--targetHOSTdestinationhosttorunagainst-uUSERNAME,--userUSERNAMEusertouseforsshauthentication--sudo-userUSERNAMEsudousertousetoruncommands--keep-logindicatesbeetokeephostlogsinsteadofdeleting
csv2xlsx.py
converts multiple CSV file to XLSX file. Each CSV file will end on its own sheet.
usage:csv2xlsx.py[-h][-dDELIMITER][-oOUTFILE]CSV[CSV...]CreatesExcelfilefromoneormoreCSVfiles.IfmultipleCSVareprovided,theywiullbemappedtoseparatedsheets.If"-"isprovided,inputwillbeacquirefromstdin.positionalarguments:CSVcsvfilestomergeinxlsx;if-,stdinisassumedoptionalarguments:-h,--helpshowthishelpmessageandexit-dDELIMITER,--delimiterDELIMITERselectdelimitercharacter-oOUTFILE,--outOUTFILEoutputxlsxfilename
mail.py
send mail utility and function API
usage:mail.py[-h][-aATTACHMENT][-oFILE]-sSUBJECT[-bBODY][-fMAILFROM][-cCC]-tRECIPIENTSendthecontentsofadirectoryasaMIMEmessage.Unlessthe-ooptionisgiven,theemailissentbyforwardingtoyourlocalSMTPserver,whichthendoesthenormaldeliveryprocess.YourlocalmachinemustberunninganSMTPserver.optionalarguments:-h,--helpshowthishelpmessageandexit-aATTACHMENT,--attachATTACHMENTMailthecontentsofthespecifieddirectoryorfile,Onlytheregularfilesinthedirectoryaresent,andwedon't recurse to subdirectories.-oFILE,--outputFILEPrintthecomposedmessagetoFILEinsteadofsendingthemessagetotheSMTPserver.-sSUBJECT,--subjectSUBJECTSubjectforemailmessage(required).-bBODY,--bodyBODYBobytextforthemessage(optional).-fMAILFROM,--mailfromMAILFROMThevalueoftheFrom:header(optional);ifnotprovided$USER@$HOSTNAMEwillbeuseassender-cCC,--maliccCCThevalueoftheCC:header(optional)-tRECIPIENT,--mailtoRECIPIENTATo:headervalue(atleastonerequired)
prettyxml.py
Reformat XML in hierarchical structure.
usage:pretty-xml.py[-h][-oOUTFILE][XML[XML...]]PrettyprintsXMLfilethatisnotpretty.positionalarguments:XMLXMLfilestoprettyprint;if-ornoneprovided,stdinisassumedoptionalarguments:-h,--helpshowthishelpmessageandexit-oOUTFILE,--outOUTFILEoutputfilename;defaultstostdout
sshcmd
Runs single shh command on remote host
defsshcmd(cmd,host,password,)Args:cmd:commandtoexecutehost:remotehosttorunonpassword:user's password on remote host
touch
UNIX like touch with ability to create missing folders.
touch(path,times=None,dirs=False)Args:path:totouchtimes:a2-tupleoftheform(atime,mtime)whereeachmemberisanintorfloatexpressingseconds.defaultstocurrenttime.dirs:ifset,createmissingfolders
mrun
Runs UNIX command on multiple directories.
usage:mrun.py[-h][--cwd[DIR[DIR...]]][--exceptionTAG][--nostop][--verbose]...Runcommandinmultipledirectories.Example:mrun--cwddir1dir2--gitadd..positionalarguments:cmdcommandtorun.optionalarguments:-h,--helpshowthishelpmessageandexit--cwd[DIR[DIR...]]pathwherecommandshouldcdto;orfilethatcongainglistofdirectoriestooperateon.--exceptionTAGtagexceptionmessage.--nostopcontinueeveniffailedtoruninoneplace.--verbose,-vprintmessagesasitgoes.
Misc
camel2snake and snake2camel
camel2snake(name) and snake2camel(name) will convert name from camel to snake and from snake to camel respectively.
xlsx2rst
xlsx2rst is a utility and function to convert xlsx to restructuredtext.
usage:xlsx2rst.py[-h][-oRST][-s[SHEET[SHEET...]]][--start-row[NUMBER]][--end-row[NUMBER]][--start-col[NUMBER]][--end-col[NUMBER]][-r[NUMBER]][--one-file]XLSXConvertsxlsxworkbookintorestructuredtextformatpositionalarguments:XLSXxlsxfilestoconvertoptionalarguments:-h,--helpshowthishelpmessageandexit-oRST,--outputRSTdestinationrstfile-s[SHEET[SHEET...]],--sheet[SHEET[SHEET...]]listofsheets;defaulttoallavailablesheets--start-row[NUMBER]tablestartrow,defaultsto1--end-row[NUMBER]tablestartcol,defaultsto1--start-col[NUMBER]tablestartrow,defaultsto0--end-col[NUMBER]tablestartcol,defaultsto0-r[NUMBER],--header[NUMBER]headerrowcount--one-filewhenset,singlefileiscreated
Change History
Version 2.3
- Improvement in how threaded passes result.
- Add xlsx2rst utility.
- Fix bug with MpLogger multiprocessing queue (changed to use Manager().)
Version 2.2
- MpLogger was change to have single log instead of two (error and debug).
- MpLogger add new arguments: name, console, force_global, etc.
Version 3.0
- MpLogger moved to acrilog project
- Some functions moved to acrilib project
- Added mrun for execute command on multiple directories (for git operations)
欢迎加入QQ群-->： 979659372
