TopMind ETL helper
etl-helper-talenttech的Python项目详细描述
ETL表转换器和创建者
用于将表从一个数据库转换到另一个数据库的Python工具包
数据库类型
支持的数据库。你必须使用数据库的短名称 短_名称：类型数据库
- ch：点击屋
- pg：postgresql
- 在mysql：mysql在
- 在眩晕：眩晕在
- 在exasol:埃克森在
支持的数据库转换
- mysql到pg
- mysql到vertica
- mysql到exasol
- ch到vertica
- pg到vertica
证书
sql_credentials={"pg": {"database": os.getenv("PG_DATABASE"), "schema": os.getenv("PG_SCHEMA"), "user": os.getenv("PG_USER"), "host": os.getenv("PG_HOST"), "port": os.getenv("PG_PORT"), "password": os.getenv("PG_PASSWORD"), #additional params bellow (optional) "executemany_mode": "values", "executemany_values_page_size": 10000, "executemany_batch_page_size": 500, }, "ch": {"database": os.getenv("CH_DATABASE"), "user": os.getenv("CH_USER"), "host": os.getenv("CH_HOST"), "port": os.getenv("CH_PORT"), "password": os.getenv("CH_PASSWORD"), #additional params bellow (optional) "connect_args": {"alt_hosts": "{},{}:{}".format( os.getenv("CH_HOST_B"), os.getenv("CH_HOST_C"), os.getenv("CH_PORT"))}, }, "vertica": {"database": os.getenv("VERTICA_DATABASE"), "schema": os.getenv("VERTICA_SCHEMA"), "user": os.getenv("VERTICA_USER"), "host": os.getenv("VERTICA_HOST"), "port": os.getenv("VERTICA_PORT"), "password": os.getenv("VERTICA_PASSWORD"), #additional params bellow (optional) "connect_args": {"connection_load_balance": True, "backup_server_node": json.loads(os.getenv("VERTICA_CONFIGS"))["backup_server_node"], }, }, "mysql": {"database": os.getenv("MYSQL_DATABASE"), "user": os.getenv("MYSQL_USER"), "host": os.getenv("MYSQL_HOST"), "port": os.getenv("MYSQL_PORT"), "password": os.getenv("MYSQL_PASSWORD"), }, }
使用
^{pr2}$importosfromconverter.fields_converterimportFieldsConverterfrom_db="mysql"to_db="vertica"tables=["users","user_to_mc"]converter=FieldsConverter(sql_credentials,from_db,to_db,tables=tables,advanced_features=[0,1])res=converter.create_list_of_tables(tables,to_create=True,dir_ddl=None)fields_from=converter.db_worker_from.get_columns(tables[0])fields_to=converter.db_worker_to.get_columns(tables[0])print(fields_from)print(fields_to)
- 项目
标签：
欢迎加入QQ群-->： 979659372
