我试图计算应用程序上用户名状态从原始安装/重新安装到卸载(反之亦然)之间的时间。我已经用Python编程大约一年了,所以我的函数有时较慢或不是最佳的;我在这项任务中遇到了类似的问题
我使用下面的循环来计算状态之间的时间差。主要的问题是这个函数速度非常慢(对于我正在处理的700k行数据),并且它不会返回所有的时间间隔
# Import modules
import pandas as pd
import numpy as np
# Load sample data
data = pd.read_csv('username_status_dates.csv').drop(columns='index')
data['Date'] = pd.to_datetime(data['Date'], format='%Y-%M-%d')
# Create dataframe to record time intervals
customerStatuses_df = pd.DataFrame(columns=['username', 'status1', 'status2', 'timeToUninstall', 'timeToReinstall'])
# Iterate over data, calculate time intervals (by username), and add to customerStatuses
for user in data:
# Get user sub data
user_df = data.loc[ customerStatuses_df['Username'] == user ]
dates = list(user_df['Date'])
status = list(user_df['Status'])
for val in np.arange(0, len(status) - 1):
# If status in installed (original or re)
if (status[val] == 'Original Install') or (status[val] == 'Reinstall'):
# If next status is uninstall (un)
if status[val + 1] == 'Uninstall':
# Calculate time interval between statuses
d_unin = dates[val + 1] - dates[val]
# Add to customerStatuses
customerStatuses_df = customerStatuses_df.append({'username': user, 'status1': status[val], 'status2': status[val+1], 'timeToUninstall': d_unin}, ignore_index=True)
# If status in not installed (un)
elif (status[val] == 'Uninstall'):
# If next status is installed (re)
if status[val + 1] == 'Reinstall':
# Calculate time interval between statuses
d_rein = dates[val + 1] - dates[val]
# Add to customerStatuses
customerStatuses_df = customerStatuses_df.append({'username': user, 'status1': status[val], 'status2': status[val+1], 'timeToReinstall': d_rein}, ignore_index=True)
示例数据:username\u status\u dates.csv
Username Status Date
0 Joey74 Original Install 2019-01-20 00:09:00
1 Sam22 Original Install 2019-01-15 00:10:00
2 Chloe35 Uninstall 2019-01-30 00:10:00
3 Sam22 Uninstall 2019-01-01 00:11:00
4 Zoe44 Reinstall 2019-01-10 00:12:00
5 Joey74 Reinstall 2019-01-20 00:12:00
6 Shelby99 Original Install 2020-01-07 00:01:00
7 Joey74 Uninstall 2020-01-05 00:01:00
8 Joey74 Reinstall 2020-01-30 00:01:00
9 Susane11 Reinstall 2020-01-04 00:02:00
函数的结果:客户状态
username status1 status2 timeToUninstall timeToReinstall
0 Joey74 Reinstall Uninstall 349 days 23:49:00 NaN
1 Joey74 Uninstall Reinstall NaT 25 days 00:00:00
2 Sam22 Original Install Uninstall -14 days +00:01:00 NaN
我错过了什么,在这里
我和一位同事意识到,这个循环的核心问题是它试图迭代数据帧本身,与其他梳理数据帧的方法相比,它的速度异常缓慢。这部分解释了为什么我的函数(上面)在使用更大的数据集时速度如此之慢。我们还发现使用矢量化数字比使用日期列的timedelta格式快得多
我们发现了一个很棒的StackOverflow source,它展示了如何将数据帧转换(或内爆)为字典,以取代对数据帧本身进行迭代的需要为了回答上述问题,我按用户名组织数据,对日期列进行矢量化,并将数据框转换为字典,然后计算状态更改之间的时间:
结果\u df
相关问题 更多 >
编程相关推荐