group-lasso 1.0.0

关于本项目

该项目由YNGVE Mardal Moe开发，由麻省理工学院发布 李森斯。我还在做一些事情，这样变化可能会很快到来。

安装指南

群套索需要python 3.5+，numpy和scikit学习。 要通过pip安装群lasso，只需运行命令：

pip install group-lasso

或者，您可以手动拉取此存储库并运行 setup.py文件：

git clone https://github.com/yngvem/group-lasso.git
cd group-lasso
python setup.py

文档

您可以阅读 readthedocs。

示例

importnumpyasnpfromgroup_lassoimportGroupLasso# Dataset parametersnum_data_points=10_000num_features=500num_groups=25assertnum_features%num_groups==0# Generate data matrixX=np.random.standard_normal((num_data_points,num_features))# Generate coefficients and interceptw=np.random.standard_normal((500,1))intercept=2# Generate groups and randomly set coefficients to zerogroups=np.array([[group]*20forgroupinrange(25)]).ravel()forgroupinrange(num_groups):w[groups==group]*=np.random.random()<0.8# Generate target vector:y=X@w+interceptnoise=np.random.standard_normal(y.shape)noise/=np.linalg.norm(noise)noise*=0.3*np.linalg.norm(y)y+=noise# Generate group lasso object and fit the modelgl=GroupLasso(groups=groups,reg=.05)gl.fit(X,y)estimated_w=gl.coef_estimated_intercept=gl.intercept_[0]# Evaluate the modelcoef_correlation=np.corrcoef(w.ravel(),estimated_w.ravel())[0,1]print("True intercept: {intercept:.2f}. Estimated intercept: {estimated_intercept:.2f}".format(intercept=intercept,estimated_intercept=estimated_intercept))print("Correlation between true and estimated coefficients: {coef_correlation:.2f}".format(coef_correlation=coef_correlation))

True intercept: 2.00. Estimated intercept: 1.53
Correlation between true and estimated coefficients: 0.98

importnumpyasnpfromsklearn.pipelineimportPipelinefromsklearn.linear_modelimportRidgefromgroup_lassoimportGroupLasso# Dataset parametersnum_data_points=10_000num_features=500num_groups=25assertnum_features%num_groups==0# Generate data matrixX=np.random.standard_normal((num_data_points,num_features))# Generate coefficients and interceptw=np.random.standard_normal((500,1))intercept=2# Generate groups and randomly set coefficients to zerogroups=np.array([[group]*20forgroupinrange(25)]).ravel()forgroupinrange(num_groups):w[groups==group]*=np.random.random()<0.8# Generate target vector:y=X@w+interceptnoise=np.random.standard_normal(y.shape)noise/=np.linalg.norm(noise)noise*=0.3*np.linalg.norm(y)y+=noise# Generate group lasso object and fit the model# We use an artificially high regularisation coefficient since#  we want to use group lasso as a variable selection algorithm.gl=GroupLasso(groups=groups,group_reg=0.1,l1_reg=0.05)gl.fit(X,y)new_X=gl.transform(X)# Evaluate the modelpredicted_y=gl.predict(X)R_squared=1-np.sum((y-predicted_y)**2)/np.sum(y**2)print("The rows with zero-valued coefficients have now been removed from the dataset.")print("The new shape is:",new_X.shape)print("The R^2 statistic for the group lasso model is: {R_squared:.2f}".format(R_squared=R_squared))print("This is very low since the regularisation is so high."# Use group lasso in a scikit-learn pipelinepipe=Pipeline(memory=None,steps=[('variable_selection',GroupLasso(groups=groups,reg=.1)),('regressor',Ridge(alpha=0.1))])pipe.fit(X,y)predicted_y=pipe.predict(X)R_squared=1-np.sum((y-predicted_y)**2)/np.sum(y**2)print("The R^2 statistic for the pipeline is: {R_squared:.2f}".format(R_squared=R_squared))

The rows with zero-valued coefficients have now been removed from the dataset.
The new shape is: (10000, 280)
The R^2 statistic for the group lasso model is: 0.17
This is very low since the regularisation is so high.
The R^2 statistic for the pipeline is: 0.72

折叠工作

待办事项按重要性降序排列

1. python 3.5兼容性

实施细节

使用基于梯度的fista优化程序[4]来解决这个问题。 自适应重启方案[5]。当前未实现行搜索，但 我希望以后再看。

尽管速度很快，但FISTA优化程序并没有达到 二阶内点法速度明显减慢。这可能，在 乍一看，似乎是个问题。然而，它确实恢复了稀疏性 数据模式，可用于训练具有给定 特征的子集。

另外，即使fista乐观者不是随机的 乐观主义，我的经验是没有遭受过大的挫折 当小批量足够大时的性能。因此我有 使用fista实现了小批量优化，因此能够 基于大约500列和10000行数据的模型 价格昂贵的笔记本电脑。

最后，我们注意到由于fista使用nesterov加速度，因此 下降算法。因此，我们不能指望损失会减少 单调地。

参考文献