Whither the “bet on sparsity principle” in a nonsparse world?

Rob Tibshirani writes: Hastie et al. (2001) coined the informal “Bet on Sparsity” principle. The l1 methods assume that the truth is sparse, in some basis. If the assumption holds true, then the parameters can be efficiently estimated using l1 penalties. If the assumption does not hold—so that the truth is dense—then no method will … Continue reading Whither the “bet on sparsity principle” in a nonsparse world?