.. note::
    :class: sphx-glr-download-link-note

    Click :ref:`here <sphx_glr_download_auto_examples_plot_reweighted.py>` to download the full example code
.. rst-class:: sphx-glr-example-title

.. _sphx_glr_auto_examples_plot_reweighted.py:


Adaptive / Reweighted Lasso
===========================

The adaptive or reweighted Lasso minimizes the non-convex L0.5 pseudo norm
which is a better sparsity proxy function for the Basis Pursuit L0 norm. In
practice, it promotes sparser coefficients with less amplitude bias. As with
`IndLasso`, `IndRewLasso` is independent across tasks.

`IndRewLasso` solves the optimization problem::


        (1 / (2 * n_samples)) * ||Y - XW||_Fro^2
        + sum_k alpha_k * ||W_k||_0.5




.. code-block:: default


    # Author: Hicham Janati (hicham.janati@inria.fr)
    #
    # License: BSD (3-clause)

    import numpy as np
    from matplotlib import pyplot as plt

    from mutar import IndLasso, IndRewLasso







Generate multi-task data



.. code-block:: default


    rng = np.random.RandomState(42)
    n_tasks, n_samples, n_features = 2, 20, 100
    X = rng.randn(n_tasks, n_samples, n_features)

    support = rng.rand(n_features, n_tasks) > 0.95
    coef = support * rng.randn(n_features, n_tasks) * 5

    y = np.array([x.dot(c) for x, c in zip(X, coef.T)])

    # add noise
    y += 0.5 * np.std(y) + rng.randn(n_tasks, n_samples)

    alpha = 0.05 * np.ones(n_tasks)






Fit the independent Lasso


.. code-block:: default


    lasso = IndLasso(alpha=alpha)
    lasso.fit(X, y)








Fit the independent reweighted Lasso


.. code-block:: default


    rewlasso = IndRewLasso(alpha=alpha)
    rewlasso.fit(X, y)








Plot the supports of the true and obtained coefficients.
Reweighting reduces the size of the active features and corrects
the amplitudes of the relevant coefficients


.. code-block:: default


    coefs = [coef, lasso.coef_, rewlasso.coef_]
    labels = ["Truth", "Lasso", "Re-Lasso"]
    colors = ["k", "b", "r"]
    lines = ["-", "--", "--"]
    f, axes = plt.subplots(1, 2, figsize=(12, 4))
    for c, ll, coefs, label in zip(colors, lines, coefs, labels):
        for i, (ax, coef) in enumerate(zip(axes, coefs.T)):
            # plot non-zero coefs only for clarity
            xx = np.where(coef)[0]
            yy = coef[xx]
            ax.stem(xx, yy, label=label, linefmt=c + ll,
                    markerfmt=c + 'x', basefmt=c + ll)
            ax.set_title("Coeffcient of task %d" % (i + 1))
    ax.legend(loc=2, bbox_to_anchor=[1.01, 0.75])
    plt.show()



.. image:: /auto_examples/images/sphx_glr_plot_reweighted_001.png
    :class: sphx-glr-single-img




**Estimated memory usage:**  10 MB


.. _sphx_glr_download_auto_examples_plot_reweighted.py:


.. only :: html

 .. container:: sphx-glr-footer
    :class: sphx-glr-footer-example



  .. container:: sphx-glr-download

     :download:`Download Python source code: plot_reweighted.py <plot_reweighted.py>`



  .. container:: sphx-glr-download

     :download:`Download Jupyter notebook: plot_reweighted.ipynb <plot_reweighted.ipynb>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_