last updated : 5 Nov 2013
created      : 5 Nov 2013

DESCRIPTION:
------------

This code implements the Variational Garrote algorithm [1] for sparse linear
regresion, a mean field variational approximation for the Spike and Slab model.

USAGE:
------

In MATLAB

>> model = train_VG(X, Y, valp)

where

X       : n x p matrix (n number of features, p number of samples)
Y       : p x 1 vector (outputs)
valp    : proportion of dataset used for validation (default 0.1)

The function is a wrapper for the vg_learn.m script.  It will learn the model
parameters using (1-valp) percent of the p samples in X and choose the sparsity
hyper-parameter (gamma) using valp percent of the p samples in X.

After execution, the output will contain the predictive model together with
other fields:

    model.m :       expected activation values, 1 x n
    model.v :       weight vector, 1 x n
    model.beta :    inverse noise
    model.gamma :   sparsity hyper-parameter 
    model.x :       input training dataset (after normalization)
    model.xv :      input validation dataset (after normalization)
    model.y :       output training dataset (after normalization)
    model.yv :      output validation dataset (after normalization)

Check the script code for additional parameters such as

    plotfigures = true;     % plot results?
    METHOD = 'dual';        % method 'dual' or 'regression'

These other parameters should not be changed in general: 

    epsilon = 1e-4;         % determines gamma_min (smallest m)
    ng = 50;                % number of gammas (equiv step-size)
    eps_gamma = 1e-4;       % determines gamma_max
    max_sum_m = n;          % increases gamma until sum(m) == max_sum_m
    beta_max = sigmay;      % increases gamma until beta == beta_max
    ninits_vg = 1;          % number of random initializations
    dmmin = 1e-12;          % convergence criterion for mean field eqs
    eta0 = 1;               % initial step size of mean field update
    eta_w0 = 0.02;          % initial step size for change in w in dual.m
    maxiter = 1e4;          % maximal number of iterations 

For running the VG using toy data, type:

>> train_VG([])

To see the parameters used for the toy data generation

>> help make_toy_data.m script 

If executed using toy data, the output struct will contain additional values:
    
    model.l1err :       l1 norm of the difference between ground truth w and v
    model.auc_v_mf :    area under the curve using the solution v


REQUIREMENTS:
-------------

- It runs on MATLAB R2012a

-------------------------------------------------------------
Vicenç Gomez,
v.gomez@science.ru.nl


REFERENCES:
-----------

[1] "The Variational Garrote", H. J. Kappen, V. Gómez,
    Machine Learning (to appear), 2013
