R: Implicit Stochastic Gradient Descent Optimizer

opt_isgd {bamlss} | R Documentation

## Implicit Stochastic Gradient Descent Optimizer

### Description

This optimizer performs an implicit stochastic gradient descent algorithm. It is mainly used within a `bamlss` call.

### Usage

    opt_isgd(x, y, family, weights = NULL, offset = NULL,
         gammaFun = function(i) 1/(1 + i), shuffle = TRUE,
         CFun = function(beta) diag(length(beta)),
         start = NULL, i.state = 0)

### Arguments

`x` | For function `boost()` the `x` list, as returned from function `bamlss.frame`, holding all model matrices and other information that is used for fitting the model.
`y` | The model response, as returned from function `bamlss.frame`.
`family` | A bamlss family object, see `family.bamlss`.
`weights` | Prior weights on the data, as returned from function `bamlss.frame`.
`offset` | Can be used to supply model offsets for use in fitting, returned from function `bamlss.frame`.
`gammaFun` | Function specifying the step length.
`shuffle` | Should the data be shuffled?
`CFun` | Hessian approximating function.
`start` | Vector of starting values.
`i.state` | Added to `gammaFUN()`.

### Details

tpf

### Value

For function `opt_isgd()` a list containing the following objects:

`fitted.values` | A named list of the fitted values based on the last iteration of the modeled parameters of the selected distribution.
`parameters` | A matrix, each row corresponds to the parameter values of one iteration.
`sgd.summary` | The summary of the stochastic gradient descent algorithm which can be printed and plotted.

### Warning

CAUTION: Arguments `weights` and `offset` are not implemented yet!

### Note

Motivated by the lecture 'Regression modelling with large data sets' given by Ioannis Kosmidis in Innsbruck, January 2017.

### Author(s)

Thorsten Simon

### References

Toulis, P and Airoldi, EM (2015): Scalable estimation strategies based on stochastic approximations: Classical results and new insights. Statistics and Computing, 25, no. 4, 781–795. doi: 10.1007/s11222-015-9560-y

### See Also

`bamlss.frame`, `bamlss`

### Examples

    ## Not run:
    set.seed(111)
    d <- GAMart(n = 10000)
    f <- num ~ s(x1) + s(x2) + s(x3) + te(lon, lat)
    b <- bamlss(f, data = d, optimizer = opt_isgd, sampler = FALSE)
    plot(b, ask = F)

    ## loop over observations a 2nd time
    b <- bamlss(f, data = d, optimizer = opt_isgd, sampler = FALSE, start = parameters(b),
                i.state = b$model.stats$optimizer$sgd.summary$i.state)
    plot(b, ask = F)

    ## try differeent gammaFuns, e.g.,
    # gammaFun <- function(i) .3/sqrt((1+i)) + 0.001

    ## testing some families
    f2 <- bin ~ s(x1) + s(x2) + s(x3) + te(lon, lat)
    b2 <- bamlss(f2, data = d, optimizer = opt_isgd, sampler = FALSE, family = "binomial")

    f3 <- cens ~ s(x1) + s(x2) + s(x3) + te(lon, lat)
    b3 <- bamlss(f3, data = d, optimizer = opt_isgd, sampler = FALSE, family = "cnorm")

    ## End(Not run)

[Package bamlss version 1.1-7 Index]