pyrfm.linear_model.SAGAClassifier¶

class pyrfm.linear_model.SAGAClassifier(transformer=RBFSampler(gamma=1.0, n_components=100, random_state=None), eta0=0.01, loss='squared_hinge', C=1.0, alpha=1.0, l1_ratio=0.0, intercept_decay=0.1, normalize=False, fit_intercept=True, max_iter=100, tol=1e-06, learning_rate='optimal', power_t=1.0, is_saga=False, warm_start=False, random_state=None, verbose=True, fast_solver=True, shuffle=True)[source]¶

Bases: pyrfm.linear_model.saga.BaseSAGAEstimator, pyrfm.linear_model.base.LinearClassifierMixin

SAGA solver for linear classifier with random feature maps.

Random feature mapping is computed just before computing prediction and gradient. minimize sum_{i=1}^{n} loss(x_i, y_i) + alpha/C*reg

Parameters

transformer (scikit-learn Transformer object (default=RBFSampler())) – A scikit-learn TransformerMixin object. transformer must have (1) n_components attribute, (2) fit(X, y), and (3) transform(X).
eta0 (double (default=0.01)) – Step-size parameter.
loss (str (default="squared_hinge")) –
Which loss function to use. Following losses can be used:
- ’squared_hinge’
- ’hinge’
- ’logistic’
C (double (default=1.0)) – Weight of the loss term.
alpha (double (default=1.0)) – Weight of the penalty term.
l1_ratio (double (default=0)) –
Ratio of L1 regularizer. Weight of L1 regularizer is alpha * l1_ratio and that of L2 regularizer is 0.5 * alpha * (1-l1_ratio).
- l1_ratio = 0 : Ridge.
- l1_ratio = 1 : Lasso.
- Otherwise : Elastic Net.
intercept_decay (double (default=0.1)) – Weight of the penalty term for intercept.
normalize (bool (default=False)) – Whether normalize random features or not. If true, the SAGA solver computes running mean and variance at learning, and uses them for inference.
fit_intercept (bool (default=True)) – Whether to fit intercept (bias term) or not.
max_iter (int (default=100)) – Maximum number of iterations.
tol (double (default=1e-6)) – Tolerance of stopping criterion. If sum of absolute val of update in one epoch is lower than tol, the SAGA solver stops learning.
learning_rate (str (default='optimal')) – The method for learning rate decay. {‘constant’|’pegasos’|’inv_scaling’ |’optimal’} are supported now.
power_t (double (default=1.)) – The parameter for learning_rate ‘inv_scaling’ and ‘optimal’.
is_saga (bool (default=True)) – Whether SAGA (True) or SAG (False).
warm_start (bool (default=False)) – Whether to activate warm-start or not.
random_state (int, RandomState instance or None, optional (default=None)) – If int, random_state is the seed used by the random number generator; If RandomState instance, random_state is the random number generator; If None, the random number generator is the RandomState instance used by np.random.
verbose (bool (default=True)) – Verbose mode or not.
fast_solver (bool (default=True)) – Use cython fast solver or not. This argument is valid when transformer is implemented in random_features_fast.pyx/pxd.
shuffle (bool (default=True)) – Whether to shuffle data before each epoch or not.

self.coef_¶

The learned coefficients of the linear model.

Type: array, shape (n_components, )

self.averaged_grad_coef_¶

The averaged gradient of coefficients.

Type: array, shape (n_components, )

self.intercept_¶

The learned intercept (bias) of the linear model.

Type: array, shape (1, )

self.averaged_grad_intercept_¶

The averaged gradient of intercept.

Type: array, shape (1, )

self.dloss_¶

The gradient of loss for each samples.

Type: array, shape (n_samples, )

self.mean_, self.var_

The running mean and variances of random feature vectors. They are used if normalize=True (they are None if False).

Type: array or None, shape (n_components, )

self.t_¶

The number of iteration.

Type: int

References

[1] SAGA: A Fast Incremental Gradient Method with Support for Non-Strongly Convex Composite Objectives. Aaron Defazo, Francis Bach, and Simon Lacoste-Julien. In Proc. NIPS 2014. (https://arxiv.org/pdf/1407.0202.pdf)

[2] Minimizing Finite Sums with the Stochastic Average Gradient. Mark Schmidt, Nicolas Le Roux, and Francis Bach. Mathematical Programming Vol 162, 2017. (https://arxiv.org/pdf/1309.2388.pdf)

LEARNING_RATE = {'constant': 0, 'inv_scaling': 2, 'optimal': 3, 'pegasos': 1}¶

LOSSES = {'hinge': <pyrfm.linear_model.loss_fast.Hinge object>, 'log': <pyrfm.linear_model.loss_fast.Logistic object>, 'logistic': <pyrfm.linear_model.loss_fast.Logistic object>, 'squared_hinge': <pyrfm.linear_model.loss_fast.SquaredHinge object>}¶

decision_function(X)¶

fit(X, y)¶

Fit model according to X and y.

Parameters

X (array-like, shape = [n_samples, n_features]) – Training vectors, where n_samples is the number of samples and n_features is the number of features.
y (array-like, shape = [n_samples]) – Target values.

Returns

self – Returns self.

Return type

classifier

get_params(deep=True)¶

Get parameters for this estimator.

Parameters: deep (boolean, optional) – If True, will return the parameters for this estimator and contained subobjects that are estimators.
Returns: params – Parameter names mapped to their values.
Return type: mapping of string to any

predict(X)¶

Perform classification on an array of test vectors X.

Parameters: X (array-like, shape = [n_samples, n_features]) –
Returns: Predicted target values for X
Return type: array, shape = [n_samples]

predict_proba(X)¶

score(X, y, sample_weight=None)¶

Returns the mean accuracy on the given test data and labels.

In multi-label classification, this is the subset accuracy which is a harsh metric since you require for each sample that each label set be correctly predicted.

Parameters

X (array-like, shape = (n_samples, n_features)) – Test samples.
y (array-like, shape = (n_samples) or (n_samples, n_outputs)) – True labels for X.
sample_weight (array-like, shape = [n_samples], optional) – Sample weights.

Returns

score – Mean accuracy of self.predict(X) wrt. y.

Return type

float

set_params(**params)¶

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as pipelines). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Returns
Return type: self

stochastic = True¶