pyrfm.linear_model.DoublySGDClassifier

class pyrfm.linear_model.DoublySGDClassifier(transformer=RBFSampler(gamma=1.0, n_components=100, random_state=None), eta0=0.1, loss='squared_hinge', C=1.0, alpha=0.001, l1_ratio=0.0, intercept_decay=0.0001, fit_intercept=True, max_iter=100, batch_size=10, n_bases_sampled=1, tol=1e-06, learning_rate='optimal', power_t=1, eta1=0.0001, warm_start=False, random_state=None, verbose=True, shuffle=True)[source]

Bases: pyrfm.linear_model.doubly_sgd.BaseDoublySGDEstimator, pyrfm.linear_model.base.LinearClassifierMixin

Doubly SGD solver for linear classifier with random feature maps.

Random feature mapping is computed just before computing prediction and gradient. minimize sum_{i=1}^{n} loss(x_i, y_i) + alpha/C*reg

Parameters

  • transformer (scikit-learn Transformer object (default=RBFSampler())) – A scikit-learn TransformerMixin object. transformer must have (1) n_components attribute, (2) fit(X, y), and (3) transform(X).

  • eta0 (double (default=0.1)) – Step-size parameter.

  • loss (str (default="squared_hinge")) –

    Which loss function to use. Following losses can be used:

    • ’squared_hinge’ (for classification)

    • ’hinge’ (for classification)

    • ’logistic’ (for classification)

  • C (double (default=1.0)) – Weight of the loss term.

  • alpha (double (default=1e-3)) – Weight of the penalty term.

  • l1_ratio (double (default=0)) –

    Ratio of L1 regularizer. Weight of L1 regularizer is alpha * l1_ratio and that of L2 regularizer is 0.5 * alpha * (1-l1_ratio).

    • l1_ratio = 0 : Ridge.

    • l1_ratio = 1 : Lasso.

    • Otherwise : Elastic Net.

  • intercept_decay (double (default=1e-4)) – Weight of the penalty term for intercept.

  • fit_intercept (bool (default=True)) – Whether to fit intercept (bias term) or not.

  • max_iter (int (default=100)) – Maximum number of iterations.

  • batch_size (int (default=10)) – Number of samples in one batch.

  • n_bases_sampled (int (default=1)) – Number of new random bases (called “number of feature blocks” in original paper).

  • tol (double (default=1e-6)) – Tolerance of stopping criterion. If sum of absolute val of update in one epoch is lower than tol, the SGD solver stops learning.

  • learning_rate (str (default='optimal')) –

    The method for learning rate decay.

    • ’constant’: eta = eta0

    • ’pegasos’: eta = 1.0 / (alpha * (1-l1_ratio) * t)

    • ’inv_scaling’: eta = eta0 / pow(t, power_t)

    • ’optimal’: eta = eta0 / pow(1 + eta0*alpha*(1-l1_ratio)*t, power_t)

    • ’original’: eta = eta0 / (1 + eta1*t)

  • power_t (double (default=0.75)) – The parameter for learning_rate ‘inv_scaling’ and ‘optimal’.

  • eta1 (double (default=1e-4)) – The parameter for learning_rate ‘original’.

  • warm_start (bool (default=False)) – Whether to activate warm-start or not.

  • random_state (int, RandomState instance or None, optional (default=None)) – If int, random_state is the seed used by the random number generator; If RandomState instance, random_state is the random number generator; If None, the random number generator is the RandomState instance used by np.random.

  • verbose (bool (default=True)) – Verbose mode or not.

  • fast_solver (bool (default=True)) – Use cython fast solver or not. This argument is valid when transformer is implemented in random_features_fast.pyx/pxd

  • shuffle (bool (default=True)) – Whether to shuffle data before each epoch or not.

self.coef_

The learned coefficients of the linear model.

Type

array, shape (n_components, )

self.intercept_

The learned intercept (bias) of the linear model.

Type

array, shape (1, )

self.t_

The number of iteration.

Type

int

self.transformer_doubly_

The cdef object of learned transformer.

Type

BaseCDoublyRandomFeature

References

[1] Scalable Kernel Methods via Doubly Stochastic Gradients Bo Dai, Bo Xie, Niao He, Yingyu Liang, Anant Raj, Maria-Flornia Balcan, and Le Song. In Proc. NIPS 2014. (https://papers.nips.cc/paper/5238-scalable-kernel-methods-via-doubly-stochastic-gradients.pdf)

[2] Large-Scale Machine Learning with Stochastic Gradient Descent. Leon Bottou. In Proc. COMPSTAT’2010. (https://leon.bottou.org/publications/pdf/compstat-2010.pdf)

[3] Stochastic Gradient Descent Tricks. Leon Bottou. Neural Networks, Tricks of the Trade, Reloaded, 430–445, Lecture Notes in Computer Science (LNCS 7700), Springer, 2012 (https://link.springer.com/content/pdf/10.1007%2F978-3-642-35289-8_25.pdf)

LEARNING_RATE = {'constant': 0, 'inv_scaling': 2, 'optimal': 3, 'original': 4, 'pegasos': 1}
LOSSES = {'hinge': <pyrfm.linear_model.loss_fast.Hinge object>, 'log': <pyrfm.linear_model.loss_fast.Logistic object>, 'logistic': <pyrfm.linear_model.loss_fast.Logistic object>, 'squared_hinge': <pyrfm.linear_model.loss_fast.SquaredHinge object>}
decision_function(X)
fit(X, y)

Fit model according to X and y.

Parameters

  • X (array-like, shape = [n_samples, n_features]) – Training vectors, where n_samples is the number of samples and n_features is the number of features.

  • y (array-like, shape = [n_samples]) – Target values.

Returns

self – Returns self.

Return type

classifier

get_params(deep=True)

Get parameters for this estimator.

Parameters

deep (boolean, optional) – If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns

params – Parameter names mapped to their values.

Return type

mapping of string to any

predict(X)

Perform classification on an array of test vectors X.

Parameters

X (array-like, shape = [n_samples, n_features]) –

Returns

Predicted target values for X

Return type

array, shape = [n_samples]

predict_proba(X)
score(X, y, sample_weight=None)

Returns the mean accuracy on the given test data and labels.

In multi-label classification, this is the subset accuracy which is a harsh metric since you require for each sample that each label set be correctly predicted.

Parameters

  • X (array-like, shape = (n_samples, n_features)) – Test samples.

  • y (array-like, shape = (n_samples) or (n_samples, n_outputs)) – True labels for X.

  • sample_weight (array-like, shape = [n_samples], optional) – Sample weights.

Returns

score – Mean accuracy of self.predict(X) wrt. y.

Return type

float

set_params(**params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as pipelines). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Returns

Return type

self

stochastic = True