Classification (classification
)¶
Logistic Regression¶

class
Orange.classification.
LogisticRegressionLearner
(penalty='l2', dual=False, tol=0.0001, C=1.0, fit_intercept=True, intercept_scaling=1, class_weight=None, random_state=None, solver='auto', max_iter=100, multi_class='auto', verbose=0, n_jobs=1, preprocessors=None)[source]¶ A wrapper for sklearn.linear_model._logistic.LogisticRegression. The following is its documentation:
Logistic Regression (aka logit, MaxEnt) classifier.
In the multiclass case, the training algorithm uses the onevsrest (OvR) scheme if the ‘multi_class’ option is set to ‘ovr’, and uses the crossentropy loss if the ‘multi_class’ option is set to ‘multinomial’. (Currently the ‘multinomial’ option is supported only by the ‘lbfgs’, ‘sag’, ‘saga’ and ‘newtoncg’ solvers.)
This class implements regularized logistic regression using the ‘liblinear’ library, ‘newtoncg’, ‘sag’, ‘saga’ and ‘lbfgs’ solvers. Note that regularization is applied by default. It can handle both dense and sparse input. Use Cordered arrays or CSR matrices containing 64bit floats for optimal performance; any other input format will be converted (and copied).
The ‘newtoncg’, ‘sag’, and ‘lbfgs’ solvers support only L2 regularization with primal formulation, or no regularization. The ‘liblinear’ solver supports both L1 and L2 regularization, with a dual formulation only for the L2 penalty. The ElasticNet regularization is only supported by the ‘saga’ solver.
Read more in the User Guide.
Random Forest¶

class
Orange.classification.
RandomForestLearner
(n_estimators=10, criterion='gini', max_depth=None, min_samples_split=2, min_samples_leaf=1, min_weight_fraction_leaf=0.0, max_features='auto', max_leaf_nodes=None, bootstrap=True, oob_score=False, n_jobs=1, random_state=None, verbose=0, class_weight=None, preprocessors=None)[source]¶ A wrapper for sklearn.ensemble._forest.RandomForestClassifier. The following is its documentation:
A random forest classifier.
A random forest is a meta estimator that fits a number of decision tree classifiers on various subsamples of the dataset and uses averaging to improve the predictive accuracy and control overfitting. The subsample size is controlled with the max_samples parameter if bootstrap=True (default), otherwise the whole dataset is used to build each tree.
Read more in the User Guide.
Simple Random Forest¶

class
Orange.classification.
SimpleRandomForestLearner
(n_estimators=10, min_instances=2, max_depth=1024, max_majority=1.0, skip_prob='sqrt', seed=42)[source]¶ A random forest classifier, optimized for speed. Trees in the forest are constructed with
SimpleTreeLearner
classification trees.Parameters:  n_estimators (int, optional (default = 10)) – Number of trees in the forest.
 min_instances (int, optional (default = 2)) – Minimal number of data instances in leaves. When growing the three, new nodes are not introduced if they would result in leaves with fewer instances than min_instances. Instance count is weighed.
 max_depth (int, optional (default = 1024)) – Maximal depth of tree.
 max_majority (float, optional (default = 1.0)) – Maximal proportion of majority class. When this is exceeded, induction stops (only used for classification).
 skip_prob (string, optional (default = "sqrt")) –
Data attribute will be skipped with probability
skip_prob
. if float, then skip attribute with this probability.
 if “sqrt”, then skip_prob = 1  sqrt(n_features) / n_features
 if “log2”, then skip_prob = 1  log2(n_features) / n_features
 seed (int, optional (default = 42)) – Random seed.
Softmax Regression¶

class
Orange.classification.
SoftmaxRegressionLearner
(lambda_=1.0, preprocessors=None, **fmin_args)[source]¶ L2 regularized softmax regression classifier. Uses the LBFGS algorithm to minimize the categorical cross entropy cost with L2 regularization. This model is suitable when dealing with a multiclass classification problem.
When using this learner you should:
 choose a suitable regularization parameter lambda_,
 consider using many logistic regression models (one for each value of the class variable) instead of softmax regression.
Parameters:  lambda_ (float, optional (default=1.0)) – Regularization parameter. It controls tradeoff between fitting the data and keeping parameters small. Higher values of lambda_ force parameters to be smaller.
 preprocessors (list, optional) –
Preprocessors are applied to data before training or testing. Default preprocessors: [RemoveNaNClasses(), RemoveNaNColumns(), Impute(), Continuize(), Normalize()]
 remove columns with all values as NaN
 replace NaN values with suitable values
 continuize all discrete attributes,
 transform the dataset so that the columns are on a similar scale,
 fmin_args (dict, optional) – Parameters for LBFGS algorithm.
kNearest Neighbors¶

class
Orange.classification.
KNNLearner
(n_neighbors=5, metric='euclidean', weights='uniform', algorithm='auto', metric_params=None, preprocessors=None)[source]¶ A wrapper for sklearn.neighbors._classification.KNeighborsClassifier. The following is its documentation:
Classifier implementing the knearest neighbors vote.
Read more in the User Guide.
Naive Bayes¶

class
Orange.classification.
NaiveBayesLearner
(preprocessors=None)[source]¶ Naive Bayes classifier. Works only with discrete attributes. By default, continuous attributes are discretized.
Parameters: preprocessors (list, optional (default="[Orange.preprocess.Discretize]")) – An ordered list of preprocessors applied to data before training or testing.
The following code loads lenses dataset (four discrete attributes and discrete class), constructs naive Bayesian learner, uses it on the entire dataset to construct a classifier, and then applies classifier to the first three data instances:
>>> import Orange
>>> lenses = Orange.data.Table('lenses')
>>> nb = Orange.classification.NaiveBayesLearner()
>>> classifier = nb(lenses)
>>> classifier(lenses[0:3], True)
array([[ 0.04358755, 0.82671726, 0.12969519],
[ 0.17428279, 0.20342097, 0.62229625],
[ 0.18633359, 0.79518516, 0.01848125]])
For datasets that include continuous attributes,
Support Vector Machines¶

class
Orange.classification.
SVMLearner
(C=1.0, kernel='rbf', degree=3, gamma='auto', coef0=0.0, shrinking=True, probability=False, tol=0.001, cache_size=200, max_iter=1, preprocessors=None)[source]¶ A wrapper for sklearn.svm._classes.SVC. The following is its documentation:
CSupport Vector Classification.
The implementation is based on libsvm. The fit time scales at least quadratically with the number of samples and may be impractical beyond tens of thousands of samples. For large datasets consider using
sklearn.svm.LinearSVC
orsklearn.linear_model.SGDClassifier
instead, possibly after asklearn.kernel_approximation.Nystroem
transformer.The multiclass support is handled according to a onevsone scheme.
For details on the precise mathematical formulation of the provided kernel functions and how gamma, coef0 and degree affect each other, see the corresponding section in the narrative documentation: svm_kernels.
Read more in the User Guide.
Linear Support Vector Machines¶

class
Orange.classification.
LinearSVMLearner
(penalty='l2', loss='squared_hinge', dual=True, tol=0.0001, C=1.0, multi_class='ovr', fit_intercept=True, intercept_scaling=True, random_state=None, preprocessors=None)[source]¶ A wrapper for sklearn.svm._classes.LinearSVC. The following is its documentation:
Linear Support Vector Classification.
Similar to SVC with parameter kernel=’linear’, but implemented in terms of liblinear rather than libsvm, so it has more flexibility in the choice of penalties and loss functions and should scale better to large numbers of samples.
This class supports both dense and sparse input and the multiclass support is handled according to a onevstherest scheme.
Read more in the User Guide.
NuSupport Vector Machines¶

class
Orange.classification.
NuSVMLearner
(nu=0.5, kernel='rbf', degree=3, gamma='auto', coef0=0.0, shrinking=True, probability=False, tol=0.001, cache_size=200, max_iter=1, preprocessors=None)[source]¶ A wrapper for sklearn.svm._classes.NuSVC. The following is its documentation:
NuSupport Vector Classification.
Similar to SVC but uses a parameter to control the number of support vectors.
The implementation is based on libsvm.
Read more in the User Guide.
Classification Tree¶
Orange includes three implemenations of classification trees. TreeLearner is homegrown and properly handles multinominal and missing values. The one from scikitlearn, SklTreeLearner, is faster. Another homegrown, SimpleTreeLearner, is simpler and stil faster.

class
Orange.classification.
TreeLearner
(*args, binarize=False, max_depth=None, min_samples_leaf=1, min_samples_split=2, sufficient_majority=0.95, preprocessors=None, **kwargs)[source]¶ Tree inducer with proper handling of nominal attributes and binarization.
The inducer can handle missing values of attributes and target. For discrete attributes with more than two possible values, each value can get a separate branch (binarize=False), or values can be grouped into two groups (binarize=True, default).
The tree growth can be limited by the required number of instances for internal nodes and for leafs, the sufficient proportion of majority class, and by the maximal depth of the tree.
If the tree is not binary, it can contain zerobranches.
Parameters:  binarize (bool) – if True the inducer will find optimal split into two subsets for values of discrete attributes. If False (default), each value gets its branch.
 min_samples_leaf (float) – the minimal number of data instances in a leaf
 min_samples_split (float) – the minimal nubmer of data instances that is split into subgroups
 max_depth (int) – the maximal depth of the tree
 sufficient_majority (float) – a majority at which the data is not split further
Returns: instance of OrangeTreeModel
Simple Tree¶

class
Orange.classification.
SimpleTreeLearner
(min_instances=2, max_depth=1024, max_majority=1.0, skip_prob=0.0, bootstrap=False, seed=42)[source]¶ Classification or regression tree learner. Uses gain ratio for classification and mean square error for regression. This learner was developed to speedup random forest construction, but can also be used as a standalone tree learner.
 min_instances : int, optional (default = 2)
 Minimal number of data instances in leaves. When growing the three, new nodes are not introduced if they would result in leaves with fewer instances than min_instances. Instance count is weighed.
 max_depth : int, optional (default = 1024)
 Maximal depth of tree.
 max_majority : float, optional (default = 1.0)
 Maximal proportion of majority class. When this is exceeded, induction stops (only used for classification).
 skip_prob : string, optional (default = 0.0)
Data attribute will be skipped with probability
skip_prob
. if float, then skip attribute with this probability.
 if “sqrt”, then skip_prob = 1  sqrt(n_features) / n_features
 if “log2”, then skip_prob = 1  log2(n_features) / n_features
 bootstrap : data table, optional (default = False)
 A bootstrap dataset.
 seed : int, optional (default = 42)
 Random seed.
Majority Classifier¶

class
Orange.classification.
MajorityLearner
(preprocessors=None)[source]¶ A majority classifier. Always returns most frequent class from the training set, regardless of the attribute values from the test data instance. Returns class value distribution if class probabilities are requested. Can be used as a baseline when comparing classifiers.
In the special case of uniform class distribution within the training data, class value is selected randomly. In order to produce consistent results on the same dataset, this value is selected based on hash of the class vector.
Neural Network¶

class
Orange.classification.
NNClassificationLearner
(hidden_layer_sizes=(100, ), activation='relu', solver='adam', alpha=0.0001, batch_size='auto', learning_rate='constant', learning_rate_init=0.001, power_t=0.5, max_iter=200, shuffle=True, random_state=None, tol=0.0001, verbose=False, warm_start=False, momentum=0.9, nesterovs_momentum=True, early_stopping=False, validation_fraction=0.1, beta_1=0.9, beta_2=0.999, epsilon=1e08, preprocessors=None)[source]¶ A wrapper for Orange.classification.neural_network.MLPClassifierWCallback. The following is its documentation:
Multilayer Perceptron classifier.
This model optimizes the logloss function using LBFGS or stochastic gradient descent.
New in version 0.18.
CN2 Rule Induction¶
Induction of rules works by finding a rule that covers some learning instances, removing these instances, and repeating this until all instances are covered. Rules are scored by heuristics such as impurity of class distribution of covered instances. The module includes common rulelearning algorithms, and allows for replacing rule search strategies, scoring and other components.

class
Orange.classification.rules.
CN2Learner
(preprocessors=None, base_rules=None)[source]¶ Classic CN2 inducer that constructs a list of ordered rules. To evaluate found hypotheses, entropy measure is used. Returns a CN2Classifier if called with data.
References
“The CN2 Induction Algorithm”, Peter Clark and Tim Niblett, Machine Learning Journal, 3 (4), pp261283, (1989)

class
Orange.classification.rules.
CN2UnorderedLearner
(preprocessors=None, base_rules=None)[source]¶ Construct a set of unordered rules.
Rules are learnt for each class individually and scored by the relative frequency of the class corrected by the Laplace correction. After adding a rule, only the covered examples of that class are removed.
The code below loads the iris dataset (four continuous attributes and a discrete class) and fits the learner.
import Orange data = Orange.data.Table("iris") learner = Orange.classification.CN2UnorderedLearner() # consider up to 10 solution streams at one time learner.rule_finder.search_algorithm.beam_width = 10 # continuous value space is constrained to reduce computation time learner.rule_finder.search_strategy.constrain_continuous = True # found rules must cover at least 15 examples learner.rule_finder.general_validator.min_covered_examples = 15 # found rules may combine at most 2 selectors (conditions) learner.rule_finder.general_validator.max_rule_length = 2 classifier = learner(data)
References
“Rule Induction with CN2: Some Recent Improvements”, Peter Clark and Robin Boswell, Machine Learning  Proceedings of the 5th European Conference (EWSL91), pp151163, 1991

class
Orange.classification.rules.
CN2SDLearner
(preprocessors=None, base_rules=None)[source]¶ Ordered CN2SD inducer that constructs a list of ordered rules. To evaluate found hypotheses, Weighted relative accuracy measure is used. Returns a CN2SDClassifier if called with data.
In this setting, ordered rule induction refers exclusively to finding best rule conditions and assigning the majority class in the rule head (target class is set to None). To later predict instances, rules will be regarded as unordered.
Notes
A weighted covering algorithm is applied, in which subsequently induced rules also represent interesting and sufficiently large subgroups of the population. Covered positive examples are not deleted from the learning set, rather their weight is reduced.
The algorithm demonstrates how classification rule learning (predictive induction) can be adapted to subgroup discovery, a task at the intersection of predictive and descriptive induction.
References
“Subgroup Discovery with CN2SD”, Nada Lavrač et al., Journal of Machine Learning Research 5 (2004), 153188, 2004

class
Orange.classification.rules.
CN2SDUnorderedLearner
(preprocessors=None, base_rules=None)[source]¶ Unordered CN2SD inducer that constructs a set of unordered rules. To evaluate found hypotheses, Weighted relative accuracy measure is used. Returns a CN2SDUnorderedClassifier if called with data.
Notes
A weighted covering algorithm is applied, in which subsequently induced rules also represent interesting and sufficiently large subgroups of the population. Covered positive examples are not deleted from the learning set, rather their weight is reduced.
The algorithm demonstrates how classification rule learning (predictive induction) can be adapted to subgroup discovery, a task at the intersection of predictive and descriptive induction.
References
“Subgroup Discovery with CN2SD”, Nada Lavrač et al., Journal of Machine Learning Research 5 (2004), 153188, 2004
Calibration and threshold optimization¶

class
Orange.classification.calibration.
ThresholdClassifier
(base_model, threshold)[source]¶ A model that wraps a binary model and sets a different threshold.
The target class is the class with index 1. A data instances is classified to class 1 it the probability of this class equals or exceeds the threshold

base_model
¶ base mode
Type: Orange.classification.Model


class
Orange.classification.calibration.
ThresholdLearner
(base_learner, threshold_criterion=0)[source]¶ A learner that runs another learner and then finds the optimal threshold for CA or F1 on the training data.

base_leaner
¶ base learner
Type: Learner


class
Orange.classification.calibration.
CalibratedClassifier
(base_model, calibrators)[source]¶ A model that wraps another model and recalibrates probabilities

base_model
¶ base mode
Type: Mode

calibrators
¶ list of functions that get a vector of probabilities and return calibrated probabilities
Type: list of callable


class
Orange.classification.calibration.
CalibratedLearner
(base_learner, calibration_method=0)[source]¶ Probability calibration for learning algorithms
This learner that wraps another learner, so that after training, it predicts the probabilities on training data and calibrates them using sigmoid or isotonic calibration. It then returns a
CalibratedClassifier
.
base_learner
¶ base learner
Type: Learner
