Variable Descriptors (variable)

Every variable is associated with a descriptor that stores its name and other properties. Descriptors serve three main purposes:

  • conversion of values from textual format (e.g. when reading files) to the internal representation and back (e.g. when writing files or printing out);
  • identification of variables: two variables from different datasets are considered to be the same if they have the same descriptor;
  • conversion of values between domains or datasets, for instance from continuous to discrete data, using a pre-computed transformation.

Descriptors are most often constructed when loading the data from files.

>>> from Orange.data import Table
>>> iris = Table("iris")

>>> iris.domain.class_var
DiscreteVariable('iris')
>>> iris.domain.class_var.values
['Iris-setosa', 'Iris-versicolor', 'Iris-virginica']

>>> iris.domain[0]
ContinuousVariable('sepal length')
>>> iris.domain[0].number_of_decimals
1

Some variables are derived from others. For instance, discretizing a continuous variable gives a new, discrete variable. The new variable can compute its values from the original one.

>>> from Orange.preprocess import DomainDiscretizer
>>> discretizer = DomainDiscretizer()
>>> d_iris = discretizer(iris)
>>> d_iris[0]
DiscreteVariable('D_sepal length')
>>> d_iris[0].values
['<5.2', '[5.2, 5.8)', '[5.8, 6.5)', '>=6.5']

See Derived variables for a detailed explanation.

Constructors

Orange maintains lists of existing descriptors for variables. This facilitates the reuse of descriptors: if two datasets refer to the same variables, they should be assigned the same descriptors so that, for instance, a model trained on one dataset can make predictions for the other.

Variable descriptors are seldom constructed in user scripts. When needed, this can be done by calling the constructor directly or by calling the class method make. The difference is that the latter returns an existing descriptor if there is one with the same name and which matches the other conditions, such as having the prescribed list of discrete values for DiscreteVariable:

>>> from Orange.data import ContinuousVariable
>>> age = ContinuousVariable.make("age")
>>> age1 = ContinuousVariable.make("age")
>>> age2 = ContinuousVariable("age")
>>> age is age1
True
>>> age is age2
False

The first line returns a new descriptor after not finding an existing desciptor for a continuous variable named “age”. The second reuses the first descriptor. The last creates a new one since the constructor is invoked directly.

The distinction does not matter in most cases, but it is important when loading the data from different files. Orange uses the make constructor when loading data.

Base class

Continuous variables

Discrete variables

String variables

Time variables

Time variables are continuous variables with value 0 on the Unix epoch, 1 January 1970 00:00:00.0 UTC. Positive numbers are dates beyond this date, and negative dates before. Due to limitation of Python datetime module, only dates in 1 A.D. or later are supported.

Derived variables

The compute_value mechanism is used throughout Orange to compute all preprocessing on training data and applying the same transformations to the testing data without hassle.

Method compute_value is usually invoked behind the scenes in conversion of domains. Such conversions are are typically implemented within the provided wrappers and cross-validation schemes.

Derived variables in Orange

Orange saves variable transformations into the domain as compute_value functions. If Orange was not using compute_value, we would have to manually transform the data:

>>> from Orange.data import Domain, ContinuousVariable
>>> data = Orange.data.Table("iris")
>>> train = data[::2]  # every second row
>>> test = data[1::2]  # every other second instance

We will create a new data set with a single feature, “petals”, that will be a sum of petal lengths and widths:

>>> petals = ContinuousVariable("petals")
>>> derived_train = train.transform(Domain([petals],
...                                 data.domain.class_vars))
>>> derived_train.X = train[:, "petal width"].X + \
...                   train[:, "petal length"].X

We have set Table’s X directly. Next, we build and evaluate a classification tree:

>>> learner = Orange.classification.TreeLearner()
>>> from Orange.evaluation import CrossValidation, TestOnTestData
>>> res = CrossValidation(derived_train, [learner], k=5)
>>> Orange.evaluation.scoring.CA(res)[0]
0.88
>>> res = TestOnTestData(derived_train, test, [learner])
>>> Orange.evaluation.scoring.CA(res)[0]
0.3333333333333333

A classification tree shows good accuracy with cross validation, but not on separate test data, because Orange can not reconstruct the “petals” feature for test data—we would have to reconstruct it ourselves. But if we define compute_value and therefore store the transformation in the domain, Orange could transform both training and test data:

>>> petals = ContinuousVariable("petals",
...    compute_value=lambda data: data[:, "petal width"].X + \
...                               data[:, "petal length"].X)
>>> derived_train = train.transform(Domain([petals],
                                    data.domain.class_vars))
>>> res = TestOnTestData(derived_train, test, [learner])
>>> Orange.evaluation.scoring.CA(res)[0]
0.9733333333333334

All preprocessors in Orange use compute_value.

Example with discretization

The following example converts features to discrete:

>>> iris = Orange.data.Table("iris")
>>> iris_1 = iris[::2]
>>> discretizer = Orange.preprocess.DomainDiscretizer()
>>> d_iris_1 = discretizer(iris_1)

A dataset is loaded and a new table with every second instance is created. On this dataset, we compute discretized data, which uses the same data to set proper discretization intervals.

The discretized variable “D_sepal length” stores a function that can derive continous values into discrete:

>>> d_iris_1[0]
DiscreteVariable('D_sepal length')
>>> d_iris_1[0].compute_value
<Orange.feature.discretization.Discretizer at 0x10d5108d0>

The function is used for converting the remaining data (as automatically happens within model validation in Orange):

>>> iris_2 = iris[1::2]  # previously unselected
>>> d_iris_2 = iris_2.transform(d_iris_1.domain)
>>> d_iris_2[0]
[<5.2, [2.8, 3), <1.6, <0.2 | Iris-setosa]

The code transforms previously unused data into the discrete domain d_iris_1.domain. Behind the scenes, the values for the destination domain that are not yet in the source domain (iris_2.domain) are computed with the destination variables’ compute_value.

Optimization for repeated computation

Some transformations share parts of computation across variables. For example, PCA uses all input features to compute the PCA transform. If each output PCA component was implemented with ordinary compute_value, the PCA transform would be repeatedly computed for each PCA component. To avoid repeated computation, set compute_value to a subclass of SharedComputeValue.

The following example creates normalized features that divide values by row sums and then tranforms the data. In the example the function row_sum is called only once; if we did not use SharedComputeValue, row_sum would be called four times, once for each feature.

iris = Orange.data.Table("iris")

def row_sum(data):
    return data.X.sum(axis=1, keepdims=True)

class DivideWithMean(Orange.data.util.SharedComputeValue):

    def __init__(self, var, fn):
        super().__init__(fn)
        self.var = var

    def compute(self, data, shared_data):
        return data[:, self.var].X / shared_data

divided_attributes = [
    Orange.data.ContinuousVariable(
        "Divided " + attr.name,
        compute_value=DivideWithMean(attr, row_sum)
    ) for attr in iris.domain.attributes]

divided_domain = Orange.data.Domain(
    divided_attributes,
    iris.domain.class_vars
)

divided_iris = iris.transform(divided_domain)