Empirical Confidence Statistics By Predicted Utility

The Information shown here

The data linked to in this page is intended to help answer the question:

How reliable is a given prediction?

Given that a protein localizes to site A, what is the probability that it has a predicted utility of some value u or more for site A?

Utility

I borrowed the word utility from decision theory, with the intention that eventually WoLF PSORT could predict the utility of believing that a protein localization to a give site -- which combines the probability that the protein combines to each site with the (in general non-uniform) cost of the various possible mistakes. For example when only interested in predicted secreted proteins, one could consider predicting a mitochondrial protein to be a nuclear protein to be a "right" answer -- since both sites are non-secreted.

In practice I have only used this functionality as one way to address proteins with multiple localizations, by lightly penalizing misclassifications between related sites (localization classes), for example to make predicting nuclear for a protein with dual localization to the cytoplasm and nucleus be more acceptable than to predict mitochondria for that protein. For details please see our APBC06 Paper.

Example graph explained

Near the bottom of this page there is a table containing graphs like the following pair for each localization site.

Histogram	Smoothed Proportion

The left hand graph is a histogram plotting the observed frequency of predicted utilities for lysosome for proteins which actually localize there (light bars) vs. proteins which localize to other sites.

The right hand graph is a smoothed curve representing the probability that a protein is a lysosome, given that it has a particular predicted utility for lysosome, under the assumption that the prior probability that a protein localizes to the lysosome is equal to the proportion of lysosome proteins in WoLF PSORT's dataset.

By inspecting these graphs, one may make some conclusions. For example

• WoLF PSORT is not capable of making high confidence predictions for lysosome
• If the predicted utility of lysosome is more than eight, than the protein is more likely than not to localize to the lysosome
• If there is some prior reason to believe a protein localizes to the lysosome, than a predicted utility of three is no reason to be discouraged -- this is a typical value for lysosome proteins

Links to graphs for each (organism, site) pair

Copyright (C) National Institute of Advanced Science and Technology (AIST), Computational Biology Research Center (CBRC). All Rights Reserved.