Supervised learning with probabilistic morphisms and kernel mean embeddings
share
In this paper I propose a concept of a correct loss function in a generative model of supervised learning for an input space ๐ณ and a label space ๐ด, both of which are measurable spaces. A correct loss function in a generative model of supervised learning must accurately measure the discrepancy between elements of a hypothesis space โ of possible predictors and the supervisor operator, even when the supervisor operator does not belong to โ. To define correct loss functions, I propose a characterization of a regular conditional probability measure ฮผ_๐ด|๐ณ for a probability measure ฮผ on ๐ณร๐ด relative to the projection ฮ _๐ณ: ๐ณร๐ดโ๐ณ as a solution of a linear operator equation. If ๐ด is a separable metrizable topological space with the Borel ฯ-algebra โฌ (๐ด), I propose an additional characterization of a regular conditional probability measure ฮผ_๐ด|๐ณ as a minimizer of mean square error on the space of Markov kernels, referred to as probabilistic morphisms, from ๐ณ to ๐ด. This characterization utilizes kernel mean embeddings. Building upon these results and employing inner measure to quantify the generalizability of a learning algorithm, I extend a result due to Cucker-Smale, which addresses the learnability of a regression model, to the setting of a conditional probability estimation problem. Additionally, I present a variant of Vapnik's regularization method for solving stochastic ill-posed problems, incorporating inner measure, and showcase its applications.
READ FULL TEXT