arXiv:1808.06324v1 [cs.LG] 20 Aug 2018 PAC-L EARNING IS U NDECIDABLE APREPRINT Sairaam Venkatraman * S Balasubramanian R Raghunatha Sarma Department of Mathematics and Computer Science, Sri Sathya Sai Institute of Higher Learning, Andhra Pradesh, India August 21, 2018 ABSTRACT The problem of attempting to learn the mapping between data and labels is the crux of any machine learning task. It is, therefore, of interest to the machine learning community on practical as well as theoretical counts to consider the existence of a test or criterion for deciding the feasibility of attempting to learn. We investigate the existence of such a criterion in the setting of PAC-learning, basing the feasibility solely on whether the mapping to be learnt lends itself to approximation by a given class of hypothesis functions. We show that no such criterion exists, exposing a fundamental limitation in the decidability of learning. In other words, we prove that testing for PAC-learnability is undecidable in the Turing sense. We also briefly discuss some of the probable implications of this result to the current practice of machine learning. Keywords PAC-learning, Turing Machines, Machine Learning 1 Introduction The existence of a criterion, preferably efficient in time and space, for deciding whether a function is learnable by a given family of hypotheses can prove to be useful to both practitioners and users of machine learning. For example, such a criterion can help make crucial decisions based on the feasibility of applying machine learning to problems of interest, potentially saving time and resources. Furthermore, if such a criterion could also allow for a comparison of learning algorithms and models, it would only be all the more useful. In order that a meaningful investigation into the existence of the intuitive aforementioned criterion be made, we must assume a particular, defined model for learnability and use a formal, mathematical framework for studying the problem of learning. In this study, we restrict ourselves to the well-known PAC definition of learning [1] and use the mathematics of Turing machines to treat the problem of learnability in a systematic manner. The use of Turing machines in this setting is by no means accidental; showing that no Turing machine can decide the learnability of every (computable) function can be seen as evidence of the non-existence of any algorithm to do the same by use of the Church-Turing thesis [2]. Further, this undecidability can also be seen as proof of fundamental limitations in learning, similar to the limitation on computing proved by the existence of incomputable functions. We now give a brief outline of the article. The next section mentions some of the work in the literature that shares similarity with this article in philosophy and scope. We then introduce concepts necessary to the development of the central result of this paper in the subsequent section. We then state and prove the main undecidability result. We conclude with a discussion on a few hypothesized implications of our result. * vsairaam@sssihl.edu.in