Yes, it is trivially true that most inductive reasoning can be viewed as an application of conditional probability (however, *not* necessarily Bayes theorem), but this shouldn't be too surprising since conditional probability is essentially just formalized common sense. It's also not a very useful observation since the probabilities you'd need to do actual calculations typically aren't available.
It's a reference to physics where certain theories explain small-scale phenomena but fail to explain large-scale phenomena. Saying 2 + 2 = 5 for extremely large values of 2 is a complete joke. It obviously doesn't make sense, that's why it's funny, and Lawrence Krauss knows this. William Lane Craig is just a charlatan.