Information theory: Uncertainty and self-information

[Cleric775]

What she's saying here is that if you knew the coin is always falling on the same side -no matter what- then there's no new information being gained.

[upvan111]

Thank u ....

it was greatly helpful.. pls ignore comments by stupid schmucks.. they don't even worth ur Attention...

[shrinkiwannakill]

@coolkat042495 *faggot *though *you're *that's *dropped *school *you've *proven *can't *that's *you're

This is ninth grade math? That was kindergarten grammar.

[Nademox]

I think he wears sunglasses as a trick; he doesn't want us to notice his eyes moving while reading text off the screen :) Good job anyway.

[JaannaChataHai]

@coolkat042495 9th grade math huh? wait till you do your bachlor's and at 5th,6th,7th semester you encounter a subject by the name information coding techniques or perhap even Information theory and you come back running here!!!

@ZJemptv you go ZJ \m/ and thank you!

[coolkat042495]

but like honestly tho this is 9th grade math your just trying to look smart haha thats why your ass droped out of high schooolllll hahahahahhaahahahah stupid shit youve already proved you cant handle anything thats why your on youtube bitch -your friend ryan

[coolkat042495]

hhahahahahahahahaha fagget you wasted your whole life

[AndroBubbles]

@coolkat042495 Educating others is a waste of a life? What does posting troll comments of youtube count as?

[kavana85]

@AndroBubbles fun

[way2jaded1]

hey he just made an argument for religion. Religion as previously stated works by taking events that we cannot explain and fitting an explanation to them. As we learn more such events become increasingly improbable thus increasing there importance as we get more information from such events. In a world where there is a large probability that anything may happen the probability of an event that is outside of understanding is huge but it's information is also huge and can be explained by Religion

[TheViciousSquare]

Ow, my brains!

[samuelinify]

I actually learned something today!!

[allegrobas]

It's eloquent and all that, but I got lost somewhere. Good job, Zinnia.

[Rawc90]

Please stop talking like your a robot with one single Tone of voice. Its so annoying! You must be an utter bore to be around.

[boochainletters12]

@Rawc90 If you've watched videos of this type where people don't talk monotone they get really hard to understand

[Rawc90]

@boochainletters12 No i understand them perfectly fine, they bring some enthusiasm to the topic. This just makes me want to kill myself its so depressing and boring. Perhaps you find it easier to understand because you have mental problems? Just letting you know... no offense intended :)

[boochainletters12]

@Rawc90 First thing I want to say is no offense taken, but saying "No offense intended" is kind of redundant. It seems weird to me, like saying "Your mother is an absolute whore and your father beat her because of it... no offense!" and getting surprised when the other person is upset by it.

Now that thats out of the way, when a person doesn't plan out their 'speech' and they talk differently to ZJ here, they make no valid points and say "and... um" continuously, which gets even more annoying.

[labrat1807]

Good vid, but I'm pretty sure you're only wearing dark glasses to hide the fact that you're reading from a script. No need to hide that. This is complex stuff and only very experienced professors would not need to prepare their lectures with scripted notes.

I notice that you don't wear dark glasses in other vids.  They work better, you seem more natural. As I said though, still a nice vid. Keep it up.

[aznlalaland]

@max4hunter1 Pretty sure you are just a dumb ass.

[Llawliet246]

@max4hunter1 He is a human being with feelings, dreams, and aspirations. He is not an animal or a piece of filth. Human beings are still human no matter what they happen to be attracted to, and your ignorance and appalling homophobia is disgusting.

[Llawliet246]

@max4hunter1 That was exactly what he was saying, illiterate milk livered barnacle.

[BvsMAcosh]

6.141 or so and 0.024 don't add up to one.

[ianmathwiz7]

@BvsMAcosh So what? They're not supposed to. Information density is not supposed to add up to one.

[pran1zzle]

I'm an internet troll, who's at heart a philosopher and a thinker. I've been thinking about information throughout my whole life, and I started to gather a huge interest in mathematics a while ago. Watching how both of my understandings about those two subjects got married here, on a theory that I gaged interest in very recently (information theory) due to some realizations in the field, was simply unreal. Thank you Ms. Smith.

[ObamaSucks12345]

For anyone who came upon this video hoping to learn about the information theory, I just would like to let you know your being tought by a pathetic fag. Good day

[natedaug1]

@ObamaSucks12345 Your homophobic.

[SageTheAtheist]

@ObamaSucks12345 Dildo

[Llawliet246]

@ObamaSucks12345 How is he pathetic, exactly? Because he is feminine? So femininity is pathetic. Thank you for the update, Mr. Misogynist :)

[purplchocolat]

glad i found you:)

[DrakeTimbershaft]

Damn it to hell, why do I have to see a Liberty University pop-up ad every time I play a ZJ Vlog? I mean the first time around I would click it off no sweat, but seeing ads for this evangelical version of Hogwarts is getting on my LAST damn nerve!

That aside, this is my favorite vlog.

[Axon36]

Awesome channel.... great job.

[Alzareon]

Your exposition is very clear. Good job.

One thing that I find difficult in the 'strange coin' example is: We still have only two possible states (1 bit of information), but the information we gain when we learn it is tails (probability=1/4) is 2 bits. How can we learn 2 bits of information when the system holds only 1 bit of information?

Depending on the probabilities, a system with two states can give us 100 bits of information if we learn it is in the state of probability 2^(-100). How?

[lalishelaide]

I want this glasses, even knowing that it wont work with me.

[MasauFuku]

@moreslaw It's not an issue with information theory, I'm not very familiar with it either, it's an issue with mathematics. x doesn't represent the information source, but rather, the number of possible outcomes of a given information source. In his equation, he incorrectly assigned that quantity to the variable M. If I were to take a simple equation, say y=2+x, and write it as a function, it would be f(x)=2+x. In both cases, f and I are functions of x.

[MasauFuku]

@MasauFuku Thus, the variable in the equation should be x, not M or anything else. This function, f(x)=ax, means that as x changes, f(x) changes by a factor of some undeclared constant value a. In ZJ's equation, M would technically represent some undeclared constant value, NOT the number of possible outcomes. It would have also been correct to write the function as I(M)=log2(M).

[bradstagangsta88]

lol wtf

[joe4lily]

are you a robot?

[metrx330]

I am not very good at maths but I actually understood most (most, I hasten to add) of what you said. Very interesting stuff.

[elbnderi]

I liked the way you simplified it. Great Great job.

I just I wished it was with no background sounds.

[Nalminji]

great video, thanks.

[FibonaziProductions]

After 1:42, I had absolutely no idea what was going on >_<. Complex mathematical equations + FibonaziProductions, does not compute.

[MasauFuku]

I cringed every time you used the wrong variable in your function (i.e. if the function is I(x), or I of x, the equation would b log2(x), or log base 2 of x, not log2(M)). Another issue is at the start when you go from log(M), or log base 10 of M, to log2(M). In addition, a(sub)i and p(sub)i should be the same variable set, since no data is being stored in your a(sub)i set.

But aside from your poor mathematical form, the video was very interesting and informative.

[fnglcnessie]

Taking me back to school, are you? Nuts, I actually should have pressed thumbs down... (Just kidding!)

[menonfire12]

It would be better without the music. 

[fnglcnessie]

@menonfire12 I agree. The music is kinda distracting... Especially with these kinds of subjects.

[NJATC2012]

hey im uncertain what you are 

[trombone7]

Say you and I are going to have fun and you are going to xmit a 1 for a heads and a 0 for a tails. Only I don't know it's a biased coin. Aren't I receiving the higher rate of information, since I'm doing the calculations for equal 50/50 frequency ? Don't I get a higher self-information / surprisal rate since I only slowly learn of the bias over time ?

[oMisFiTx]

SHEMALE!!!!!!!!!!

[kiroo54]

greaatt!!

[GameStar1300]

broken coin?

[freemaghreb]

Nice sun glasses.

[SmileyNerd101]

Information isn't a theory. It hasn't had to be proven because it's so obvious. So this is a waste of 8 minutes.

[Pudersepp]

Does the low probability face of the coin flipping up mean that the information received is good quality and more meaningful than the 0.99 probability one?

So, is the amount of bits directly related to the quality of the information? I have a feeling that it is not.

[brudswick]

yeah, but , you get the same information really. its which one it IS. it will be one or the other, and thats probablity, what actualy occures is information. idk, this whole study seems much more comfortable by a fireplace then by a chalk board.

[thelouisfanclub]

What if there's more than one outcome, like, say I read a book on Elizabeth I's Foreign Policy and why she did it. how would you calculate how much information I got from that?

Because some things, don't you find, the more information you have, the less certain you are about it?

If the book only said one answer, that would be fine, but if then you had another book that said something different, you'd be better informed but less certain.

[loyesj]

brilliant

[FiverBeyond]

@Mr88playmaker

Um... Shannon's Information Theory is hardly the same as 'random' or 'chaos'.

[coulditbekronau]

Holy fuck I want thoes shades

[MrLexguy2007]

OMG! Does anyone really even care??

[ZJemptv]

Had you listened and understood the video -- evidently at least one component of your comprehension has failed you -- you would know that this is about the formal definition of information as used in (classical) information theory.

[Sundrownpaw45]

i just died in my head XD

[YouTees]

hint: stop playing spiritual ping pong with 'christians'. they have set it up for you to LOSE everytime; due to the incomplete information they have AND the insecurity that drives their lives. If you apply 'faith' in invisible gods...you can make even a TOMATO a god. Except that you can see a tomato. it is red and juicy and good and so nobody argues about that! Invisible gods make great conversation pieces but are like a dog chasing it's tail. Let the christians ramble. THAT is what they do.

[YouTees]

Christians cannot even agree with THEMSELVES on what is real. They cannot even settle on a visually appealing 'Jesus'! Go to any so-called Christian bookstore and you will find WAY more contradiction than can possibly represent the so called ' one true christian god' ; even if there is not even one atheist within miles! Christians don't need atheists to appear silly...you have done it to yourselves! Funny also how UN christian they get when an atheist speaks out. UH OH! 'WWJS'(say)

[sarahj666666]

you get stars for all of your vids, but I find the music often disturbing (*) - maybe, you should only use it as an intro ?

(*) maybe that's due to english not being my native language - so I may be more affected.

[founoe]

After watching this video several times, checking the purpose of logarithms on wiki, making math problems for my self to solve I came to the very embarrassing conclusion that I have no idea why:

x^0 = 1

[alawrence89]

@alawrence89

Another way is to use the formula:

x^(n-1)= (x^n)/x

Plug in n=1 and you'll see that

(x^0)=(x^1)/x=1

[zackindy312]

5^2 = 1*5*5

5^1 = 1*5

5^0 = 1

[Syrenna94]

You made my brain 'splode. XD

Was pretty interesting video though.

[itskelvinkkc]

Nice ... just listening to these kinda things can make me dizzy. I'm so jealous that you can actually do it so clearly.

[kmacmuzikmafia27]

Wow thanks, I think "You're a legend" is possibly the best compliment ever lol! :)

[TonsofFun777]

She is an ugly fuck thats for sure..

[SmileyNerd101]

I think its a guy, or an alien, or a monkey. Yeah, hes a monkey. Believing in evolution turned him into a monkey. LOL

[Awesomo4003]

@SmileyNerd101 your a waste of meat

[SmileyNerd101]

Wow, that comeback was so bad I can't even call it a comeback

[Awesomo4003]

@SmileyNerd101 couldn't possible be as bad as the ignorant remark you made

[SmileyNerd101]

You fagots say ignorant so frickin much.

[Awesomo4003]

@SmileyNerd101

Ignorant means oblivious to the facts, clearly this the best description of you, now do you see why there is a trend?

[SmileyNerd101]

Why the fuck would it be a trend? Fucking twilight shirts are trends not the word "ignorant".

[Awesomo4003]

@SmileyNerd101 There is a trend of people describing you as ignorant. Please take the time to read comments carefully before getting so angry it makes you look very unintelligent.

[SmileyNerd101]

@NotAwesomo4003

And please stop acting like such a douche. You contribute nothing to society or to yourself by sitting on the computer arguing with a 10 year old.

[Awesomo4003]

@SmileyNerd101 Well if you don't think it is right for people not to argue with ten year olds and you happen to be a ten year old DONT FUCKING ARGUE WITH ANYONE

[SmileyNerd101]

Sounds like someone is getting mad. Yeah, I can argue with who ever the Hell I want to because I at least have enough common sense to realize that we didn't transform from fucking monkeys.

[Awesomo4003]

@SmileyNerd101 You just wrote that people shouldn't argue with ten year olds ( or people who argue with ten year olds contribute nothing to society, which makes absolutely no sense ) and we didn't transform from monkeys we evolved or a long period of time, you clearly have zero knowledge of anything evolution related.

[SmileyNerd101]

You dumb ass, it was a joke. I know you believe we evolved, I was making fun the stupid ass THEORY. And you obviously don't have enough common sense to get that I was saying that Atheists contribute nothing to society they just sit around and argue with ten year old. This is exactly what I'm talking about, Atheists lack common sense, which makes you ignorant.

[Awesomo4003]

@SmileyNerd101 How does that make me ignorant? just because you claim something to be fact does not make it true. You are discriminating against Atheist people and it is wrong, I don't care how old you are. Oh please its quite clear you know nothing about the theory of evolution.

[SmileyNerd101]

I know everything about the retarded fish frog theory. You believe the big bang created everything and there was a single celled organism that mated with another which made a offspring and made a new species. But then they started mating with land animals such as lizards then monkeys and that made human beings. And I have found several flaws in that THEORY. First off how the fuck are you going to get a lizard to have sex with a squire or a monkey?

[Awesomo4003]

@SmileyNerd101 It occurs over a very long period of time no lizard has ever has sex with a monkey, you VERY CLEARLY do not "know everything about the retarded fish frog theory" . It is the most scientifically supported theory of our past, there is always going to be some doubt but unless you can provide some information that would make believe otherwise . Also I think you may be a troll so I am not responding to what ever reply you may leave.

[SmileyNerd101]

OK, you are clearly closed minded so I am just going to leave the conversation at this. Good luck with that when the rapture comes.

[xskywarpzero]

Wait......what.......raptors are coming?!?!

[xskywarpzero]

wait why is mister religion swearing? won't your god get angry?

[SmileyNerd101]

Are you stupid. No he won't, they are words. It bad when you cuss against God but not when you say shit because you stumped your toe. Sure he won't like that your saying it but I believe in a reasonable God. If the was a cuss then people would be cussing all the time. Cuss words are only he because we made them cusses. In the Bible a donkey is an ass. Damn is Hell and hell is hell. A bastard is son without a child. So there you go. If you don't get that then I will be glad to speak slowly again.

[xskywarpzero]

.....but i do it for the luls!

[gagnashdiak]

Ugh, guys, ZJ is talking about Information Theory, not about information in general. Read the title before criticising.

[kirstiederossett]

The glasses are not good for your face shape. Your scarf is amazing though. ily.

[Yesmaamiagree]

Im sorry, but arithmetic is not a very good way to explain information. Especially at this pace. I've been through 5 years of college, and I had to pause and look over this again.

[X6XMegadethX6X]

shut the fuck up and kill yourself already kid

[kmacmuzikmafia27]

someone who wishes suicide on someone for no other reason than existing, is not worth living themselves. Taking one's life is not a fucking joke.

[X6XMegadethX6X]

i know it's not a joke, my question is, what makes you think i was joking? i think it would take one sick fuck to accuse anyone of something like that if you ask me.

[JohnLRice]

Well, it depends on what information you receive when you are uncertain, doesn't it? No information or bad information doesn't nessesarily reduce uncertainty, quite the opposite.

If you flip the coin and right before it lands it disappears, a small nun screams "ARDVARK BANANA FIGHT!" and you uncontrollably soil yourself, your uncertainty level will probably shoot up expotentially. (although you would probably be pretty certain that you wont be flipping any more coins any time soon! ;-)

[sam213521352135]

PORN! :)

[takezaspirit]

"Once again we have this problem of equating ordinary meaningful information with the information in Information Theory. It is true that meaningful information is the opposite of randomness but the information in Information Theory is NOT the opposite of randomness." - Scott Uminsky

(see the link below)

[kentie1964]

Solid! Thanks for posting this.

[takezaspirit]

You began the video saying that information is what you get when you REDUCE uncertainty. I don't agree.

But then you went on to prove that information is what you get when you INCREASE uncertainty.

:)

Did I miss something?

[yr1ntrly]

Hello,

I am curious: When you say that you "don't agree" with the statement that information is a reduction of uncertainty, what do you mean? Do you mean that that is not the definition (based on the literatures of information theory) or that you simply do not accept that definition (based on your personal taste)? In either case, can you kindly quote or suggest an alternative definition?

[takezaspirit]

ideacenter (d0t) org (slash) contentmgr (slash) showdetails (d0t) php (slash) id (slash) 1376

[Bongwyre2112]

You can take the red pill or the blue pill.

[HeavensOmega]

that's so Karl Popper

[RemsUnheardofArmy]

X.X its 4 in the morning why am I still and this is giving me a headache. I guess I'll have to watch it again tomorrow to get it

[sy1234]

*brain explodes*

[DethFromAbv]

so can you ever reduce uncertainty but not gain any information

[ZJemptv]

Nope. The amount of information you gain is determined by how much uncertainty is reduced. If you have 1 bit of uncertainty about an information source, which is eliminated when it returns an outcome, you now have 1 bit of information.

[TomConger]

Wait, would you still use log-base 2 when you're doing it for a die? I thought log-base 2 was binary.

[ZJemptv]

I didn't really get into the practical applications, but it was originally meant to find out, for example, how much "space" you need in order to transmit certain information. It's about communication.

For example, you'll need on average 2.585 bits to store an outcome of a fair die. For an unbalanced die, you'll need fewer bits.

We use base 2 to measure things in bits, because most computers and communications systems nowadays operate in binary, 0 and 1.

[TomConger]

Oh... I was confused. I thought you used Log2 because there were two options. Not sure how I mixed that up.

Great vid, by the way.

[Ninwa]

Ah, this actually answers my question in the other thread. I'll make a note of it there so you don't waste your time answering my question. :)

[randyhelzerman]

Great summary of Shannon information.

[ZJemptv]

I'm also considering doing a brief synopsis of Kolmogorov complexity, using Python. Your series with Prolog was fascinating -- I doubt I could make 8 videos on the subject, though!

[randyhelzerman]

Go for it! When its up I'll feature it.