Information theory: Coding (1 of 2)

[upvan111]

Great ,, both ur lecture and background music .. its bethhovens symphony i think

[tord77]

Thanks !! :) Again, your videos on inf.theory rocks!

[tuuurdle]

niggas a genius

[blueshogun96]

Wow, I had no idea this person was into coding. I thought "her" greatest hobby was beating down shockofgod with a digital baseball bat, lol.

[ugenlik]

plse dont use classical music next time when you upload a video, nice video but when music rises i am distracted or try to use some dubstep instead

[sigh2infinity]

the only thing confusing about this is ....are you a man or a woman?

[chewy6i]

excellent presentation. 

[MightyTokez]

amazing comprehension and explanation.. ZJ what do you do for a living or studying in school>?

[raduni131]

Thanks, with your help i figured out how to do huffman coding.

[killerzguy]

thanx buddy.. its really useful information for me even though im a newbie in coding thing ^^

[dedenapraia]

you save me, thankss !!!!!

[ibrr2]

r u man or woman 

[azzy314159]

@ibrr2 She is a gay person, She prefers dressing as a woman. On an intellectual level she is amazingly bright. Which is no wonder as a great number of geniuses were openly or closeted gays. And I love her.

<3 and Peace, ibrr2.

[jkf77700]

@ibrr2 I'm confused too!..but hey, s/he's good at this!!

[ThaFacka]

You are so smart (Gödel), thank you!

[steamwheedlecartel]

It's Pat!

[jmaxwell2]

@ZJemptv: Thank you for this. Despite my being a programmer for over a decade, mathematics was never my strongpoint. I had been able to figure out other encoding methods such as RLE and and simple dictionary based ones, but Hoffman and its siblings have confused me. Thanks again for this!

[mabsahmed786]

the H(A) is approximately equal to 2.579 in the example towards the end of the video, not 2.585. 2.585 represents the I(X). am i right?

if i am right, then please tell me what the relation between I(X) and H(X) is?

[apolloe23]

thanks!! that was really helpful!!!!

[Texmurphy51]

Lets try and make it complex enough to confuse the issue.

[BillyMays9111]

that was like in chinese right?

[Atilla3]

Thanks, the whole thing was explained really nice. Although I would give the higher values 1 and lower a 0. Yes it does not matter technically but the order matters to me.

I am not trying to make fun of you when asking you if you. Are a feminine male or a masculine female?

[fnglcnessie]

Aaah, now I remember...

I thought the Huffman code was fascinating when I first heard of it... Thanks for the refresher course! :D

[gregrutz]

This is really exciting,,,,,,,,,,,,NOT.

[zantrua]

@gregrutz

That was really clever ,,,, NOT.

[HebaruSan]

You kind of skipped the justification for why the encoding algorithm is necessary before jumping into the hairy details that no one will remember or be able to apply the first time they see it; with only six plaintext symbols, it may be simpler to attempt to create the shorter code ad hoc starting from the 3-bit version, and explore how and why various coding systems would or would not be ambiguous.

[560e]

Nice video Zinnia: It was clear, well organized and not one unnecessary word spoken.

[luno44]

0:00 "last time". What last time? I don't see a pattern in your naming structure to determine an order so that I can watch part 1. I figured "Coding (1 of 2)" would mean this was part 1, but it isn't.

[Apsychowar]

you're great man

[greeppl]

also name of the video should have "huffman code" words in it

[greeppl]

great commentary, great video presentation but:

the example chosen is abstract (a1 a2) I would rather use a real word & letters to show how it is used. also the assignment of 0s and 1s was confusing.

Still video is good but having a concrete example & better way of assigning the 0s and 1s would result in the 'ideal' video.

I probably would not unterstand the concept without looking at other, similar videos.

But it's usually the case that different videos complement each other so...

[hodonkain]

You probably know this but I figured I'd say it anyways...

You can create a Statistical Model and simply have the computer guess what data is going to appear next in the data stream using the model for it's guessing.

The word entropy is complicated. With small data, there is a fixed amount of space it can take up but with large binary numbers, it can be converted to it's original BASE10 number then represented with an equation. Surpassing the entropy.

[socratus1]

Information / Consciousness can be transfer only by Electromagnetic waves

There arent Electromagnetic waves without Electron

We know Electron is very important particle

It acts in Maxwells electrodynamics

It acts in the atom

But how Electron acts in cell we dont know

We need time to understand this fact

And when we understand Electron

we will know the Ultimate Nature of Reality and the material basis of Quantum Consciousness

[1001011001001010011]

@socratus1

That's pure gibberish. It's also obviously wrong.

Information is not limited to electromagnetic waves. Consider: every time you hear someone speak, their voice (and the information carried by it) is made up of mechanical (not electromagnetic!) vibrations of air molecules.

There can be electromagnetic waves without electrons: photons work just fine.

We already have extensive knowledge of how electrons (and the chemicals they are a part of) behave in the cell.

[Howie47]

bits do NOT represent pertinent, specific, purposeful information. Those bits must be (designed) into real information that relates a message. (This encoding equals this translation. This wording equals this reasonable purpose.)

[ZJemptv]

What are the criteria for distinguishing between "pertinent, specific, purposeful information"/"real information" as opposed to, presumably, "unreal information"?

[Howie47]

Shannon in his theory used the word "information" as a convenience. Uncoded bits have the potential of carrying, specific purposeful information, (like the writing we are sharing.) Shannon called those non-coded bits (information). But they don't inform us of any thing until they are coded to do so. Hard core Darwinist, don't like to admit there is such a thing as real info verses chaotic nonsense.

[ZJemptv]

Whether information is "specific" or "purposeful", or whether it "informs us" of anything, is beside the point and has nothing to do with how information is quantified. The problem is that "specific" and "purposeful" are relative terms -- "purpose" is in the eye of the beholder. For example, a long string of random numbers may seem meaningless to you or me, but it may serve as a critical component of a cryptographic program. Thus, things like "purpose" are irrelevant to quantifying information.

[Howie47]

Thanks for the morning laugh. I've been saying for some time, atheist will just say

meaning is all in the eye of the beholder; like they do with beauty. Quantifying meaning is adding it up. Real information can't be counted with numbers. Because it is multidimensional in meaning. Info. needs both a writer and translator. But no matter how sophisticated the reader, it's not going to make sense out of (chaos). Some thing has to be programmed in first. The bits can be added, but not meaning.

[ZJemptv]

Information theory isn't about the "meaning" of information. It doesn't cover that. Further, those who propose that "real information" can be differentiated from "unreal information", whatever that may be, do not have any sort of formal or rigorous systems by which information can be categorized as "real" or "not real".

[Howie47]

What B.S. "No Free Lunch" information theorem; states computers can't write meaning. They can only transfer meaning. I added the word "real" to emphasis that meaning is real & not in the eye of the beholder. A rigorous system is not needed to categorize some thing as simple as meaning. Scripts mean something or they don't. DNA is translatable into specific life forms. It is (repeatable). It is more then a scientific fact. It is an actual fact! So stop your mortification of truth.

[ZJemptv]

The "no free lunch" theorems say nothing of the sort, and have nothing to do with any of this. If you think "a rigorous system is not needed" to categorize this, and they "mean something or they don't", then why haven't you or anyone else formalized this if it's so simple? Perhaps the question of what is and isn't meaningful is more complex than you make it out to be.

[Howie47]

It's called NFL because information doesn't arise from computer algorithms for free. That is a basic tenant of NFL theory. Meaning can only be detected and produced by a mind. IF it could be quantified we could build a thinking computer. Now if your suggesting some science discussions have descended into the realm of mindless arguments, I would agree. Mind can't be detected in some scientific disciplines. LOL

[ZJemptv]

Yes, information can arise out of literally nothing -- no "cost" is incurred, so it makes no sense to talk about whether it is "free" or not. The term is meaningless here. Again, the "no free lunch" theorems are not relevant here.

"Meaning can only be detected and produced by a mind."

False. We can find meaning in things that arose without being "produced by a mind".

"IF it could be quantified we could build a thinking computer."

Are you saying we can't?

[Howie47]

1. Name one instance of specific, purposeful, information arising out of nothing! 2. What do you mean by here? Our on going discussion, information theories, or origin of usefull information? Which NFL says can't happen with out a cost!

What information with meaning & purpose arose with out being a product of a mind?

We haven't yet. It would probably be beyound math and would need some combination of life and your calculating machine, (computer) As specified information is intuitive.

[ZJemptv]

Once again, until you have some criteria in place for distinguishing "specific, purposeful, information" from everything else, it's pointless to ask for examples of such. How is anyone expected to provide them when there's no way to discern whether they qualify?

And, yet again, the no free lunch theorems do not say anything about this. They do not support your assertions. In fact, they're completely unrelated and were just misused by Bill Dembski as support for "specified complexity".

[ultravioleNt87]

OXW PUSTH KI ASXHME

[Spacemonkey2084]

Planning a video on entropy and how statistical model's information gain can be measured through relative entropy. Had to come back and watch these to make sure I wouldn't be too redundant.

What do you use to overlay the text on the video? I've never been fancy at all with my video editing.

[ZJemptv]

Just Photoshop + Sony Vegas.

[CompuViz]

Its transparency effect in editing if editor has it, this way you can overlay layers without fading the back layer totally, it usually can be two layers of videos and any number of layers of graphics. Etc

[AbsalomPitchingfork]

What are the applications of this in cryptology?

[armando101it]

Yoo ZJempty, I'm a developer and is nice that u also like computing!

[TheApatheticDespot]

Nice video, I love Huffman coding. It was probably the coolest thing I learned in my whole first year as a CS student. If I could make a suggestion, though, I do think it might have been more clear to non-computer scientist viewers if you had explained it in terms of a traversal through a tree. Your graphic was perfectly correct, of course, but it did get a bit cluttered and hard to follow and I think it's just easier to get a feel for it once you think of it in those terms.

[Deuteriumboy]

okay... now i´m confused...

[sasankashehan]

can u put a one for arithmatic coding as well

[sasankashehan]

thanx soooooooo much

[TheReasonWhyGuy]

Is this technique similar to that used to compress files.

Now I "know" that compression is usually over sorta filler areas, [strings of 0's]

I am going to also assume that in the place of the memory saved, there is a little issue with the speed at which the information is able to be accessed.

Is that a correct assumption?

[shoofeer1]

Many thanks sister for the gr8 video!! thanks......

[dmix09]

this is excellent! thanks

[pyrogamer]

The 2/6 from a3 and a4 should be placed above the 2/6 from a5 and a6.

[inhabihar]

U did a gr8 job...tomorrow i have exam and 1 topic is huffman......u explained it amzingly....Thanks.

[magicman5789]

keep it up, its really helpful to understand the concept and it will be very helpful if u can upload the video in dynamic Huffman coding and arithmetic encoding

[Ninwa]

As more symbols arise, are the compressed codes always completely unambiguous when stringed? I noticed, to my pleasant surprise, that I cannot form a code where the symbols can't be disambiguated. I tried using a 6-string code which could be either two three bit codes, or three two bit codes, and found I couldn't. Was this as a result of the method, or could this potentially be a problem for the receiver in other situations?

[ZJemptv]

I'm actually going to cover that in the next video (we're thinking in the same direction, how wonderful!), but yes, Huffman coding does generate unambiguous codes. If they were ambiguous, that would indeed be a problem for the receiver.

[madrobot]

At 3:37 you said the probabilities were listed in descending order, so a5 and a6 are the least probable symbols, but at 7:26 you gave these symbols the shortest (2-bit) codes.

Obviously it would be more efficient to use shorter codes for symbols with higher probabilities. I assume you meant to say the probabilities were listed in ascending order.

[ZJemptv]

Actually, since this was a fair die and they're all equally probable, those could just as well have been at the top anyway, the Huffman coding just happened to place them at the bottom. The next video will have a clear demonstration of Huffman coding for unequal probabilities, and the more probable symbols do receive shorter codes.

[Scrapheap71]

Is there a point behind all this obsessing on Information theory??

[ZJemptv]

Nope, it's just something that interests me, and hopefully some other people. I'll be done with it some time soon.

[Awwa1]

This is sooo incredibly boring, yet, I can't turn away!

I like your monotone speaking style. I believe that and the classical music playing in the background, help me concentrate on what you're saying.

It's still boring! But I am interested in learning more! Sorry, I never was one much for math. Maybe it's because I intuit much of it. Counting only makes it more precise!

Peace,

|A/

[Stephen5000]

The first part reminded me of DNA

This topic is really interesting!