I've learned a significant amount of JavaScript tricks from the MEAP Ninja book, blog posts, and have been real interested to see what you'd come up with at Khan-this is awesome.
I'm curious on some of the motivations to handle audio playback by synchronizing a Youtube video as opposed to using a HTML5 Audio element.
I could see that by creating Youtube screencasts first, you can present both a video to users with unsupported browsers and an interactive experience to others. Any other reasons?
@reconor Yep, that's the basic premise - it'll help to remove the amount of duplicate work that we have to do. With Youtube I can work with a unified API that'll work identically everywhere, which is quite enticing. HTML 5 Audio is certainly something to explore but I worry about creating duplicate work and encodings for us to deal with. We'll see.
@jeresig Ah yah from that standpoint it makes perfect sense, you aren't losing any of Khan's existing accessibility while finding ways to expand on it with minimal duplicated efforts.
Great implementation idea and I appreciate the preview into some of your working ideas ( as well as the response ) - it's good stuff. :)
If the screen-casting interface will have the ability to execute code during playback, it might be handy for the viewer have a popup box in which the values of certain variables are animated/tracked (like the Workspace/Values tab in RStudio).
@figital Absolutely - I definitely want to provide tons of debugging information to the user (value of variables, as an example). That would be especially useful to a beginning programmer, I think.
@kaninepete /Maybe/ Right now it seems to be quite hard to track that sort of information. The only browser that exposes pressure sensitivity information is Firefox (WebKit hasn't landed it yet). And even then it's only basic pressure-sensitive information, not information like stylus direction, tilt, etc. I'll certainly keep exploring it but I'm not holding my breath.
@kaninepete Makes sense - but yeah, as I mentioned, that information is only in Firefox right now, unfortunately. I can certainly make use of it but it'll have to be limited.
@11DRealityHacker I coded this in JavaScript. I used jQuery, jQuery UI, jQuery Simulate, ACE Editor, and the HTML 5 Canvas element. I'll probably end up using a templating plugin soon as well (probably the micro templating plugin I wrote).
@CoderDBF Yep! This is something that I've been thinking about for a while. I've been displeased with the simplicity of many Computer Science teaching tools - while at the same time the Khan Academy-style videos weren't interactive enough to really engage a student. It felt like some combination of the two would be the ultimate union of those technologies. Hopefully it'll be especially useful for students!
I just want to say that this is a fantastic application and one that I will be making use of as soon as I get my hands on it :) I am also working on a collaborative art project using html5 JavaScript and php. It would be nice to find out how you did your recording feature because I am currently trying got implement that?
@3y3n5tyn Unfortunately there is no fool-proof way to do a recording feature, it's just a lot of hard work. You need to record every action, give it a timestamp, and play it back based upon timers. It gets even harder if you want to support 'undo' (which I am).
@jeresig thanks for the reply :) i am actually saving every element placed on the canvas as an image and replaying that back. Encoding the images into base64, storing them into an XML file and the reading that back from the server using PHP. I had truncated errors which i am trying to fix by breaking the base64 encoded images into smaller chunks.
@3y3n5tyn Hmm, that sounds complicated. I just track all the mouse movements and keyboard presses and play them back - much less expensive. An entire movie can be reduced to only about 32KB or so in that way.
@jeresig i was thinking about recording mouse movements but i also wanted to store information about brush types, eraser and colour. It worked out easier to just encode the entire canvas and break it into chunks, which slightly increases performance. Also when reading it back I just need to store it into an array and link it to a slider for replay functionality. Another idea was to store the UInt8ClampedArray to a file and read it back but only the first row of the canvas was populated...:(
@Martendemus Perhaps - although we can kill two birds with one stone here. 1) Not everyone will be able to visit this in a browser and use a code editor - but it might be useful for them to learn anyway (so we can give them a video as a fallback alternative). 2) Since we need the audio track anyway, might as well just use the audio from the YouTube video. I agree that the extra bandwidth usage isn't ideal, but it works :)
@yuraz1985 Thankfully since these are also saved as Youtube videos we'll be able to make use of our existing captioning capabilities - and even support multiple languages as a result (much like what we're doing now with the upcoming Khan Academy iPad application). It may not be in version 1 but it's definitely something that we want to do.
Something like a animated stack of how variable are changed over time loops flags etc, would be nice
thefirstfirefox 1 week ago
Comment removed
JScodeZ 2 months ago
where to download?
magnetismewave 2 months ago
I wish this was available when I started creating videos about programming in Processing 90 episodes ago! :)
hamoid 3 months ago
Does this mean that crowdsourcing review questions is on the back burner, or are you working on both simultaneously?
kaninepete 3 months ago
I've learned a significant amount of JavaScript tricks from the MEAP Ninja book, blog posts, and have been real interested to see what you'd come up with at Khan-this is awesome.
I'm curious on some of the motivations to handle audio playback by synchronizing a Youtube video as opposed to using a HTML5 Audio element.
I could see that by creating Youtube screencasts first, you can present both a video to users with unsupported browsers and an interactive experience to others. Any other reasons?
reconor 3 months ago
@reconor Yep, that's the basic premise - it'll help to remove the amount of duplicate work that we have to do. With Youtube I can work with a unified API that'll work identically everywhere, which is quite enticing. HTML 5 Audio is certainly something to explore but I worry about creating duplicate work and encodings for us to deal with. We'll see.
jeresig 3 months ago
@jeresig Ah yah from that standpoint it makes perfect sense, you aren't losing any of Khan's existing accessibility while finding ways to expand on it with minimal duplicated efforts.
Great implementation idea and I appreciate the preview into some of your working ideas ( as well as the response ) - it's good stuff. :)
reconor 3 months ago
@jeresig going off of your twitter stream - it looks like you're investigating soundcloud as well?
long100 3 months ago
@jeresig please, pretty please, don't use comic sans in the final version.
CountingStones 3 months ago 2
@CountingStones I'm not using Comic Sans now? If you're referring to the code editor font it's Droid Sans.
jeresig 3 months ago
Just curious, how did you personally learn your computer programming skills?
TheAustinConlon 3 months ago
@TheAustinConlon Reading books and lots of experimentation! A rather painstaking process :)
jeresig 3 months ago
Looking forward to watching you build a GUI ! :)
If the screen-casting interface will have the ability to execute code during playback, it might be handy for the viewer have a popup box in which the values of certain variables are animated/tracked (like the Workspace/Values tab in RStudio).
figital 3 months ago
@figital Absolutely - I definitely want to provide tons of debugging information to the user (value of variables, as an example). That would be especially useful to a beginning programmer, I think.
jeresig 3 months ago
It's awesome you're collaborating with Khan Academy. It looks like it really blends interaction well with instruction.
GeorgeKovats 3 months ago
@jeresig can I download it from khan academy's website or any other website.
If not, then tell me any other way to use this tool you've created..
shekharsharma705 3 months ago
@shekharsharma705 It's not really publicly usable yet - will hopefully be available soon.
jeresig 3 months ago
Fantastic work.
Will the draw tool be pressure sensitive like SmoothDraw?
kaninepete 3 months ago
@kaninepete /Maybe/ Right now it seems to be quite hard to track that sort of information. The only browser that exposes pressure sensitivity information is Firefox (WebKit hasn't landed it yet). And even then it's only basic pressure-sensitive information, not information like stylus direction, tilt, etc. I'll certainly keep exploring it but I'm not holding my breath.
jeresig 3 months ago
@jeresig I think basic pressure is all that smoothdraw uses.
as long as it has high enough resolution/fidelity, I'm sure you can get by without pressure information.
kaninepete 3 months ago
@kaninepete Makes sense - but yeah, as I mentioned, that information is only in Firefox right now, unfortunately. I can certainly make use of it but it'll have to be limited.
jeresig 3 months ago
Two questions, what language did you program this in and what libraries did you use in the creation?
11DRealityHacker 3 months ago
@11DRealityHacker I coded this in JavaScript. I used jQuery, jQuery UI, jQuery Simulate, ACE Editor, and the HTML 5 Canvas element. I'll probably end up using a templating plugin soon as well (probably the micro templating plugin I wrote).
jeresig 3 months ago
Did you yourself came up with this whole concept of putting an exercise inside a video and a drawing ability inside a code editor?
To me it looks very revolutionary stuff.
CoderDBF 3 months ago
@CoderDBF Yep! This is something that I've been thinking about for a while. I've been displeased with the simplicity of many Computer Science teaching tools - while at the same time the Khan Academy-style videos weren't interactive enough to really engage a student. It felt like some combination of the two would be the ultimate union of those technologies. Hopefully it'll be especially useful for students!
jeresig 3 months ago
Woah. A few weeks ago, as I was walking, this exact idea came to mind so I jotted it down in Evernote. Turns out you're making it a reality!
TheAustinConlon 3 months ago
@TheAustinConlon Awesome! Great minds think alike, I guess ;)
jeresig 3 months ago
I just want to say that this is a fantastic application and one that I will be making use of as soon as I get my hands on it :) I am also working on a collaborative art project using html5 JavaScript and php. It would be nice to find out how you did your recording feature because I am currently trying got implement that?
3y3n5tyn 3 months ago
@3y3n5tyn Unfortunately there is no fool-proof way to do a recording feature, it's just a lot of hard work. You need to record every action, give it a timestamp, and play it back based upon timers. It gets even harder if you want to support 'undo' (which I am).
jeresig 3 months ago
@jeresig thanks for the reply :) i am actually saving every element placed on the canvas as an image and replaying that back. Encoding the images into base64, storing them into an XML file and the reading that back from the server using PHP. I had truncated errors which i am trying to fix by breaking the base64 encoded images into smaller chunks.
3y3n5tyn 3 months ago
@3y3n5tyn Hmm, that sounds complicated. I just track all the mouse movements and keyboard presses and play them back - much less expensive. An entire movie can be reduced to only about 32KB or so in that way.
jeresig 3 months ago
@jeresig i was thinking about recording mouse movements but i also wanted to store information about brush types, eraser and colour. It worked out easier to just encode the entire canvas and break it into chunks, which slightly increases performance. Also when reading it back I just need to store it into an array and link it to a slider for replay functionality. Another idea was to store the UInt8ClampedArray to a file and read it back but only the first row of the canvas was populated...:(
3y3n5tyn 3 months ago
This might works nicely with physics/math teaching as well!
heartnetkung 3 months ago
The ideas are awesome!
On the YouTube hidden in the background part, wouldn't it be more sensible to embed the audio track instead?
Martendemus 3 months ago
@Martendemus Perhaps - although we can kill two birds with one stone here. 1) Not everyone will be able to visit this in a browser and use a code editor - but it might be useful for them to learn anyway (so we can give them a video as a fallback alternative). 2) Since we need the audio track anyway, might as well just use the audio from the YouTube video. I agree that the extra bandwidth usage isn't ideal, but it works :)
jeresig 3 months ago
@jeresig The two birds one stone is a fair argument, but it does feel like a hack to me!
Martendemus 3 months ago
John, this is some great stuff . Keep up the good work.
TheEnzoMar 3 months ago
captioning would be nice too
yuraz1985 3 months ago
@yuraz1985 Thankfully since these are also saved as Youtube videos we'll be able to make use of our existing captioning capabilities - and even support multiple languages as a result (much like what we're doing now with the upcoming Khan Academy iPad application). It may not be in version 1 but it's definitely something that we want to do.
jeresig 3 months ago
@jeresig great. This would be a nice opportunity to support i18n if not with audio but at least with translatable captions.
yuraz1985 3 months ago
This is fantastic - excited to see what Khan Academy does to move the CS section forward.
phenow21 3 months ago