Finally got round to skimming the Media Source Extensions spec; it's starting to make some kind of sense to me. :) https://dvcs.w3.org/hg/html-media/raw-file/tip/media-source/media-source.html

Basically it allows JavaScript to send chunks of a compressed media stream into an actual <video> or <audio> element, as if they had come from a single logical stream. This lets you do clever things like managing streaming and switching between resolutions or bitrates dynamically via JavaScript logic (eg MPEG-DASH implementation), or combining streams together (internet radio usage, inserting ads in YouTube/Hulu, etc).

Unfortunately the current spec doesn't allow you to, say, send raw uncompressed video frames and audio samples into a <video> element -- it wants compressed data chunks in a known format, and the browser implements demuxing and decoding just as it does for conventional streams referenced via an HTTP URL.

Either allowing sending raw uncompressed frames/audio, or creating an interface for pluggable JavaScript codecs could be quite useful -- for instance I could then extend ogv.js to play Ogg media in an actual <video> or <audio> element in Safari and IE instead of manually drawing on a <canvas> and sending sound to an AudioContext or Flash.

Perhaps one day. :)