DIY – Your Minimal Media Server with Node.js and Content Scripts

This entry  is about a tool that can mine the music blogs for youtube videos, and  create a nice playlist to be played latter on. I call it the Minimal Media Server, because it’s actually a local HTTP server in your computer that you will use to push videos using REST like calls and play them back using a customized youtube player.

I have created this prototype  with Node.js and Content-Script with a few hours of effort.  The source code is available in git,  here  . The code base is actually pretty small ~150 loc !

In order to prevent threats from JavaScript execution, browsers run JavaScript loaded by a web page  in a lower privilege mode. That limits JavaScript’s access only to the web page it is in.  In order to mine the web pages in different blogs,  I needed to run some privileged code on the browser. Such code should execute across all of the pages  I opened in the browser. Google Chrome supports a concept called Content-Script that enables such JavaScript  to run in an elevated privilege in the browser to access and update html content of the loaded web pages. Content-Script is similar to browser extensions in terms of privilege and the script author can specify when to run such script.

I wrote the following content-script to parse youtube videos embedded in the blogs:

var doAjax = function(dataString, onSuccess){
    $.ajax({
        type: "GET",
        url: "http://127.0.0.1:8888/add?",
        data: dataString,
        success: onSuccess
    });
};
var onSuccess = function(response) {
    console.log("added to the playlist");
};

$.each( $("iframe"), function(index, obj){
        var src = $(obj).attr('src');
        if(src.indexOf("youtube.com/embed/") != -1) {
            var values = src.split('/');
            if(values.length == 5) {
                doAjax("id=" + values[4], onSuccess);
            }
        }
});

Typically a video from youtube is embedded inside an iframe by blog sites like blogspot. Hence the script execution starts towards line number 13, from $.each.., that iterates over all iframes in the web page loaded. The typical format of an embedded youtube video URL is, “http://www.youtube.com/embed/%5Bvideo_id%5D“. The script looks for the string “youtube.com/embed”    in the src attribute of the iframe, copies the video_id portion, and, sends it towards the HTTP server that is listening on 127.0.0.1:8888 via an AJAX call.

There are a few points I want to highlight regarding the above script.  First, content-script  works only in a chrome browser. Second, there is a piece of metadata associated with the script to make it  a content script, in a file called “manifest.json”.

{
    "name": "Video parser",
        "description": "Parses videos out of http://www.youtube.com/embed/[video_id]",
        "version": "0.1",
        "permissions": ["contextMenus"],
        "content_security_policy": "default-src 'self'",
        "content_scripts": [
            {
                "matches": ["*://*/*"],
                "js": ["jquery.min.js","parser.js"]
            }
        ]

}

The most important part of the metadata file is the “content_scripts” portion. The “js” attribute indicates which JavaScripts are to be treated as content-scripts.  “parser.js” is the JavaScript described above. Since we are using JQuery inside parser.js we want to load JQuery as content-script as well.  The parser script is executed for each of the webpage loaded in the browser that matches the regex in the “matches” attribute, in this case for all web pages. It can be tweaked to limit to only a few domains. And finally, the content-script can be installed in Chrome by navigating to “chrome://settings/extensions” and then loading the content-script as Unpacked Extension in the developer mode. Here is a screen-shot:

Let’s discuss the server. The server listens to HTTP requests at 127.0.0.1:8888 and it is written using JavaScript, deployed as a Node.js application. Node.js provides several ready-to-deploy modules to develop and deploy client-server applications with minimal effort. It internally uses the Google Chrome’s V8 Javascript Runtime Engine which enables scripting of such applications in JavaScript. Node.js is trending now. There are several cloud vendors (no.de, etc.) who let you deploy node applications on internet.

With the content-script in place, I tried to implement a server that would store the video ids retrived by the script and provide a way to play them back as a playlist. The main server logic is the following piece of code:

var sys = require('util'),
    http = require('http'),
    fs = require('fs');

function readFile(filename) {
    var content = "hello";
    content = fs.readFileSync('./' + filename);
    return content;
}
// Execution starts here
http.createServer(function (request, response) {
  console.log(request.url);
  var result = request.url.match(/^\/(.*\.js)/);
  if(result) {
      // Serve JavaScript
      response.writeHead(200, {'Content-Type': 'text/javascript'});
      response.write(readFile(result[1]));
  } else if(request.url.indexOf("add?&id=") != -1){
      // Process video_ids
      var vals = request.url.split("=");
      console.log(vals[vals.length-1]);
      var playlist = fs.openSync("./playlist.csv", "a+");
      fs.writeSync(playlist, ", '" + vals[vals.length -1 ] + "'");
      fs.close(playlist);
  }
  else{
      // Serve index.html
      var ifile = fs.openSync("./playlist.js", "w");
      var icontent = readFile("./playlist.csv");
      var list = "var playlist = [" + icontent + "];\n";
      console.log(list);
      fs.writeSync(ifile, list);
      fs.close(ifile);
      response.writeHead(200, {'Content-Type': 'text/html'});
      response.write(readFile('index.html'));
  }
  response.end();
}).listen(8888);

console.log('Server running at http://127.0.0.1:8888/');

The server can be started  as “node server.js” from the shell.  The server logic is executed in the callback passed to the createServer method. As a client connects to 127.0.0.1:8888 over HTTP, the callback is executed. In our case, as the content-script parses a video_id, it makes an HTTP GET call to 127.0.0.1:8888. As a result, the callback is invoked, the video_id from the GET request is retrieved, and stored in a file called playlist.csv. On the otherhand, when the user points the browser to http://127.0.0.1:8888/,  the callback serves out the index.html file and a few JavaScript, that load an youtube player to play the videos in the playlist.csv.

Here is the content of the index.html


<script type="text/javascript" src="jquery.min.js"></script>
<script type="text/javascript" src="swfobject.js"></script>
<script type="text/javascript" src="playlist.js">

</script><script type="text/javascript">
var player;
function onYouTubePlayerReady(playerId) {
    console.log("invoked onplayer ready");
    player = document.getElementById("myytplayer");
    player.loadPlaylist(playlist);
}
$(function(){
    var params = { allowScriptAccess: "always" };
    var atts = { id: "myytplayer" };
    swfobject.embedSWF("http://www.youtube.com/v/JzGAmLNwLjw?enablejsapi=1&version=3&playerapiid=ytplayer",
                                   "ytapiplayer", "425", "356", "8", null, null, params, atts);
    $("#prev_button").click(function() {
        if(player) {
          player.previousVideo();
        }
     });
    $("#next_button").click(function() {
        if(player) {
          player.nextVideo();
        }
     });
});

<div id="ytapiplayer">You need Flash player 8+ and JavaScript enabled to view this video.</div>
<div id="controls">
   <input id="prev_button" type="button" value="prev" />
   <input id="next_button" type="button" value="next" />
</div>

The player is loaded using the swjobject wrapper libraries which is found here http://code.google.com/p/swfobject/.

I have been using this setup against http://xonggit.blogspot.com/, one of the music blogs I visit.

In reality, I think the content script can be much smarter to not only recognize  youtube videos, but videos/audios from sites like esnips, vimeo, etc. May be even place a custom button next to a possible link to add an option to add the video to the playlist. The server can act as a proxy to the content sites, and provide an unified interface to the users. The player hosted in the cloud can be wrapped in phonegap for android and ios. Endless possibilities!

About these ads

About Amar Deka

Software Engineer
This entry was posted in Open Source, Technology, Thoughts and tagged , , . Bookmark the permalink.

2 Responses to DIY – Your Minimal Media Server with Node.js and Content Scripts

  1. Pingback: DIY – Your own Rottentomatoes ratings system for Redbox movies | Thought Aftermath

  2. Baptiste says:

    Hi Amar, it’s quite an old topic now but you might consider using “express” to heavily simply your Node.JS code (especially fot HTTP / url parsing). it’s growing pretty fast and getting some very nice repo.
    Nice topic btw, fouded it while looking around for an easy way to stream media from a home made server right into the browser (localy). Node.JS might be a way to go.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s