28 November 2016

Fixing Screen Freeze and Hangs With Web Worker

As web applications become more responsive and advanced today, users may encounter problems from your HTML5 web applications from non-responding script errors to pages that actually hangs forever. Fortunately, thanks to HTML5 web development has given a dose of power that was used to be available to native desktop applications known as the multi-threading.

 Most of my career as a programmer/software developer has revolved around resolving performance issues. I guess there's only a handful in this country who does that. Moving on the topic, multi-threaded programming involves dealing with daemons or background processes or concurrent processes, involves management of buffers, queues asynchronously in order for an application to run smoothly when it's subjected to millions or billions of transactions per second(!).

Introducing the Web Worker, Web Worker is a Javascript API available in all browsers that support HTML5, it works the same way as the Java Thread is mainly used in desktop (SWING, JFC, RCP) and application servers (Apache, JBoss etc). In simple terms, the purpose is to keep the application running smoothly on any given condition. How the Web Worker Works? When an instance of a Web Worker is created, the browser creates a background daemon in order for a web page to run smoothly. One example is when user is querying large chunk of data and at the same scrolling the page, the application must not prevent the user from doing other tasks while the data is loading.

A working sample, while some web developers will dive and start coding on Web Worker API directly, I decided to use an open source web worker queue library from Github https://gist.github.com/kig/1188381 , the main reason for going on this strategy is to simplify handling worker pool or queue that is not readily available in the native HTML Web Worker API.

The Queue Implementation:

WorkCrew = function(filename, count) {
  this.filename = filename;
  this.count = count || 4;
  this.queue = [];
  this.results = [];
  this.pool = [];
  this.working = {};
  this.uuid = 0;
  this.fillPool();
};

WorkCrew.prototype.onfinish = function() {};

WorkCrew.prototype.oncomplete = function(res) {
  return [res.id, res.result];
};

WorkCrew.prototype.addWork = function(work) {
  var id = this.uuid++;
  this.queue.push({id: id, work: work});
  this.processQueue();
  return id;
};

WorkCrew.prototype.processQueue = function() {
  if (this.queue.length == 0 && this.pool.length == this.count) {
    if (this.onfinish)
      this.onfinish();
  } else {
    while (this.queue.length > 0 && this.pool.length > 0) {
      var unit = this.queue.shift();
      var worker = this.pool.shift();
      worker.id = unit.id;
      this.working[worker.id] = worker;
      worker.postMessage(unit.work);
    }
  }
};

WorkCrew.prototype.addWorker = function() {
  var w = new Worker(this.filename);
  var self = this;
  w.onmessage = function(res) {
    var id = this.id;
    delete self.working[this.id];
    this.id = null;
    self.pool.push(this);
    try {
      self.oncomplete({id: id, result: res});
    } catch(e) {
      console.log(e);
    }
    self.processQueue();
  };
  this.pool.push(w);
};

WorkCrew.prototype.fillPool = function() {
  for (var i=0; i    this.addWorker();
  }
};


The Worker Code:

//this where the background process is called that must be in a separate file in this sample we call it //worker.js

self.addEventListener('message', function(e) {
console.log("REMOTE. Received from main script: " + e.data + " -- " + new Date());
self.postMessage(e.data);
}, false);

The Parent client code (this is called from the parent page):

// Create worker pool with 4 workers
 var crew = new WorkCrew("worker.js", 4);

 // Page rendering can only be done here as DOM and parent objects are not accessible
 // inside the Worker thread
 // The result object structure is
 // {
 //   id: work unit ID,
 //   result: message received from worker
 // }
 crew.oncomplete = function(res) {


 // Add some work to the queue.
 // The work unit is postMessaged to one of
 // the workers.

 for (var i = 0;i < someWorkarray.length; i++){
 console.log("Work Crew: Adding work index - " + i + " -- " + new Date());
 var workId = crew.addWork(i);
 }


 // Add an onfinish event handler.
 // Fired when the queue is empty and all workers
 // are free.
 crew.onfinish = function() {
   console.log('Work Crew: All work in queue finished!' + " -- " + new Date());
 };

The code above creates 4 instance of the Web Worker and add multiple work units to the worker it can be hundreds or thousands of tasks but the thing is it will keep your app from freezing, that is if your app really has large amounts of data.  



29 July 2013

Meteor Development Process

Meteor is a really cool framework that runs on top of Node.js, it gives the developer ability to build apps in hours in what usually take days like this Bicycle Alarm App that I build over the weekend. But in a real world nothing is perfect like deploying a Meteor app to a proper host and most Platform-As-A-Service providers doesn't readily support apps built on Meteor so it's really a pain in the b*tt. What could be hours of development fun will be wasted on hours of fixing dependencies and all crap that is related more than the code but rather on the hosting config. So I made a few pointers on how to survive your first flight with Meteor. 1. When deploying your app other than on meteor.com, Demeteorize it with https://github.com/onmodulus/demeteorizer 2. Update the package.json so that the Node.js version is compatible with PaaS hosting's version 3. Make sure your app's root directory includes a Procfile that declares where your main.js is 4. properly set the hosting's environment variables like MONGO_URL, PORT etc. Some of the non-Meteor PaaS I tried with success: Heroku, Modulus, Nodejitsu Enjoy!

04 November 2011

Silicone Valley in the Philippines?

Back track to around 1984. When I was in 6th grade, the first computer magazine I was exposed to is called "BYTE", those thick glossies almost had everything. One issue even had an old "treasure" map of Silicone Valley where Apple, Intel, 3Com(?) etc. are drawn in isometric popup-book style.



During non-school days we visit our other house in Muntinlupa in which where I live now, a half-hour drive from our house in San Andres, Manila via the South Super Highway(yes, those buses you see with "...via SSH" that's where they go, very secure) now known as the South Luzon Expressway (SLEX). Along this 25km journey involved passing by front gates of what made up Silicone Valley back then; Intel's factory was located before stopping at Pasay Road intersection, American Microelectronics along the West Service Road, Motorola where their 2-way radio and the first cellphone was mass produced and then the PowerPC was mass-produced from there too, Panasonic-Technics, Toshiba, Sharp is still there, Amkor Technologies who did a lot for Silicone Valley with a large facility along the East Service now has a second factory at Laguna Technopark. On the software side, I only remember one big building at Faraday Street corner South Super Highway in Palanan, Makati with a big sign outside that said they're always on the hunt for C/C++, DB2 etc. talents. Seeing those companies advertise in BYTE magazine and actually passing by them every now and then reminded me that it doesn't look bad times at all.

My first encounter with a computer was with Sinclair ZX81 it's a microcomputer, not a Personal Computer  it uses casette tape to store data and only understands BASIC. Working with ZX81 cookbook was fun but  I wish I can do something more useful back then like "sari-sari" store inventory(?), and another guy who has a similar device like this one is an old chap with a "startup" called "CompuServe" with his home/office/garage located near Wilson Street in Greenhills. Sinclair is UK-based computer company, not connected to Silicone Valley but one of the early tools that motivated kids like me that somehow we can have something like Silicone Valley.

Sinclair ZX81 with 16KB ROM Pack

Star Trek was not the geek flick of choice around 1984, it's War Games since it's closer to reality, almost life threatening and War on Communism is of paramount importance than War on Terrorism (which was just treated as a petty criminal act those days and not worth putting an army for).

Matthew Broderick in his state-of-the-art Commodore in War Games
So much for the past now. Recently, there's some questions being raised by blogging geeks, tech guys in PHL who were probably born in the 80's or late 70's asking questions why we don't have Silicone Valley in the Philippines? Well, we almost did! We had our shot and we blew it! The bus stopped and we missed it! The Captain called out but we're not ready! We were the China back then until our power generation costs nearly killed us, causing some of the factories to shut down and move elsewhere. The old "evil" regime decided to put up a nuclear power plant to make these big guys stay and keep the money in, but the nuclear dream was fought hard by those who will benefit from it in the future, I'm almost certain that these same people who fought has to push their kids out of the country to work for those companies in similar environmental danger today. The reason it's called Silicone Valley is because it's not about software, there's a place for software and it's called Redmond. We can't be Silicone Valley if we're always paralyzed by power failures both political and electrical. Is Cebu going to be Silicone Valley? Probably, but if the work down there is just about "gluing" things together, then I don't think so.

Will there be a chance to have a Silicone Valley in PHL? Yes, if we stop fighting what's good for us.


26 October 2011

Startup Weekend Manila



I want to check my history first. My first foray in to the startup scene circles back to the first Internet Bubble between '99-'04, there's not much stuff to read about it online and the only two printed media I looked at during those years were Red Herring and Fast Company magazines. I was so amazed by how much money has changed hands with those tissue paper ideas, there were no social apps back then, it was so inspiring that I believe I can build any of those ideas myself and get funded too! But there's seems to be an invisible barrier to entry. Then I searched locally for startup scenes or forums brewing around and I only stumbled in to two; Philippine Venture Capital Group or PhilVenCap and MindShare which was curated by Luli Arroyo. PhilVenCap is a monthly meeting at Asian Institute of Management for early risers in which you can pitch your business ideas for 3 minutes and network aftwards, PhilVenCap has been around for more than 20 years, I did a couple of pitches there and it's really quite a learning experience. MindShare was more technology specific and saw some of the big local names like Ramcar, whom, back then, were attempting to expose themselves to this new "ecosystem", MindShare is slowly making a comeback now. Then comes the SMS generation, not only it's more exciting but applications were easier to develop as well, almost no GUI, everything is done at the backend. Ideas were starting to get crazier, open source APIs were already available and eventually got a chance to work with local startup specializing in SMS with artificial intelligence. And then came the Bubble burst and I had to work with one of the largest telco, yes the brick and mortar, learned a lot on how to deal with them, moved to Singapore for awhile and worked briefly with another startup in which we did touchscreens (even before everyone else got a touchscreen!) for a major telco, yes another brick and mortar.

Fast forward to present, tech entrepreneurs seems to be luckier these days. First, there are more venues like Startup Weekends, RoofCamps, Barcamps, Hackathons and whatnot, although sometimes they were run by the same people over and over again(the risk of losing fresh insights). Second, there isn't much re-inventing to do since startups can reuse most of what is available today in Twitter, Facebook etc. However, old guards still dominates like online shopping sites, online advertising, online gaming, search engines etc. transforming themselves in different mutations, but still, there are valuable lessons to be learned.

And now, the Startup Weekends. The concept of the Startup Weekend is to bring together aspiring tech entrepreneurs, investors, developers and designers in an event where they can pitch ideas, work on those ideas, and eventually launch a startup from those ideas, they may have to build prototypes over the weekend or polish a business plan or both. At the end of the event investors can pick for themselves which startup they think they can put money on. Thus, an idea can become a startup, become a business, who knows. Startup Weekend Manila is my third after Baltimore and Washington, DC. startup weekends, the last two was more like a weekend market while the one we just had is like an American Idol contest. Nevertheless, all of them were successful. Startup Weekend Manila has around 200 participants from what I heard and all were doing social apps one way or another. Well, this is not Silicone Valley.



Many teams were formed after the pitchfire and one of them was VenteeInc. The idea behind VenteeInc is event discovery or it's solving the what's-going-on or who's-going-where or what's-happening-at type of problems for mostly young people in their 20's downward who are really bored and looking for places to go or something to do. It's not rocket science. But again, we might lose the whole point of Startup Weekends, they are not meant to be coding contest of some sorts. Startup Weekends are for getting ideas turn to reality. So I signed up for this team because I think the idea is cool despite doubtful business viability, but of course, theories are almost, always 99% wrong. During the weekend, teams were grilled, ripped and tormented by some mentors to sharpen the ideas further and VenteeInc was no exception we were  at the point that we thought we're screwed but still has to go on and that has been a valuable teamwork lesson learned again and again regardless how silly an idea is. As usual VenteeInc's revenue model (this is what geeks are weak at) revolves around the economics of advertising, revenue-sharing with event organizers, ticketing networks, venue owners etc.

The 'cutters', screw you Terence, I hate that word :P (Photo pulled from Jonathan Richie Yap's Facebook album, me at extreme left, it's 1010 gets?)

The Sunday presentations were awesome, initial ideas has somewhat crystalized with teams who really worked hard with their presentations.

Project KIO, doing the magic

 TwitMusic, it's Twitter for Music

I'm not really a 'contest' or 'competition' guy, it's not my thing and I prefer killing the competition than play with it but clinching the top award together with our team somehow gave me some sort of personal in-your-face vindication from the people now I'm waiting to say 'crab' and points, clearly, went across now.

 VenteeInc, First Place and People's Choice Award

An event like this is not complete without a party and it was there where talks about business is more amplified and that's what we did. Won new networks, won new backdoor deals. Got home San Mig enlightened but wasn't able to sleep yet, Goods2Send has pending orders.

13 October 2011

RIP, Dennis Ritchie



No, it's not Steve Jobs who really changed the world. It's Dennis Ritchie, the man behind the C language and the UNIX operating system. For the non-geeks, here's how UNIX made your life livable everyday; you can sleep at night because opposing forces of the world's military organizations depend on it on a daily basis. Smart, Globe and Sun uses UNIX operating system so you can call, SMS and browse Facebook wherever you are. Your banks keeps your accounts safe with UNIX, your insurance company keeps you updated using UNIX. Steve Jobs is heavily dependent on UNIX from his Pixar gigs to his Macs. The influence of UNIX is widespread, Linus Torvalds' kernel is based on it. If you're reading this now, this blog is hosted in a UNIX platform one way or another. If you're on Twitter, Facebook, G+ etc. etc. you're engulfed by UNIX. When you book a flight to an airline and you check in to an airport, it's all done in UNIX. When you withdraw money from ATMs, it's UNIX. When you buy stuff at Amazon, it's UNIX. When you swipe your credit card at SM, Robinson's or Shopwise, it's UNIX. There's no getting away from it.

For better or for worse, Dennis Ritchie and his work has made a larger impact to mankind more than any keynotes Steve Jobs has ever made.

My early encounters with UNIX as a developer involved its different flavors from System 7, AIX, HP-UX, Solaris and SCO. The UNIX culture shaped the way how we build cool stuff, it taught hard lessons that metrosexual geeks of today might not be able to pick up.

That is our lives with UNIX. Thanks Mr. Ritchie, bad design or good design, your legacy lives on.